User talk:Aichon

Aichon:Talk

• Main
• Characters
• Sandbox

• Talk
• Userscripts
• Other

Aichon

ˈīˌkän :Talk

Announcement: I'm no longer active. My talk page is still your best bet to get in touch. —Aichon— 04:39, 15 December 2018 (UTC)

Please be aware of the following guidelines before posting here.

• New conversations should be started at the bottom using a level two header (e.g. ==Header==). Or with the +
• I like to keep conversations wherever they start, but if a conversation ends up here, I will keep it here.
• I will format comments for stylistic reasons, delete comments for whatever reason, and generally do anything else within reason.

Thanks. —Aichon—

NOTICE

Archives: 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010 | 2009 | To do

Editting User Pages

I was looking around at all the pages that call on the 5000 or so danger report templates, and I noticed that a lot of them were userpages. Since we're experiencing so much lag these days when the danger reports get updated, I was wondering if I could delete or disable the superfluous ones? Putting nowiki tags on them, with a little explanation that we disabled it because of performance issues and because they were unused. Problem is: they still are other peoples' user pages and I'm not sure what the wikipoliticians and wikilawyers would say about this, so any advice? PB&J 14:51, 18 June 2013 (BST)

The lag isn't from them. The lag is from the Danger Center needing to re-grab all 5000 templates every single time a single one of them changes, since the cache for the page needs to be recreated. Right now, updating a danger report might kick off, say, 5000 updates for the Danger Center and another 10 updates for the other pages mentioning that danger report, giving us a total of 5010 updates to do. Quite obviously, removing the 10 won't have a significant impact on performance. —Aichon— 15:11, 18 June 2013 (BST)

Quite a lot of the pages I'm talking about are like this one, and call on about 50 everytime a single one of them changes. If we look at the "What links here" of the first one in the topleft corner, Special:WhatLinksHere/User:DangerReport/Palprey_Road_Police_Department, one which is among the lesser used ones and lesser known ones, we see that several of those pages call on dozens of templates, including used, unused and personal test status maps, as well as specific building status maps (in this case the PD Status Map. That could amount to quite a lot of pages, each calling on a couple dozen templates, just for one danger report update, no? PB&J 15:18, 18 June 2013 (BST)

In this case it would make no difference because it's a by page thing and not a by call thing. --Karek^{maps 2.0?!} 16:50, 18 June 2013 (BST)

I still don't really understand? The lag increased when we added the danger center, and it occurs more when updating the danger reports than when you look at the center itself. AFAIK the amount of times a danger report is called upon has a direct influence on that lag, ergo reducing the amount of maps that call upon a danger report should reduce the overall lag. Right? PB&J 16:59, 18 June 2013 (BST)

What Karek is saying (and what I was typing up before I saw that he had already linked exactly what I was going to say) is that the Danger Center is a gigantic job, whereas the other pages are not. The reason that matters, is because the wiki is configured to process one job in the queue per page load. As such, those smaller pages are not even getting processed at the same time as the Danger Center and are thus in no way contributing to the unresponsiveness that people are suffering. To draw an imperfect analogy, the job queue is like a relay race where every time the baton is passed a page gets loaded for a user. In this race, some runners (jobs) are slower than others. Quite obviously, you can improve the speed of the faster runners all you want, but you'll still be waiting the exact same amount of time for the slower ones to complete their laps and pass the baton. So, more or less, those smaller pages aren't contributing to the problem at all, since the unresponsiveness they create is separate from what people are complaining about and is small enough that most of us have never noticed it at all. —Aichon— 18:39, 18 June 2013 (BST)

Technical gobbledegook-> Yeah, sorta this. Basically everytime you load a page the job queue processes 1 job. Everytime you have a template on a page it loads a function and then on repeat calls of the template it just references the already loaded function. Different templates do have different costs but that's based on the size of the initial template, not so much the number of times it's called(which has a much smaller impact). Everytime you call a unique template the server has to go and make a new database connection and call that template's page content to then queue it up for the preprocessor/job queue. Batch processing it with the job queue means instead of having a half hour of horrific lag(where you have thousands of database connections and nothing can load) you queue up the change and then half multiple hours of much more negligbile lag(where you just double the normal page loads). Super simple version is instead of having 5000 pages load when you edit danger reports it makes it so that for the next 5000 pages loaded you're basically loading 2 pages. --Karek^{maps 2.0?!} 18:49, 18 June 2013 (BST)

It sounds like you may have accidentally said the opposite of what you meant. Just to double check, as you said above, the queue adds jobs on a per-page basis, which means that the Danger Center is processed as a single job, just as each of those other pages is also processed as a single job (though clearly the Danger Center is a larger job). When a user visits a page, the wiki does the job at the front of the queue before it delivers the page to the user. When a danger report gets updated, each page that transcludes that danger report will receive a corresponding job to update the page. For those smaller pages, because they each have their own job, they get spread out over a number of page loads and cause no noticeable delay. For the Danger Center, it too is done as a single job attached to a single page load, but because it's a much larger job it creates an extremely noticeable delay (i.e. it does the 5000+ db operations you're talking about). I think that's what you meant to say, but please correct me if I'm mistaken. —Aichon— 19:55, 18 June 2013 (BST)

A lot of this is going to be process descriptive but not exact to how the code runs. It functions like a php include that contains a class with only a single method that's being batch processed. What happens is essentially when you update a template it checks to see how many rows return in the table(see picture below) of transclusions. It then adds those pages(where the template is transcluded) to the jobs queue if there are more than a certain number.

The jobs queue then runs a preprocessor on each of those individual pages. The preprocessor runs through the page content sequentially until it finds text matching what it sees as a template call. It then runs that call through a check against all other held templates/functions already preprocessed, if it returns that none are already stored it queues up a database call to the page(usually a template) being transcluded. It searches for include only tags then for noinclude tags to determine content and then converts the uneliminated text into XML Trees and stores this as a callable variable to be referenced when the same template is called later in the text. That means it only calls the template from the database once per page instead of however many times the template is included in the page.

Once the preprocessor sets limits on the generated text size for the final result and stops converting template content after a certain point. There are a number of reasons for this but the biggest one I can imagine would relate to the topic of this white paper microsoft released a few years back(pages are stored in blobs) about large file vs blob performance. The end result all then gets thrown through the parser when the page is called which converts wikitext like to it's output state.

My understanding is thus that when a page does not hit the rows returned in the 'Pages Included On' check it runs the preprocessor in reverse, thus processing all of the pages then and there making x calls where x is the amount of pages it's included on. When it determines that this would be too much strain for the server based on the limits set it instead queues them up for batch processing so that the calls are processed at a set interval and not all at once. --Karek^{maps 2.0?!} 03:07, 19 June 2013 (BST)

I love the technical description (honestly, I do), but that's actually tangential to what I was asking. You already explained the details below, and I thought it was well understood that that's what the function of the job queue is, since it's basically just "baking" elements for later use. What I was talking about was specifically the un-struck line you left, since the problem we're having is that it's not being broken up into 5000 jobs, which is what it sounds like you were suggesting is happening. That's why I said you sounded like you were saying the opposite of what you meant. ;) —Aichon— 05:34, 19 June 2013 (BST)

I was insanely oversimplifying, the default value is 500 operations and each job itself is 500 operations. So with a current job queue of 6 that's roughly 3000 transclusions. Basically it is just the numbers are obscured, additionally Kevan might have changed it but it's unlikely. --Karek^{maps 2.0?!} 06:16, 19 June 2013 (BST)

This may help some. The Job Queue, from my understanding, basically functions like the preprocessor for templates so that the stored wiki text can be served faster on significantly large template calls to minimize load time. Templates are thrown into the job queue based on how many pages need to be preprocessed and it converts them into wiki-text/html there. Because job queue processing isn't done through a cron or based on server load it's happenning while other things are being done on the server, so lag will increase be it 1 template call on a page or 10 as it's running the same script either way. As far as impact on performance, because there's no additional complexity to the call(you should only have to queue up the template's content once regardless) it's of neglible impact of processing speed. You get a bigger impact from an increase in unique templates as 0-1 is where you're having to open a new file stream or make a new databse connection(which is likely the cause of the lag). Preprocessor ref. This is also one of many reasons templated signatures suck, they slow down the wiki for everyone, particularly more so the more pages they're on. --Karek^{maps 2.0?!} 18:43, 18 June 2013 (BST)

Too bad, I really hoped to optimize this whole thing... Then again: the danger reports are updated regularly these days, even more with BB4, and that was always one of the goals :) Maybe I'll go on a little wiki-patrol later today and just list a ton of stuff that can be deleted or otherwise archived to clean the whole wiki a bit out. PB&J 12:54, 19 June 2013 (BST)

If you want to optimize these pages you could make the core templates use non-wikicode code, here's a list m:Help:HTML_in_wikitext, it'll reduce the work done when the parser runs. --Karek^{maps 2.0?!} 22:12, 19 June 2013 (BST)

Mob Locator Map Style Practice

Hi Aichon, you should update the comment on http://wiki.urbandead.com/index.php?title=Militant_Order_of_Barhah/Locator&action=edit&section=3 to reflect the actual practice which is used to colour the map. greetings Ja7 13:09, 25 June 2013 (BST)

Sorry, I got it wrong. See my edits to MOB Locator. —The preceding unsigned comment was added by Ja7 (talk • contribs) 13:12, 25 June 2013.

Yup, looks like you got it figured out, though I really need to update the Locator since it's several suburbs out of date at the moment. —Aichon— 15:46, 25 June 2013 (BST)

A/SA questions

I'm planning to start moving the A/PM archives into A/SA format, and I had a couple questions.

1. What do we do about awkward redlinking of archives for those who were never sysops? Example - "Akule" is redlinked in the breadcrumbs.
2. I'm planning to convert the A/PM navtemplates so they can be kept in one location, and am adding a key to make them more detailed. Would something like that (or which already exists for A/BP) be useful for A/RE and A/M as well? (Not to be included on each page, but be in one location for cross-referencing.)

Thanks! Bob Moncrief ^EBD•W! 16:40, 18 August 2013 (BST)

There's nothing saying we can't make pages for the non-sysops too, and with your changes to put it all in tables, it seems like the concerns we have about it taking up too much space have been alleviated pretty well. I'm thinking give them their own page, and kill two birds (the red-link and the fact that the table will get bulky as we add data) with one stone.

As for the key, I kinda feel like the improved categorization is designed to remove the need for keys and color-coding sort of stuff, though the categories could definitely be better-linked from A/SA in order to make them more useful. —Aichon— 21:50, 18 August 2013 (BST)

Ok! I can make pages for them easily. I'm gonna stick with making the key for now, mostly as a way for me to keep track of which ones I've moved so far; it can always be easily de-keyed later. Bob Moncrief ^EBD•W! 01:08, 19 August 2013 (BST)

IRC

Can you jump on when you've got a minute, please? --BOSCH 20:50, 20 August 2013 (BST)

Done. —Aichon— 20:58, 20 August 2013 (BST)

A/D

As a note, Revenant's revelation would be what would normally be a checkuser policy allowance. In this case he actually would probably be one of the better authorities on this since he can potentially compare it to one of the various RG boards. Odds are this is another Nubis type situtation in which case you guys should lock the account in question and send an email via the wiki, if possible, letting him know what has happened, why, and how he can contact one of you guys to resolve it. --Karek^{maps 2.0?!} 04:48, 26 August 2013 (BST)