User:Rupert

I am a stats junkie. I wrote my own Javascript/PHP page that scrapes search results from the HTML as I play Urban Dead.

News

The odds of finding a syringe are back up to 13%. Generators don't make any difference. I added Confidence Intervals to my data. Consider the binomial likelihood function: L = (n choose y) * p^(y) * (1-p)^(n-y), where n is the number of searches, y is the number of successes, p is the predicted search rate and L is the likelihood that the predicted search rate is correct. To compute the 50% confidence interval I throw away the first and last 25% of the area under the curve. This interval contains 50% of the likelihood. If p=10% and the 50% confidence interval is -1/+2 then there is a 50% chance the true search rate is in the interval 9% to 12%. --Rupert 03:02, 12 April 2009 (BST)

Latest

These results were collected starting April '09 or so. --Rupert 02:48, 12 April 2009 (BST)

By Building

building_type	cond	lights	searches	item	count	percent	50% confidence	90% confidence	AP
auto repair	okay	lit	684	nothing	519	75.9%	-1.1/+1.1%	-2.8/+2.6%	1.3
				spray can	40	5.8%	-0.5/+0.7%	-1.3/+1.6%	17.2
				fuel can	40	5.8%	-0.5/+0.7%	-1.3/+1.6%	17.2
				crowbar	34	5.0%	-0.6/+0.6%	-1.3/+1.5%	20.0
				bottle of beer	16	2.3%	-0.3/+0.5%	-0.8/+1.1%	43.5
				fire axe	16	2.3%	-0.3/+0.5%	-0.8/+1.1%	43.5
				toolbox	11	1.6%	-0.3/+0.4%	-0.7/+0.9%	62.5
				newspaper	8	1.2%	-0.3/+0.3%	-0.6/+0.8%	83.3
		dark	273	nothing	215	78.8%	-1.7/+1.6%	-4.3/+3.8%	1.3
				fuel can	16	5.9%	-1.0/+1.0%	-2.1/+2.6%	16.9
				spray can	15	5.5%	-0.9/+1.0%	-2.0/+2.6%	18.2
				crowbar	15	5.5%	-0.9/+1.0%	-2.0/+2.6%	18.2
				newspaper	5	1.8%	-0.5/+0.6%	-1.0/+1.7%	55.6
				toolbox	4	1.5%	-0.5/+0.5%	-0.9/+1.5%	66.7
				fire axe	3	1.1%	-0.4/+0.5%	-0.8/+1.4%	90.9
factory	okay	lit	536	nothing	397	74.1%	-1.3/+1.2%	-3.2/+3.0%	1.3
				length of pipe	37	6.9%	-0.7/+0.8%	-1.7/+2.0%	14.5
				fuel can	27	5.0%	-0.6/+0.7%	-1.4/+1.7%	20.0
				toolbox	23	4.3%	-0.6/+0.6%	-1.3/+1.6%	23.3
				crowbar	19	3.5%	-0.5/+0.6%	-1.1/+1.5%	28.6
				portable generator	18	3.4%	-0.5/+0.5%	-1.2/+1.4%	29.4
				fire axe	15	2.8%	-0.5/+0.5%	-1.0/+1.3%	35.7
hospital	okay	lit	2012	nothing	1505	74.8%	-0.7/+0.7%	-1.6/+1.6%	1.3
				first-aid kit	507	25.2%	-0.7/+0.7%	-1.6/+1.6%	4.0
		dark	1410	nothing	1130	80.1%	-0.7/+0.8%	-1.7/+1.8%	1.2
				first-aid kit	280	19.9%	-0.8/+0.7%	-1.8/+1.7%	5.0
necrotech	okay	lit	488	nothing	375	76.8%	-1.3/+1.3%	-3.2/+3.1%	1.3
				syringe	65	13.3%	-1.0/+1.1%	-2.4/+2.7%	7.5
				DNA extractor	27	5.5%	-0.6/+0.8%	-1.5/+1.9%	18.2
				GPS unit	21	4.3%	-0.6/+0.7%	-1.3/+1.7%	23.3
		dark	458	nothing	365	79.7%	-1.3/+1.2%	-3.2/+3.0%	1.3
				syringe	59	12.9%	-1.0/+1.1%	-2.4/+2.7%	7.8
				GPS unit	19	4.1%	-0.6/+0.7%	-1.3/+1.8%	24.4
				DNA extractor	15	3.3%	-0.6/+0.6%	-1.2/+1.5%	30.3
	ruined	dark	53	nothing	47	88.7%	-3.2/+2.7%	-8.4/+5.8%	1.1
				syringe	4	7.5%	-2.2/+2.7%	-4.5/+7.5%	13.3
				GPS unit	1	1.9%	-1.1/+1.7%	-1.7/+5.2%	52.6
				DNA extractor	1	1.9%	-1.1/+1.7%	-1.7/+5.2%	52.6
police dept	okay	lit	552	nothing	409	74.1%	-1.3/+1.2%	-3.1/+3.0%	1.3
				shotgun shell	43	7.8%	-0.8/+0.8%	-1.8/+2.0%	12.8
				pistol clip	37	6.7%	-0.7/+0.7%	-1.6/+1.9%	14.9
				pistol	16	2.9%	-0.5/+0.5%	-1.0/+1.3%	34.5
				flare gun	14	2.5%	-0.4/+0.5%	-0.9/+1.3%	40.0
				shotgun	12	2.2%	-0.4/+0.4%	-0.9/+1.2%	45.5
				flak jacket	11	2.0%	-0.4/+0.4%	-0.8/+1.1%	50.0
				radio	10	1.8%	-0.3/+0.4%	-0.8/+1.1%	55.6
		dark	519	nothing	421	81.1%	-1.2/+1.2%	-2.9/+2.7%	1.2
				pistol clip	33	6.4%	-0.7/+0.7%	-1.7/+1.9%	15.6
				shotgun shell	26	5.0%	-0.6/+0.7%	-1.4/+1.7%	20.0
				shotgun	9	1.7%	-0.3/+0.5%	-0.8/+1.2%	58.8
				flare gun	9	1.7%	-0.3/+0.5%	-0.8/+1.2%	58.8
				pistol	8	1.5%	-0.3/+0.4%	-0.7/+1.1%	66.7
				radio	7	1.3%	-0.3/+0.4%	-0.6/+1.1%	76.9
				flak jacket	6	1.2%	-0.3/+0.3%	-0.7/+0.9%	83.3
streets	-	-	1	nothing	1	99.9%	-29.3/+0.0%	-68.4/+0.1%	1.0

--Rupert 06:30, 20 April 2009 (BST)

Mall

building_type	cond	lights	bargain	searches	item	count	percent	50% confidence	90% confidence	AP
mall_drug_store	okay	lit	bargain	907	nothing	754	83.1%	-0.8/+0.9%	-2.1/+2.0%	1.2
					first-aid kit	153	16.9%	-0.9/+0.8%	-2.0/+2.1%	5.9
			no bargain	913	nothing	829	90.8%	-0.7/+0.6%	-1.7/+1.5%	1.1
					first-aid kit	84	9.2%	-0.6/+0.7%	-1.5/+1.7%	10.9
mall_gun_store	okay	lit	bargain	208	nothing	108	51.9%	-2.3/+2.3%	-5.7/+5.7%	1.9
					pistol clip	37	17.8%	-1.7/+1.8%	-4.1/+4.6%	5.6
					shotgun shell	33	15.9%	-1.7/+1.7%	-3.9/+4.4%	6.3
					shotgun	18	8.7%	-1.3/+1.3%	-2.9/+3.5%	11.5
					pistol	12	5.8%	-1.1/+1.1%	-2.3/+3.0%	17.2
			no bargain	254	nothing	187	73.6%	-1.9/+1.9%	-4.7/+4.4%	1.4
					pistol clip	23	9.1%	-1.2/+1.2%	-2.7/+3.2%	11.0
					shotgun shell	21	8.3%	-1.1/+1.2%	-2.6/+3.1%	12.0
					shotgun	15	5.9%	-0.9/+1.1%	-2.1/+2.8%	16.9
					pistol	8	3.1%	-0.6/+0.9%	-1.4/+2.2%	32.3
mall_tech_store	okay	lit	no bargain	39	nothing	32	82.1%	-4.4/+3.8%	-11.3/+8.4%	1.2
					radio	3	7.7%	-2.6/+3.2%	-5.1/+9.0%	13.0
					radio transmitter	3	7.7%	-2.6/+3.2%	-5.1/+9.0%	13.0
					mobile phone	1	2.6%	-1.4/+2.2%	-2.4/+6.9%	38.5

--Rupert 06:31, 20 April 2009 (BST)

Result Set 2

User:Rupert/Results2

Result Set 1

User:Rupert/Results1

Methods

I play every page of Urban Dead through a Javascript page on my home computer that I coded up using the Prototype Javascript library (awesome) that uses AJAX HttpRequests to a php proxy (super awesome). That's because Javascript can't contact urbandead.com directly due to the security sandbox, but it can contact php on the same machine and php can contact any website it wishes, so it's more of a two-level proxy, sometimes three. I can then over-analyze the HTML with Javascript till my heart's content, extracting all sorts of information, as well as auto-searching (a fire-and-forget time saver). Javascript is surprisingly good at this and fun to program.

Each search result scraped by Javascript is sent via another AJAX request to a similar php page that stores the data in a MySQL table with these columns:

+--------------------+---------------------+------+-----+---------+-------+
| Field              | Type                | Null | Key | Default | Extra |
+--------------------+---------------------+------+-----+---------+-------+
| name               | char(25)            | NO   |     |         |       |
| date               | date                | YES  |     | NULL    |       |
| building_name      | char(40)            | NO   |     |         |       |
| building_type      | char(20)            | NO   |     |         |       |
| inside             | tinyint(3) unsigned | NO   |     |         |       |
| building_condition | char(10)            | NO   |     |         |       |
| building_light     | tinyint(3) unsigned | NO   |     |         |       |
| has_shopping       | tinyint(3) unsigned | NO   |     |         |       |
| has_bargain        | tinyint(3) unsigned | NO   |     |         |       |
| item_found         | char(40)            | NO   |     |         |       |
| has_necro          | tinyint(3) unsigned | NO   |     | 0       |       |
| has_necro_lab      | tinyint(3) unsigned | NO   |     | 0       |       |
+--------------------+---------------------+------+-----+---------+-------+

When I'm done playing, I run a SQL script that copies that data into a temporary table, transforming all items named "pistol(*)" to just "pistol" because I don't care about the odds of pistol ammo right now, same for "shotgun(*)". Then I transform "NecroTech syringe" and "revivification syringe" into just "syringe", "rusty knife" and "pocket knife" and "hunting knife" into just "knife". Then I count identical rows. This condenses the table from thousands of rows (one per search) to only hundreds, at which point it looks like this:

+-------------+----+-----------+-------+----------+---------+-------+-----+----------------+-------+
| building    | in | condition | light | shopping | bargain | necro | lab | item           | count |
+-------------+----+-----------+-------+----------+---------+-------+-----+----------------+-------+
| auto repair |  1 | okay      |     1 |        0 |       0 |     1 |   1 | bottle of beer |     2 |
| auto repair |  1 | okay      |     1 |        0 |       0 |     1 |   1 | crowbar        |     1 |
| auto repair |  1 | okay      |     1 |        0 |       0 |     1 |   1 | fire axe       |     1 |
| auto repair |  1 | okay      |     1 |        0 |       0 |     1 |   1 | fuel can       |     4 |
| auto repair |  1 | okay      |     1 |        0 |       0 |     1 |   1 | nothing        |    34 |
| auto repair |  1 | okay      |     1 |        0 |       0 |     1 |   1 | spray can      |     3 |
| auto repair |  1 | okay      |     1 |        1 |       0 |     0 |   0 | bottle of beer |     1 |
| auto repair |  1 | okay      |     1 |        1 |       0 |     0 |   0 | crowbar        |     3 |
| auto repair |  1 | okay      |     1 |        1 |       0 |     0 |   0 | fire axe       |     1 |
| auto repair |  1 | okay      |     1 |        1 |       0 |     0 |   0 | fuel can       |     2 |
+-------------+----+-----------+-------+----------+---------+-------+-----+----------------+-------+

Then the script builds up a second temporary table populated by three queries on the first temp table:

1. all non-mall and non-necrotech rows grouped by building type, light, and item
2. all malls and mall substores further grouped by has_shopping and has_bargain
3. all necrotechs grouped by the same as (1) but also by has_necro and has_lab (because the odds appear to be different)

Then I order the table by building and condition to get the "By Building" table, and by item for the "By Item" table. Then I execute a php page to get the html tables to paste into this lovely wiki.