search

asah / footprint2009dev

original dev repo for AllForGood.org
http://AllForGood.org/
0 stars 1 forks source link

important searchqual issue: automatically set vol_dist depending on vol_loc #45

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
symptoms:
 - searching where vol_loc=crookston,MN - no listings
 - searching where vol_loc=16th and market st,SF,CA - too many listings

root cause:
In the US, there's wildly varying population density, which therefore
implies varying density of listings.  This issue is to create a map (cached
in RAM, read from datastore) of lat/long regions (rectangles) and pop
densities and automatically set vol_dist depending on vol_loc i.e. default
value of vol_dist is determined by centerpoint.

the obvious solution: in the FP API (not in the FP consumer app!!!), set a
default vol_dist based on the vol_loc.  The obvious algorithm is to lookup
the vol_loc (its lat/long, i.e. after geocoding) against a table of
rectangles and pre-computed vol_dist's to use-- 100 miles for crookston, 1
mile for SF.  Presumably, density is inversely proportional to vol_dist.

then I see two obvious ways to go with vol_dist:
1. use census-like data to select the density.
2. use our actual listings database to select the density.
#2 is the most elegant, but requires some jujitsu, since FP doesn't have
an easy equivalent of mapreduce, and indeed, the listings feeds are
independently crawled and loaded into Base, which itself doesn't offer an
easy (external) way to scan the data.

recommend: wait to see if we stick with Base.  If not, consider using the
appengine datastore + our planned backup facility to get the data, then
scan it once on the machine used for feed processing (small dataset = trivial).

Original issue reported on code.google.com by adam.sah on 3 Apr 2009 at 10:00

GoogleCodeExporter commented 9 years ago

Original comment by manz...@gmail.com on 3 Apr 2009 at 10:14

GoogleCodeExporter commented 9 years ago 
nice to have + anybody can implement it.  IMHO green light if the eng resources
exist, but don't distract core eng.  i.e. perfect 20% project.

Original comment by adam.sah on 14 Apr 2009 at 10:21

GoogleCodeExporter commented 9 years ago

Original comment by adam.sah on 17 May 2009 at 12:15

GoogleCodeExporter commented 9 years ago

Original comment by adam.sah on 19 May 2009 at 6:49

GoogleCodeExporter commented 9 years ago

Original comment by adam.sah on 20 May 2009 at 5:03

GoogleCodeExporter commented 9 years ago

Original comment by adam.sah on 20 May 2009 at 5:13

GoogleCodeExporter commented 9 years ago

Original comment by adam.sah on 26 May 2009 at 5:29

GoogleCodeExporter commented 9 years ago

Original comment by adam.sah on 11 Jun 2009 at 7:52

GoogleCodeExporter commented 9 years ago 
adding mark to featreqs

Original comment by adam.sah on 23 Jun 2009 at 7:10

GoogleCodeExporter commented 9 years ago

Original comment by adam.sah on 25 Jun 2009 at 6:03

GoogleCodeExporter commented 9 years ago

Original comment by adam.sah on 28 Jun 2009 at 4:25