Tiling of results for map view

[datadavev]

Rendering of all individual markers in a result set is problematic in the map view when there's more than a thousand or so markers to render.

One approach is to tile the results and indicate the counts per tile by coloring the tile according to some scale.

A more complicated approach would be to do a of recursive operation, starting off with very course tiling to provide an initial view, progressively refining the tiles (splitting each coarse tile into a finer set of tiles) while the user stays on the view.

Initial implementation can do something like take the full range of the results, then split that into say 5x5 tiles, and compute the counts for each of those 25 tiles.

[datadavev]

somewhat related to this - I found a trick that enables SOLR to randomly order the search results and have implemented this change to the darwin core records SOLR server. To retrieve results in random order, sort on the field "random_n" where n is any integer. Using the same value of n will return the results in the same order. For example, append to the request:

&sort=random_1%20asc

Will always return the same set of randomly selected points. Replacing "1" with another integer or actually any other string will return a different set of randomly selected records.

[coreyo]

Good Catch Dave! I myself fantasized about such a function, but assumed that it was too unlikely to actually exist. I have updated the map's searching so that it uses "random_1" as the sort field. I also added some appropriate filters to make sure that only records with valid lng/lat values will ever be returned from the query. In addition, the console log will now print the URL for all searches (this includes the RecordsView and MapView). The random function is certainly a step up, but not perfect. Please play around with it and let me know if you still want me to work on loading the initial view (or any view with more than the max query) by splitting it up into tiles.

Everything is up in the sandbox.
~ Corey O>

On Fri, Jul 29, 2011 at 10:46 AM, vdave < reply@reply.github.com>wrote:

somewhat related to this - I found a trick that enables SOLR to randomly order the search results and have implemented this change to the darwin core records SOLR server. To retrieve results in random order, sort on the field "random_n" where n is any integer. Using the same value of n will return the results in the same order. For example, append to the request:

&sort=random_1%20asc

Will always return the same set of randomly selected points. Replacing "1" with another integer or actually any other string will return a different set of randomly selected records.

Reply to this email directly or view it on GitHub: https://github.com/vdave/DwC_views/issues/41#issuecomment-1681746

[datadavev]

Let's bump up the number of results a bit and see how it goes. I did a little test of the gateway, and it was able to consistently return 20,000 points, so pushing it to 3000 instead of 1000 displayed might provide a pretty decent rendering when the points are selected randomly.

simple test code:

import urllib2, json
npoints=[100,1000,10000,20000,40000,60000,100000]
for np in npoints:
  url="http://coreyosandbox.appspot.com/gateway/records?filter=*:*&count=%d&fields=id%%2Clat%%2Clng%%2CsciName_s&orderby=random_1" % np
  f = urllib2.urlopen(url)
  o = json.loads(f.read())
  print len(o['docs'])

[coreyo]

We are going to have 2 tiling implementations: Tiling of markers, and actual rectangular tiles. For further discussion, this ticket is going to refer to tiling of markers. Ticket #47 will address the rectangular tiling.

[coreyo]

Increasing the number of points seems to be more of a problem with the client than with the solr server. Upping the number of markers above, say, 2000 makes the map slow and unresponsive ... almost unusable from a panning standpoint. Behavior appears even worse when tiling is turned on. I think that this is due to the resources required to display and redraw the markers on the client. When markers are turned off, the bad behavior stops.

[coreyo]

I believe that current implementation of record result tiling is sufficient to close this ticket. All further, specific issues should be logged as individual issues or bugs.