search

CartoDB / cartodb

Location Intelligence & Data Visualization tool
http://carto.com
BSD 3-Clause "New" or "Revised" License
2.75k stars 650 forks source link

Data points are missing at particular zoom layers #131

Closed zqshen closed 11 years ago

zqshen commented 11 years ago

A lot of data points are not displayed. I waited until the browser finished loading, and the server also finished querying. But those points are still not displayed. When I zoom in one more level, the points show up. It is always the same part of data missing at a particular zoom level. I did not see any errors in the cartodb log or the browser javascript console.

cartodb_issue

The data was recently imported into my cartodb instance. It worked fine on another cartodb instance before. The other older data in my cartodb instance all works fine. Could some errors happen at the importing step and cause this issue?

What could be the issue and where should I look into? Please help. Thanks

BTW, I posted the same question at the google group earlier. But no replies yet. . Sorry for re-posting here. I just really need some advice on this.

strk commented 11 years ago

Can you provide the shapefile ?

zqshen commented 11 years ago

@strk how do I get the shapefile? I tried export SHP, but I got the webpage not found error... other exports work.

mattvoss commented 11 years ago

@strk I've seen the same thing, as @zqshen, on my own install. A good set of shapefiles to try this on is the Census 2010 Block level shapefiles. You can get them here: http://www2.census.gov/geo/tiger/TIGERrd13/TABBLOCK10/

You have to get all the state files into one table, which I've done, but it's like 13GB so it's kind of large to export out.

However once done it looks like Windshaft is not able to render all the data at a particular zoom level and once it fails to render the data it seems to cache it. You can drill down to higher zoom levels and the data is rendered properly so it is in the table properly.

If I can get my implementation onto a publicly available server I can get a link to share to illustrate this issue.

strk commented 11 years ago

There's a limit on the rows to be fetched, see postgres.row_limit in the environment configuration, clean redis map_style* keys and restart tiler to see how/if things change.

strk commented 11 years ago

Another possible reason, if geoms are polygons, could be that line generalization makes those polygons collapse into nulls. That's also an env config setting (simplify_geometries).

zqshen commented 11 years ago

Changing the postgres.row_limit seems to work for me! It is weird, because some of my larger data sets work totally fine.

I still have question about how to do "clean redis map_style*keys". I duplicated the table instead. But for some larger tables, I hope I can do this. @strk, Could you give more details about it? Thanks!

strk commented 11 years ago

the effect of row limit depends on the distribution of rows. if the rows are spatially distributed you may not notice the limit. It also depends on zoom level as the limit is per-tile. Changing the style would be an effective way to clean the redis key. Or you've to send redis commands to the server (maybe with redis-cli), see del command on http://redis.io