LocalData / localdata-tiles

A tileserver for LocalData
5 stars 2 forks source link

[Draft] Add support for rendering features #144

Closed hampelm closed 9 years ago

hampelm commented 9 years ago

This adds support for rendering and caching tiles based on data in the features table. @prashtx, It's ready for your thoughts on implementation, but not yet ready for a merge.

I have a sample page that shows how a client would use it: https://gist.github.com/hampelm/29b3437c96954718ab30

A note on caching: the layerDef can (and should) specify query params like feature type (parcels, buildings, lighting, etc). However, the cache doesn't use those for validation; it just does a straight count or last modified. That overly-aggressively invalidates tiles but saves a query for the layerDef and some code complexity. In the interest of shipping, I'm leaning towards keeping it that way, and filing a bug for future optimization.

I also need to condense the commits and do some minor cleanup (eg remove files where I just changed whitespace),

hampelm commented 9 years ago

I'd like to include geometries in the grids past a certain zoom level, but haven't found the best way to do it. It'll happen in ds-features but I need to flag when they should be included, and re-project the geomtries (we discard the original geometries for speed)

hampelm commented 9 years ago

This also doesn't implement the mongo caching strategy. I also think that's OK for a more speculative feature.

prashtx commented 9 years ago

We should look at the timing metrics and make sure we're not combining two things into the same metric in a way that prevents useful analysis later. For example, we almost certainly want to see postgres load times separate from mongodb load times. Because it's difficult to separate the render time from the render+load time in the code, we most easily get the former by just subtracting avg load time from avg total time. We might want to track fully separate metrics for the postgres feature pipeline, so we can do that post facto math more easily.

prashtx commented 9 years ago

It'd ideally be nice to factor out the S3/knox putBuffer error handling into a promise-returning function. We'd probably need to wrap the underlying errors or failure scenarios (non-200 status). Then the catch function will mostly be some logic to decide what to log, which is less fragile than deciding that we have indeed encountered an error and passing control to the appropriate continuation.

prashtx commented 9 years ago

Awesome. Use your judgement on the various comments. Some of them we can work on as we start to actually use the code.

hampelm commented 9 years ago

Cleaned up in #147