search

koopjs / koop

Transform, query, and download geospatial data on the web.
http://koopjs.github.io
Other
675 stars 129 forks source link

Caching is too aggressive #67

Closed normanb closed 9 years ago

normanb commented 10 years ago

Caching seems agressive. Looking at AGOL.js (which is the file the Cloudant provider is following) caching is aggressive. Is this because paging is not supported by the GeoServices API (if it is, how?!).

The Cloudant driver will have data that is too large to fit on a single node so paging needs to be supported and caching needs have a limit on total size.

For now, if there is a way to add paging to the geoservices API that will be a start and I can add something like CacheManager so that we have an eviction policy for cached data.

chelm commented 10 years ago

Actually. I'm actively rewriting the AGOL provider and its a bad one to follow because the GeoServices REST API can be confusing and there is a ton of horrid logic in the koop code.

@normanb look at the Socrata Provider. It is MUCH cleaner and has a "check cache" function that is used to invalidate/expire the cache. The gist and github provider implement the same method too. The AGOL one doesn't yet, which I'm adding this week.

normanb commented 10 years ago

I followed the AGOL provider because Cloudant supports the geometry query. I will see what I can do with the Socrata driver, this is fun :)

chelm commented 10 years ago

@normanb Great! Yes use the Socrata one for now. You can easily pass the geometry query param from the contoller down to the model (and use it to fetch data).

So stoked to see what you're up to.