Open simonw opened 6 years ago
Any chance we could get this done? Anything I can do to help?
There is a lot of work related to building distributions and doing indexing using ECS happening in a another project that we are hoping to build on (here) soon.
Once it has then it becomes much easier to think about variations on the build process. I don't have a firm ETA yet.
In the meantime if you want to keep the ball rolling then either spec-ing out the interfaces (command line, packages, etc.) and/or starting a branch of wof-dist-build
would be great.
+1 for this.
Also +1-ing
The thing I'd really love to see is a weekly or monthly database that contains just the records that were updated in the past 30 days. This would be extremely useful for keeping my own copy (in this case it's a custom schema in MySQL which I populate from the Who's On First SQLite DB) up-to-date without having to pull the full 13GB file.
For example,
https://dist.whosonfirst.org/sqlite/whosonfirst-data-changed-in-july-2018.db.bz2
would be a SQLite database with exactly the same format as the giant 13TB one but that only includes the geojson records that were updated at least once during the month of July 2018.This could be weekly, monthly or something else, I don't really have a strong opinion.