search

gregrs-uk / fhodot

Food hygiene open data OpenStreetMap tool
https://gregrs.dev.openstreetmap.org/fhodot/
GNU General Public License v3.0
9 stars 3 forks source link

Update more frequently #11

Open Firefishy opened 3 years ago

Firefishy commented 3 years ago

Updates are currently daily, but it would be great if the updates we more frequent.

gregrs-uk commented 3 years ago

Thanks @Firefishy, that would certainly be useful.

I'm currently using the daily Britain and Ireland extract from Geofabrik and filtering using a polygon of the UK. In theory imposm supports the use of diffs, but I won't have time in the near future to look into this in more detail.

jnicho02 commented 3 years ago

Connected to the Britain and Ireland extract: I was talking to Frederik a while back and he would be open to providing a UK extract (British mainland + Northern Ireland + Isle of Man + Channel Islands) because I was building OSMUK-in-a-box https://github.com/osm-uk/osmuk2pgsql

I know its not a huge step to simply delete Eire, but could bring it up again.

gregrs-uk commented 3 years ago

Thanks @jnicho02. As you say, not a huge step to delete Eire each time, but if he does happen to provide a UK extract do let me know.

(In addition to the areas you've mentioned, to be used for this tool the extract would need to cover other parts of the UK such as the Western Isles, Shetland Islands, etc.)

Firefishy commented 3 years ago

https://download.openstreetmap.fr/ offers UK extract minutely diffs.

Firefishy commented 3 years ago

Sudo code example

wget https://download.openstreetmap.fr/extracts/europe/united_kingdom-latest.osm.pbf
pyosmium-up-to-date -vv united_kingdom-latest.osm.pbf
# Done. Updated to latest minutely.
gregrs-uk commented 2 years ago

Coming back to this much later, a note (mainly to self) that currently post_import.sql runs after the imposm import, creating indices, pre-calculating centroids and creating an association table between OSM objects and FHRS establishments. This would probably preclude minutely updates, but not necessarily updates more frequent than the current daily ones.

jnicho02 commented 9 months ago

Frederik has now created that https://download.geofabrik.de/europe/united-kingdom.html download

gregrs-uk commented 9 months ago

Thanks @jnicho02. Now using that UK download (https://github.com/gregrs-uk/fhodot/commit/1b6aed2090b9c67e78a6a2c7e39efcce7474efc3), especially as the GB & Ireland one wasn't being updated.