Closed gregallensworth closed 5 years ago
Collecting some URLs and notes for reference:
The import-osmborder
image, does not specify to use the start_date
and end_date
fields.
https://github.com/OpenHistoricalMap/import-osmborder/blob/master/import_osmborder_lines.sh#L18-L21
Not that the generalized tables would also need modification to include the new fields: https://github.com/OpenHistoricalMap/import-osmborder/blob/master/import_osmborder_lines.sh#L40
The source CSV used by import-osmborder
is generated by the makecsv-osmborder
command, which in turn runs https://github.com/OpenHistoricalMap/osmborder-docker/blob/master/build-osm-borders.sh to download a ohmplanet$DATE.osm.bz2
I do already have a ohm_planet.osm.bz2 dated August 2, and the resulting osmborder_lines.csv. The CSV does seem to have only the 6 fields specified in https://github.com/OpenHistoricalMap/import-osmborder/blob/master/ and no more than that. So this would be something further back in osmborder
Judging by the docs at https://github.com/pnorman/osmborder and the source code https://github.com/pnorman/osmborder/blob/b421a906de8d2fd0a5637775d8c1e480658d2111/src/adminhandler.hpp#L113-L165 it looks like the osmborder tool does specifically fetch these fields and only these, and would need modification and compile-from-source in order to add the start_date
and end_date
fields to the output CSV.
Tasks:
[x] fork and modify the osmborder tool, to include OHM's fields; should also add the name
field while we're at it
[x] alter the makecsv-osmborder
docker, to use this new version of osmborder and confirm, that the output CSVs are formatted as expected
[x] alter the import-osmborder
docker, to include these 3 new fields, re-run to check
[x] set up scheduled task to run this whole series every few hours; this chain of tools doesn't use imposm and its diff-consuming capability, so looks as if it'll be a drop-and-load?
I have modified the tools and re-run these two steps, and the osm_border_linestring
tables now properly have start_date
, end_date
, and name
fields. I have modified the OpenMapTiles CREATE VIEW
statements as well, to expose these new fields into vector tiles.
Looking good so far:
I do see some testing and improvements to the dateissues map, but good news is that boundaries now have dates and names.
There is now a scheduled task in place, which runs the modified makecsv-osmborder
and import-osmborder
from the OHM planet file, then re-creates the boundary_zX
views for use in vector tiles.
Created in response to a comment https://github.com/OpenHistoricalMap/openhistoricaltiles/issues/3#issuecomment-418573034 that this shape is not showing its
start_date
in the dateissues map.http://www.openhistoricalmap.org/way/198180496 https://openhistoricalmap.github.io/openhistoricaltiles/dateissues/#7.38/44.91845/-69.47637/
Confirmed here, that boundaries lack any date info. I have comments in here, that boundaries seem to be imported via
import-osmborder
which was a separate process (and one which may be separate from the later diff-consuming process???), and that these boundaries lacked date information in theosm_border_linestring
table and its derivatives.