search

whosonfirst / go-whosonfirst-dist

Go package for working with Who's On First distributions
BSD 3-Clause "New" or "Revised" License
1 stars 1 forks source link

Wrong properties in SQLite export #14

Closed Joxit closed 5 years ago

Joxit commented 5 years ago

I have the same issue as this one https://github.com/whosonfirst/go-whosonfirst-geojson-v2/pull/3 but now for SQLite exports.

In the spr table I have this kind of entries:

id parent_id name placetype country repo latitude longitude min_latitude min_longitude max_latitude max_longitude is_current is_deprecated is_ceased is_superseded is_superseding superseded_by supersedes lastmodified
504792239 -4193636635226208594 multipolygon XX 41.8051579212371 14.4903042439064 41.7596097025985 14.4468571828984 41.8051579212371 14.5337513049144 -1 -1 -1 -1 -1 -1
504792241 -4193572447443142830 multipolygon XX 42.0276133562964 14.4233819568573 41.9975055758807 14.3858960688714 42.0276133562964 14.4608678448433 -1 -1 -1 -1 -1 -1
504792245 -4193608676583427608 multipolygon XX 41.9687105537653 14.3517453074848 41.9231088221748 14.3340265083764 41.9687105537653 14.3694641065933 -1 -1 -1 -1 -1 -1
504792247 -4193575773305076983 multipolygon XX 41.9787497588932 14.6030095606378 41.8239950739279 14.4229290119942 41.9787497588932 14.7830901092815 -1 -1 -1 -1 -1 -1
504792249 -4193561230255624694 multipolygon XX 42.0423777703217 14.6872068870061 41.9999604211155 14.6220947916053 42.0423777703217 14.7523189824068 -1 -1 -1 -1 -1 -1
504792251 -4193569348419122975 multipolygon XX 42.0436589122404 14.5697845201179 41.9970607914715 14.523189156748 42.0436589122404 14.6163798834877 -1 -1 -1 -1 -1 -1

names, placetypes, country... looks wrong

thisisaaronland commented 5 years ago

This is the by-product of an indexing bug that should be fixed later today. Once all the code has been updated we will rebuild all the SQLite distributions.

Apologies for the inconvenience.

Related: https://github.com/whosonfirst/go-whosonfirst-geojson-v2/issues/4

thisisaaronland commented 5 years ago

Okay, this is being caused by a logic error in the go-whosonfirst-sqlite-features-index package that was introduced as part of the work to include "alt" files in distributions:

https://github.com/whosonfirst/go-whosonfirst-sqlite-features-index/blob/master/index.go#L33-L35

I will disable this for the time-being and explicitly disable indexing alt files in this package until suitable LoadWOFFeature methods can be updated to accomodate alt files (or equivalent)

Joxit commented 5 years ago

OK, thanks :-)

Be careful with LoadWOFFeatureFromReader, I tried this in my PR and I got some troubles on a full export... See: https://github.com/whosonfirst/go-whosonfirst-sqlite-features-index/pull/2#issuecomment-523538815

thisisaaronland commented 5 years ago

This should be fixed as of release 0.1.4 but I will leave this issue open until we produce a new batch of SQLite distributions.

$> ./bin/wof-dist-build -workdir ./ whosonfirst-data-postalcode-it
$> bunzip2 whosonfirst-data-postalcode-it-latest.db.bz2
$> sqlite3 whosonfirst-data-postalcode-it-latest.db
SQLite version 3.24.0 2018-06-04 14:10:15
Enter ".help" for usage hints.
sqlite> SELECT id,name,parent_id,placetype FROM spr WHERE id=504792239;
504792239|66045|-1|postalcode
Joxit commented 5 years ago

Okay, thank you @thisisaaronland :smiley: I will build my SQLite tomorrow then

Joxit commented 5 years ago

It seems to work on a full planet build ! :+1: thanks again :smile:

thisisaaronland commented 5 years ago

Excellent, thanks for the report. I will kick off a rebuild of postalcodes shortly.