calvinmetcalf / shapefile-js

Convert a Shapefile to GeoJSON. Not many caveats.
http://calvinmetcalf.github.io/shapefile-js/
715 stars 228 forks source link

Invalid dbf file causes incorrect data to be linked to GeoJSON output #134

Closed tehnrd closed 3 years ago

tehnrd commented 3 years ago

First, I don't necessarily think this is an issue with this library, but it would be great if this library would throw an exception in the situation in which there are more dbf entries than shape features. Yes, this is a bad/invalid shapefile but we don't have control over the source. There might actually be shape features but they have no coordinates/points.

Attached is a very small shapefile to reproduce the issue. badshp.zip

If you take that file and drop it here, http://leaflet.calvinmetcalf.com/#6/-33.596/138.076, you will see it cover Kangaroo Island and this is the correct location. If you click on the shape to display the info window you will see it has the incorrect data and the name is "No usual address" which is coming from the first row in the dbf file.

Screen Shot 2020-08-12 at 10 26 49 PM

Interestingly, if I load this in Qgis and select the feature, it shows the correct information.

Screen Shot 2020-08-12 at 10 28 16 PM

If this library was able to work like Qgis, great, but at the very least it should probably throw an exception as this can result in very corrupt data outputs.

calvinmetcalf commented 3 years ago

Hm I guess there might be id's I can check instead of just going row by row

On Thu, Aug 13, 2020, 1:32 AM Jason Venable notifications@github.com wrote:

First, I don't necessarily think this is an issue with this library, but it would be great if this library would throw an exception in the situation in which there are more dbf entries than shape features. Yes, this is a bad/invalid shapefile but we don't have control over the source https://www.abs.gov.au/AUSSTATS/abs@.nsf/DetailsPage/1270.0.55.003July%202013?OpenDocument .

Attached is a very small shapefile to reproduce the issue. badshp.zip https://github.com/calvinmetcalf/shapefile-js/files/5067255/badshp.zip

If you take that file and drop it here, http://leaflet.calvinmetcalf.com/#6/-33.596/138.076, you will see it cover Kangaroo Island and this is the correct location. If you click on the shape to display the info window you will see it has the incorrect data and the name is "No usual address" which is coming from the first row in the dbf file. [image: Screen Shot 2020-08-12 at 10 26 49 PM] https://user-images.githubusercontent.com/1007085/90097530-fccd6a80-dcea-11ea-90fc-886215d24467.png

Interestingly, if I load this in Qgis and select the feature, it shows the correct information. [image: Screen Shot 2020-08-12 at 10 28 16 PM] https://user-images.githubusercontent.com/1007085/90097634-41590600-dceb-11ea-8bd2-4fefc2bf87fa.png

If this was able to work like Qgis, great, but at the very least it should probably throw an exception as this can result in very correct data outputs.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/calvinmetcalf/shapefile-js/issues/134, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAITRHYXISJI4E44RV7K74DSAN3FZANCNFSM4P52J2YQ .

tehnrd commented 3 years ago

Row by row should work, as that is the spec, but looks like some files don't follow the spec. I may try to get a PR for this if I can squeeze in the time but at least wanted to get the issue logged in case anyone else came across the issue or felt ambitious about doing a PR.

calvinmetcalf commented 3 years ago

ok cool, If I have some time at some point I can take a look

tehnrd commented 3 years ago

@yakomaru @calvinmetcalf thanks for merging this fix so quickly. It definitely helps us with some data corruption issues we were experiencing.