search

koopjs / koop-socrata

Socrata provider for Koop.
Other
8 stars 7 forks source link

Numbers being interpreted as date fields #57

Closed sirws closed 8 years ago

sirws commented 8 years ago

https://data.wa.gov/dataset/WQI-Parameter-Scores-1994-2013/dn4d-x42e via: http://koop.dc.esri.com/socrata/wastate/dn4d-x42e/FeatureServer/0

Numeric fields are showing as esriFieldTypeDate. This is incorrect. Should be number OR string.

Thanks! Scott

sirws commented 8 years ago

@dmfenton

sirws commented 8 years ago

I have made some changes to fix this. Please review. I am using the socrata field types to also store numeric data as numbers instead of strings. This fix also makes sure that all fields are in the output. https://github.com/sirws/koop-socrata/commit/dcbabccae91bb99478b13c0205e9b7cc3cedd5fd

jgravois commented 8 years ago

can you submit a PR? that would make it a lot easier for me to review/test.

sirws commented 8 years ago

done! check it out. I have some test cases that work well.

https://data.wa.gov/Natural-Resources-Environment/Water-Right-Applications/9ubz-5r4b and https://data.wa.gov/dataset/WQI-Parameter-Scores-1994-2013/dn4d-x42e

Compare to the new build: http://koop.dc.esri.com/socrata/wastate/9ubz-5r4b/FeatureServer http://koop.dc.esri.com/socrata/wastate/dn4d-x42e/FeatureServer

Dates are dates, numbers are numbers, all fields are available whereas in current release, in the 9ubz dataset, there is a gpm and domestic_units fields that aren't coming over because the first row returned in socrata has null values in those fields.

jgravois commented 8 years ago

closed via #58 (nice work @sirws!)

adamajm commented 8 years ago

Hi all, this issue doesn't seem to be resolved. Even on the test cases mentioned above, numbers in Socrata are not coming in as numbers (coming in as "esriFieldTypeDate"). Same is true in our tests, and in our case Dates (w/Timezone) in Socrata are coming in as "esriFieldTypeInteger".

We have all the latest Koop and Koop-Socrata versions.

https://data.raleighnc.gov/Urban-Planning/Building-Permits-Non-Residential-BLDS-2014-Present/j8mi-4t5w http://koop.raleighnc.gov/socrata/raleigh/j8mi-4t5w/FeatureServer

adamajm commented 8 years ago

@sirws @jgravois our experience, the "Money" format in Socrata is treated as STRING by Koop and most "Text" formats in Socrata that HAVE numbers inside are converted to DATES by Koop

dmfenton commented 8 years ago

This is not an issue with koop-socrata. It's in Koop core and was fixed with the last commit. I'll publish a release that includes the fix.

adamajm commented 8 years ago

ah, ok. Thanks @dmfenton

dmfenton commented 8 years ago

I just published Koop@2.12.2. Let me know if that doesn't solve your problem with date fields at least. I doubt the money issue will be solved since the "$" character will cause the value to be treated as a string.