heremaps / here-cli

A command-line interface to work with HERE XYZ Hub and other HERE APIs
https://www.here.xyz/
MIT License
38 stars 19 forks source link

detect fields in CSV that are numbers, and offer to convert #96

Closed burritojustice closed 5 years ago

burritojustice commented 5 years ago

Analyze values of properties before uploading a CSV, detect fields that are numbers, and set their type as 'number'.

Maybe use something like

function isNumeric(n) { 
      return !isNaN(parseFloat(n)) && isFinite(n); 
}

But also provide an option for certain fields to 'opt out' -- this is useful for postal codes and Census geoIDs where there could be a leading zero and the full string is important.

mapshaper does this using string-fields=FIPS, maybe have --string-fields. Not sure what the short option would be... -t, -s,-f are all being used. -n doesn't make much sense...

Also, I'd suggest we treat the field name as case insensitive.

burritojustice commented 5 years ago

@Naitik333 re https://github.com/heremaps/here-cli/commit/87069be21596d0e7b8602829fb895736646328da#diff-25d902c24283ab8cfbac54dfa101ad31

Numbers are getting written as numbers by default, which is good!

However, I can't get --string-fields to work. The zip codes in this example still get written as numbers.

http://geojson.tools/index.html?url=https://xyz.api.here.com/hub/spaces/wjRUrbfL/search?limit=5000&clientId=cli&access_token=AIrZgZMV7dDqarJ70CCd_h8

node bin/here.js xyz upload wjRUrbfL -f /Users/joram/Downloads/City_Facilities_-_Public_Library_Jurisdiction_or_Leased.csv --string-fields zip_code

Am I using that right?

burritojustice commented 5 years ago

I also think we need to add a third dialog to -a that offers a list of fields and lets you choose which are numeric.

Fields that look like numbers and booleans will be converted as such. If you want to save numbers as strings (postal codes, FIPS, GEOIDs) select them here:

burritojustice commented 5 years ago

here's that csv with zip codes btw CityFacilities-_Public_Library_Jurisdiction_or_Leased.csv.zip

Naitik333 commented 5 years ago

@burritojustice Can you try now --string-fields option, it should work now.

burritojustice commented 5 years ago

works!