hrecht / censusapi

R package to retrieve U.S. Census data and metadata via API
https://www.hrecht.com/censusapi/
169 stars 31 forks source link

Refine numeric column handling #25

Closed hrecht closed 6 years ago

hrecht commented 7 years ago

Detecting numeric columns is messy: the APIs return all data as strings. getCensus currently types columns with numbers in the col name as numeric. This works well for the ACS and decennial Census APIs but needs improvement for several reasons:

Theoretically, it should be possible to use the variable API endpoints to get type information (the approach of Python lib census https://github.com/datamade/census) but that field doesn't uniformly exist and is often wrong. I've raised this issue with the Census dev team and hope they'll correctly type data in the future.

Until then, this package needs typing improvement particularly for the timeseries APIs.

hrecht commented 6 years ago

This package currently uses a set of rule-based workarounds to type data as string or numeric. Until properly formatted JSON is returned by the Census APIs (which I have asked about multiple times), this is an issue to be solved by the Census Bureau.