search

agrc / palletjack

A library for updating AGOL data from various external sources
https://agrc.github.io/palletjack/palletjack/
MIT License
12 stars 0 forks source link

upsert existing data: missing keys in new data can cause switch from ints to floats to handle nans #22

Closed jacobdadams closed 1 year ago

jacobdadams commented 1 year ago

If we have an int column in the live data and the new data has a row that isn't present in the live data, the .merge in _get_common_rows() will cause the int column to be a float to handle nans.

Maybe look at the updated pandas int type that can handle nans? Maybe force all original int fields back to ints?

jacobdadams commented 1 year ago

See test case TestFeatureServiceUpdater/test_get_common_rows_handles_ints_in_existing_to_float_for_rows_not_in_existing_dataset