When importing data from csv (csv_to_table/bulk) if there are empty columns pandas defaults them to Int datatype, but often this creates issues when merging or updating if the field isnt really always null, but really just very sparsely populated. This identifies fields with this condition and sets the database datatype to varchar.
Notes:
Tests are included
pyarrow functions are not modified for this behavior
When importing data from csv (csv_to_table/bulk) if there are empty columns pandas defaults them to Int datatype, but often this creates issues when merging or updating if the field isnt really always null, but really just very sparsely populated. This identifies fields with this condition and sets the database datatype to varchar.
Notes: