search

HEAL / healdata-utils

HEAL data packaging tools
https://heal.github.io/healdata-utils/
0 stars 4 forks source link

Detect file encodings #2

Closed mbkranz closed 1 year ago

mbkranz commented 1 year ago

For some scenarios, files are not saved with the standard utf-8 encoding. This happens, for example, when saving the default CSV setting in excel. Given this scenario may happen a lot (has already happened in DSC), need to detect file encodings when reading in files as the default encoding (such as for pandas) is uff-8.

mbkranz commented 1 year ago

Added as a function in healdata_utils.io called detect_file_encoding . Also added read_table that uses the detect fxn