njcuk9999 / apero-drs

A PipelinE to Reduce Observations - The DRS for SPIRou (CFHT)
MIT License
12 stars 0 forks source link

Should specify file encoding when reading csv files #633

Open cusher opened 4 years ago

cusher commented 4 years ago

On systems with a default text encoding other than UTF-8 (basically only Windows and obscure *nix flavors), reading csv files fails via _read_dict_files in core/drs_text.py. Can be fixed by replacing

                data = Table.read(filename, format='ascii.csv',
                                  fast_reader=False)

with

                data = Table.read(filename, format='ascii.csv',
                                  encoding='UTF-8',
                                  fast_reader=False)

It's probably also resolvable just by setting the locale via environment variables, but in various threads related to this type of issue, the Python devs mention it is a best practice to specify the encoding when opening existing text files where the encoding is already known. It's a small enough change so I would probably just add in the param, but I'll leave the decision to you.