search

trynthink / scout

A tool for estimating the future energy use, carbon emissions, and capital and operating cost impacts of energy efficiency and demand flexibility technologies in the U.S. residential and commercial building sectors.
https://scout.energy.gov
Other
58 stars 22 forks source link

Fix input metadata process, clean up file tests #356

Closed trynthink closed 7 months ago

trynthink commented 7 months ago

Fix mseg_meta.py exception when importing files to extract the year range of the input data. The root cause of the exception was changes to converter.py in https://github.com/trynthink/scout/commit/2d6087f781d7f1780f711acf3c761df8814a7521 that require the user to specify the conversion file to be updated instead of using hardcoded file names. Instead, specify the conversion file path and add a test checking for the presence of the file.

Remove lingering references to Travis-CI in some test decorators and supporting test infrastructure.

aspeake commented 7 months ago

@trynthink when I run locally, eia_file.py inserts blank lines at every other row to (at least) RESDBOUT.txt, is this expected behavior? It seems to be causing an error running mseg_meta.py.

trynthink commented 7 months ago

@trynthink when I run locally, eia_file.py inserts blank lines at every other row to (at least) RESDBOUT.txt, is this expected behavior? It seems to be causing an error running mseg_meta.py.

My first guess is that the blank lines are caused by differences in line ending handling between macOS and Windows. The objects to be written to files are generated using csv.DictWriter. The dialect argument specifies line ending handling, and defaults to 'excel,' but the settings should be OS-invariant. I'm not sure if the interpretation of the settings are not constant though.

@aspeake Could you inspect the files modified by eia_file.py and see which ones have this problem? And did you also encounter a separate file encoding issue with eia_file.py?