NREL / Marmot

Marmot is a data formatting and visualization tool for production cost and capacity expansion modelling results. It provides an efficient way to combine and view disaggregated results, while also creating publication ready figures and data tables.
Other
8 stars 4 forks source link

Replace data 0 column with "values" name #44

Closed daniellevie closed 2 years ago

daniellevie commented 2 years ago

We have been having the following error for as along as I can remember but I haven't been able to figure it out until now.

PerformanceWarning: your performance may suffer as PyTables will pickle object types that it cannot map directly to c-types [inferred_type->integer,key->axis0] [items->None]

This error usually means that columns have mixed data types which they never have had making this hard to solve. I discovered that calling our data column 0 was causing this issue and renaming it "values" has stopped the warning. It has also reduced the size of a formatted hdf5 file by a factor of 5. The PLEXOS RTS test system went from 36MB to 7.2MB. This will likely significantly speed up both our formatting and plotting as less data needs to be handled now.