search

OSeMOSYS / OSeMOSYS_GNU_MathProg

The GNU MathProg implementation of OSeMOSYS
Apache License 2.0
9 stars 14 forks source link

preprocess_data.py script is not compatible with any released version of the code. #67

Open tniet opened 3 years ago

tniet commented 3 years ago

The preprocessing of data and creation of the MODEX sets is faster with the python script than having GLPK do it during model generation. Currently, the preprocessing script is not compatible with the fast code (or is the fast code not compatible with the script?). The fast code should be released with the fastest way of running the model. I propose we make the fast code script compatible, and provide a short intro to the fast code with explanation on how to run the script to use it.

abhishek0208 commented 3 years ago

preprocess_data.py works with a modified version of osemosys_fast. Specifically, it requires that the sets of sets (e.g. MODExTECHNOLOGY*) are declared but not calculated in the model file. I'm not sure how best to approach this issue of compatibility of having the GNU Mathprog model file and Python preprocessing file interact. Could Amply help with this?

willu47 commented 3 years ago

Hi @abhishek0208 - thanks for the bug fixes and explanation. Perhaps just updating the docstring for the pre-processing script with more detailed instructions on how to make an osemosys model "script compatible" would be fine?

willu47 commented 3 years ago

@abhishek0208 - @JoPapp mentioned that it is difficult to understand which flag (otoole or momani) to use with a standard datafile (e.g. not generated by otoole or momani). What does this flag do?

willu47 commented 3 years ago

Also, in the fast code @JoPapp mentioned that there is an extra subset defined - MODEperTechnologyEmission which does not appear in the preprocessing script.

willu47 commented 3 years ago

The other option is that we pre-add the set definitions to all of the model codes - so that the long, short and fast versions are all preprocessing-compatible.

JoPapp commented 3 years ago

In the preprocessing script Line 291-292 is mentioned/used MODEperTechnologyEmission. This set is been generated only if there are emissions accounted in the osemosys model. However, glpk will not run since the preprocessing script generates the MODEperTechnologyEmission for a technology without having any value and the osemosys_Fast_Code has a constraint Line 225-226 that emissionactivityratio of technology needs to be <>0. So there is a conflict

abhishek0208 commented 3 years ago

@abhishek0208 - @JoPapp mentioned that it is difficult to understand which flag (otoole or momani) to use with a standard datafile (e.g. not generated by otoole or momani). What does this flag do?

@willu47 the otoole/momani flag is to specify the format of the data file to be (pre-)processed. They could be called long and wide instead. For now I'll update the readme.