Feedback and Advice on Otoole with cbc sovler

[H3yfinn]

Dea Otoole Team,

I hope this message finds you well. I would like to share some feedback and provide some advice based on my experience using Otoole. As a user of your library, I have found it to be a valuable tool for my project, and I appreciate the effort that has gone into its development. However, I have encountered some challenges that I believe could be addressed to improve the overall user experience.

Documentation for osemosys_fast.txt with CBC solver: The documentation for using osemosys_fast.txt with the CBC solver is not very clear. It took me a long time to understand that only a few variables are outputted when using CBC with osemosys_fast.txt. It would be helpful if the documentation could better outline how to understand which variables won't be produced, as the current statement in the documentation, "The short and fast versions omit a large number of these calculated result variables so as to speed up the model matrix generation and solution times," is not very informative, and can easily be missed.

Simplifying installation of CBC solver: While I understand that it is not an issue with Otoole, installing the CBC solver from source can be challenging, especially on Windows. I found it much easier to install the 'mip' library as it contains CBC anyway.

Long variable names issue with CBC solver: I encountered an issue where the CBC solver reported errors due to variable names being too long. Although this is not an issue with Otoole, it would be helpful if Otoole could warn users when creating the datafile if their variable names are too long, and provide guidance on how to address this issue, such as shortening variable names. However, I understand that otoole should be ambivalent to the kind of solver used. Perhaps just a page of documentation with tips for using the different solvers would be really useful. < note that this is both the variable names within the osemosys.txt model file (eg. SC1_LowerLimit_BeginningOfDailyTimeBracketOfFirstInstanceOfDayTypeInFirstWeekConstraint) and the TECHNOLOGY, EMISSIONS, FUEL [etc] variables that the user sets.

Difficulty in creating the user_config file: Creating the user_config file was challenging for me, and I am still unsure if I have filled in all the options correctly. It seems that all the config options in https://github.com/OSeMOSYS/otoole/blob/master/src/otoole/preprocess/config.yaml can be filled in with 'calculated=True' to get the calculated values. However, it would be helpful to have clearer documentation or guidance on this point to ensure that users are setting up the user_config file correctly.

Thank you for considering this feedback. I hope it will be helpful in improving the usability and user experience of Otoole. I am happy to help you more with these improvements.

Best regards, Finn

[trevorb1]

Hi @H3yfinn. Thank you so much for using otoole and clearly reporting back your issues; we really appreciate it! We will review your comments and work on improving the documentation to make it clearer. Once done, we will address this issue with a PR. Thanks again!

[H3yfinn]

hey @trevorb1 something i noticed which i forgot to mention last time is related to this excerpt from my readme: One possible reason that is hard to spot is that installing/uninstalling otoole might fix it. this is because when otoole is installed it seems to record the location of itself in its code. So if you install it in one location, then move it, it will still be looking for the old location. To fix this, uninstall otoole, then reinstall it in the new location.

[trevorb1]

Hi @H3yfinn! Thank you for the follow up, and sincere apologies for not getting to this issue yet. I promise it is still on my radar! Again, thank you for providing feedback on otoole, we really appreciate it! I will address your comment(s) with a dedicated PR in the future and tag you for review! :)

[H3yfinn]

No problem @trevorb1 please take your time.

[trevorb1]

Hi @H3yfinn! Thank you so much for your patience as we get to these comments. Please find below (hopefully!) some answers:

Documentation for osemosys_fast.txt with CBC solver The latest version of otoole has updated instructions on how to use GLPK to build an OSeMOSYS model and run each solver from the command line; see here for the cbc example. Moreover, in the results conversion, the user now must provide the input data used to build the model (see here for functionality, and here for an example). This means that whatever version of OSeMOSYS you use, you will get the same result CSVs (provided you are not altering the existing variable definitions long/short/fast versions).

Simplifying installation of CBC solver Agreed. We have updated instructions on how to install cbc here. Specifically, if you use Anaconda to manage your packages, we have included instructions on how to download it via conda as it is quite a bit easier. I am unfamilar with the mip library. But it is just a python package that contains cbc?

Long variable names issue with CBC solver I agree with you that this is an issue, and that it should probably be addressed in the OSeMOSYS repository rather that otoole's. While I do think that having a "Solver Tips" page would be beneficial; I am unsure if otoole is the place to host this. Given that solver warnings/errors can be quite cryptic (in my opinion), this page may get quite complex and be outside the scope of otoole; which is (currently) to pre- and post-processing of data. However, maybe @willu47 could give his thoughts on this point as well?

Difficulty in creating the user_config file An example on how to create a pre-populated config.yaml file can be found here. If you are using a default version of OSeMOSYS (ie. the long/short/fast version on GitHub), this config file should work as is. In terms of modifying the config file, information on it can be found here. Apologies for the confusion on the calculated field. It's functionality was deprecated early on, however, the field was mistakenly not removed from the config validation. This has now been corrected! Thank you for flagging this! :)

Installing/uninstalling otoole hmm.... unfortunately I am not seeing where in the readme you are looking. Would you be able to elaborate on this issue please?

Again, thank you so much @H3yfinn for using otoole are reporting issues!!

[trevorb1]

Hey @H3yfinn, Im gonna go ahead and close this issue. If anything above is unclear please feel free to reopen :) Thanks again!!