Open vignesh1987 opened 1 year ago
Hi @vignesh1987 - thanks for the bug report. Initially, I thought this is likely an issue with Amply, but I see that Amply does support this format for representing parameters (see tests).
Please could you share the full error report from otoole by running otoole with the -vvv
flag. e.g. otoole -vvv convert datafile csv data_lao.txt data_lao_csv
Hi @willu47,
Thanks for the pointers. I ran the code with the -vvv
flag, and the error log is attached.
error_log_TabularRecord.txt
It refers to the first instance of the records list (line 308, Amply.py) and evaluates the length not equal to 1, if I am thinking correctly.
Hi @willu47
I tried to debug by comparing the SIMPLICITY example. Solved the issue for this case, but I am not sure it is a fix for every data file. Some pointers below
TechnologyActivityByModeLowerLimit TechnologyActivityByModeUpperLimit TechnologyActivityDecreaseByModeLimit TechnologyActivityIncreaseByModeLimit
Hi @vignesh1987 - thanks for reporting back.
So the issue is that if the list of parameters and sets in the config.yaml
file does not match what is in the data file, we get a really cryptic error message?
For otoole, there's no issue with using fuel or commodity, as long as the set name is updated in the config.yaml file.
Hi @vignesh1987 and @willu47! I had a similar error message from Amply pop up before (see issue #151 which was replaced by issue #160).
Also, I think @willu47's good point of "So the issue is that if the list of parameters and sets in the config.yaml file does not match what is in the data file, we get a really cryptic error message?" is expanded upon in the (still open) issue of issue #169. In this issue, the error message of OtooleNameMismatch
is explained to be a little to nondescript, so maybe moving to a more descriptive one (such as DatafileParseError
as described by @willu47 in issue #182) would be better!
Thanks so much @vignesh1987 for reporting this issue!!
Thanks, @trevorb1 and @willu47 : I wrote something simple to compare the datafile and the config.yml file. Basically, something that warns the user of parameters that are in the config file but not in the data file. Will that be useful here? Should it be made generic (excel, datafile and CSV folder VS the config file)? Also will make some effort to get involved more in contributing better (I have been wanting to do this for ages and never materialised).
Sounds useful. We have validation for the other formats. The data files are the tricky ones because of how the amply parser works. Technically amply could be modified to extract the parameter names, but it is a difficult library (build on pyparsing) to work with. Are you using a regular expression to search for the param and set keywords?
Hi, I have a question on the otoole convert command. I see that the format for the simplicity data file (generated through Otoole) is slightly different from the older Gnumathprog formats. For example: In the otoole format, the Operational life parameter is defined as in the example below
In the older data file format, it used to be as in the example below. where the region index is defined as a group.
I get the following error when I want to use Otoole to convert old data files to CSV or Excel.
Is there an easier way to work around this issue? Attached is the older data file I am referring to: data_lao.txt