willu47
commented
2 years ago Hi @danielhuppmann - many thanks for taking a look at the testing.
I've decided not to implement the changes to the testing framework at present. I think introducing IAMDataFrames within the filter_* functions, as the library is currently structured, does not make sense as it would result in a lot of passing in of metadata to create the IAMDataFrames.
Also, these filter functions still need .group_by().sum() aggregation performed upon them, i.e. the data is not quite ready to be inserted into the pyam data structure.
I think if we refactor the code to make it a bit more general, we can create a wrapper function (perhaps a decorator) which we can use to reshape the outputs from each of these functions into something that can be compared with the pyam testing functions.
When running the tests locally (python 3.8), I got a number of dtype-errors on the value-column (observed:
object, expected:float).To fix this issue, I suggest to refactor to the pyam testing module (docs), because this gets around the issue of having to set an index, fix the data types, or sort.
The first commit implements this change only for the first test, for illustration that this results in leaner, easier-to-read code - if you agree that this is a useful improvement, I'll also implement it for the other tests.