Closed FelixMau closed 10 months ago
Checks are still failing but I cannot really fix them until a propper Datapackage can be created and has been tested valid. Also #44 needs to be closed before better tests can be written, I'll work on #44 on Monday and then start working on tests again from this point onwards.
I would have merged period scalars before parametrizing facades, aka one step before https://github.com/sedos-project/data_adapter_oemof/blob/39c9c469805853741151d3118a80a81feddcd606/data_adapter_oemof/build_datapackage.py#L372 This would preserve facade internal checks. Additionally, we would not have to create more facades than needed (for each period) and aggregate them later into single facades (which is also confusing IMO). Could you refactor code, so that period-aggregation is done before L372 and period scalars go into parametrization? I think as you already prepared the functions this is easy...
Moving aggregation functionality to begining of build_datapackage
for clean pipeline. Yet the moving leads to an issue related to saving the preiodic values.
When parametrizing the components and periodic values
are found we discussed to save them as lists of full length into the field. When saving the datapackage the full length list would be saved, taking up a lot of memory and leading to humanly unreadable data.
I have two ideas for solving the issue:
new_init
can be extended to not plug values into plumbing and save them as they are. @nailend Do have a draft already how and wehre multiperiods are read in or have another Idea on this issue?List
(or _Sequence
) on as_dict
call. If it is a list write the unique values of the list. When parametrizing the components and
periodic values
are found we discussed to save them as lists of full length into the field. When saving the datapackage the full length list would be saved, taking up a lot of memory and leading to humanly unreadable data.
No, we decided to move the "to full length" extraction into tabular
. In the oemof.adapter
periodic values
shall be written in a list
with periodic-values
only. e.g. [1,2,3]. So just check if the values are unique and if not write them in a list.
@nailend Do have a draft already how and wehre multiperiods are read in or have another Idea on this issue?
Not yet, but this shouldn't be a dependency for this.
Closes #40 Check on periods and write them to the Data a method within Datapackage has been created. The Method
plumbin._Sequence
is not allowed) check if length the sum of found listsis equal to sequence length those sequences are added to timeseries data and foreign keys are written and updated 4.3 If all entries in the column are equal, only unique values is written 4.4 If the entries in columns are not all equal, a list with its values is written