More specifically, having multiple Datasets ending in parallel tasks doesn't work if those Datasets are in a module below the egon.data.datasets package. In that case the code removing the module name prefix from task ids and the code generating the final dataset task which updates the dataset version once all parallel tasks have finished interact in a way that generates non-distinct task ids so that tasks generated later clobber the ones generated earlier. This leads to spurious cycles and other inconsistencies and bugs in the graph.
More specifically, having multiple
Dataset
s ending in parallel tasks doesn't work if thoseDataset
s are in a module below theegon.data.datasets
package. In that case the code removing the module name prefix from task ids and the code generating the final dataset task which updates the dataset version once all parallel tasks have finished interact in a way that generates non-distinct task ids so that tasks generated later clobber the ones generated earlier. This leads to spurious cycles and other inconsistencies and bugs in the graph.