TOML DataTest Issue

[rprospero]

I'm having an issue converting the test for tests/exchangeable/watermeth.txt into a TOML file. The data1D keyword adds the Data1DStoreKeyword to the module. The first example of this occurs in line 13 of the TOML file. When the file is parsed back in, the data is correctly added back into the Data1D keyword on the module and could be accesed from there. However, on line 29 of src/modules/datatest/process.cpp, It searches the processingModuleData for the data set. I cannot find any corresponding call to realise that would have populated processingModuleData, though it obviously exists somewhere, given that the test passes on the non-TOML version of the code.

The quick solution is to put in an override for DataTestModule where it automatically add the data store keywords to the processingModuleData. However, it feels like I'm missing something fundamental about populating this data structure. The code would also be a fairly ugly hack, since that part of the parser currently does not have access to the Dissolve instance.

As a side note, would I be correct that the elements in Data1DStore must have unique tags? If so, I can probably clean up the TOML file from looking like:

[[layers.Processing.modules."Total_weighted_S(Q)".data1D]]
tag = "HHH//WeightedSQ//Total"

[layers.Processing.modules."Total_weighted_S(Q)".data1D.data]
format = "xy"
filename = "../_data/epsr25/water300methanol600/watermeth.EPSR.u01"

[[layers.Processing.modules."Total_weighted_S(Q)".data1D]]
tag = "H5H//WeightedSQ//Total"

[layers.Processing.modules."Total_weighted_S(Q)".data1D.data]
format = "xy"
filename = "../_data/epsr25/water300methanol600/watermeth.EPSR.u01"

[layers.Processing.modules."Total_weighted_S(Q)".data1D.data.keywords]
y = 4

to

[layers.Processing.modules."Total_weighted_S(Q)".data1D."HHH//WeightedSQ//Total"]

[layers.Processing.modules."Total_weighted_S(Q)".data1D."HHH//WeightedSQ//Total".data]
format = "xy"
filename = "../_data/epsr25/water300methanol600/watermeth.EPSR.u01"

[layers.Processing.modules."Total_weighted_S(Q)".data1D."H5H//WeightedSQ//Total"]

[layers.Processing.modules."Total_weighted_S(Q)".data1D."H5H//WeightedSQ//Total".data]
format = "xy"
filename = "../_data/epsr25/water300methanol600/watermeth.EPSR.u01"

[layers.Processing.modules."Total_weighted_S(Q)".data1D."H5H//WeightedSQ//Total".data.keywords]
y = 4

[trisyoungs]

@rprospero So the mechanism of the Data1DStore is to store a set of Data1D whose data are the reference data (from disk) and the tag (e.g. HHH//WeightedSQ//Total) is the corresponding data to compare against in the processing module data. This data doesn't exist at the time of the input file read, but it is expected that it will be generated by some module at some point during the main simulation. Does that answer your question?

In fact the Data1D members of the Data1DStore don't have to have unique tags (i.e. unique references to data "yet to be generated") but I can't think of a convincing case why you wouldn't enforce it. That said, I can't think of a convincing case why we would really need to, either!

[rprospero]

@trisyoungs If I'm understanding you correctly, then I've been looking in the wrong area. In the specific example of the tests/exchangeable/watermeth.txt file, the data should have been added to processingModuleData by the NeutronSq modules in the "Correlations" layer and then compare against the files that were loaded in the "Processing" layer.

Looking closer, this makes sense. When loading the TOML file, the "Processing" layer is running before the "Correlations" layer. That doesn't occur with the input file, so that is clearly the cause of the issue.

[trisyoungs]

@rprospero Does this mean to say that the order of the layers is changing in the TOML? That's not healthy! :)