LAMMPS: each process has number of atoms changing over time. Time aggregation works fine with data that does not change in size. Current code estimates at every step in adios_open() what the expected buffersize is for N steps and changes the max buffersize accordingly. This accommodates varying size of data somewhat but it is still not enough to handle bad cases: e.g. aggregating 20 steps, 19 steps are similar in size and the 20th step becomes larger just a bit than the average of the first 19 steps. In this case the max buffer size will limit the resize requests during the 20th step.
Suggestion: keep track of the individual step size and estimate the variance among them, and use that in calculating the expected buffer size. Or use the maximum size so far instead of the sum of sizes. Also add some extra wiggle rooom, like 10%. Also, maybe it's worth to do this in adios_close() as well and just dump 19 steps if we are not sure about storing 20 steps.
LAMMPS: each process has number of atoms changing over time. Time aggregation works fine with data that does not change in size. Current code estimates at every step in adios_open() what the expected buffersize is for N steps and changes the max buffersize accordingly. This accommodates varying size of data somewhat but it is still not enough to handle bad cases: e.g. aggregating 20 steps, 19 steps are similar in size and the 20th step becomes larger just a bit than the average of the first 19 steps. In this case the max buffer size will limit the resize requests during the 20th step.
Suggestion: keep track of the individual step size and estimate the variance among them, and use that in calculating the expected buffer size. Or use the maximum size so far instead of the sum of sizes. Also add some extra wiggle rooom, like 10%. Also, maybe it's worth to do this in adios_close() as well and just dump 19 steps if we are not sure about storing 20 steps.