for small amounts of raw data in DATA_LMT the "lmtinfo.py build" commands is fast, less than a minute, but on unity all the data exists, plus unity is just slow. So we need a mode in which this can be run incrementally.
In one example on a cpu node it took 2 hours, in another case via slurm it ran out of time, as our timelimit was set to 4 hours.
for small amounts of raw data in DATA_LMT the "lmtinfo.py build" commands is fast, less than a minute, but on unity all the data exists, plus unity is just slow. So we need a mode in which this can be run incrementally.
In one example on a cpu node it took 2 hours, in another case via slurm it ran out of time, as our timelimit was set to 4 hours.