astroumd / lmtoy

Toy Codes for LMT
https://www.astro.umd.edu/~teuben/LMT/lmtoy/html/
0 stars 2 forks source link

lmtinfo database needs to done incrementally #44

Open teuben opened 1 year ago

teuben commented 1 year ago

for small amounts of raw data in DATA_LMT the "lmtinfo.py build" commands is fast, less than a minute, but on unity all the data exists, plus unity is just slow. So we need a mode in which this can be run incrementally.

In one example on a cpu node it took 2 hours, in another case via slurm it ran out of time, as our timelimit was set to 4 hours.

teuben commented 1 year ago

basically done, though not automated:

  make new2 OBSNUM0=xxxxxx

where the file last.obsnum should store the last obsnum that was catalogued