search

c3-time-domain / SeeChange

A time-domain data reduction pipeline (e.g., for handling images->lightcurves) for surveys like DECam and LS4
BSD 3-Clause "New" or "Revised" License
0 stars 4 forks source link

Files not getting pulled properly from the archive when moving to a new machine #277

Open rknop opened 6 months ago

rknop commented 6 months ago

While working on the stress test, thanks to perlmutter downtime, I started swapping between machines. I started on one machine, then moved to another. The database was already loaded with calibration files, but then I was getting "file not found" errors for the linearity file.

I suspect that there's a bug somewhere in the file that pulls preprocessor calibration files from the archive. Look into this.

rknop commented 6 months ago

...having lots of other problems with files. It may be something else in my setup in the latest machine I'm trying to get to work.

rknop commented 6 months ago

I don't think it's the config. I wiped the database, and everything worked on the new machine.

So, there's something wrong somewhere with syncing files from the archive when they exist there but aren't present on the local machine. (I think the archive code tests this, so this may require some digging to find.)

guynir42 commented 6 months ago

Does this only happen for calibration files? I think this may get solved by itself if you simplify the way they are loaded (see issue #162)