spacepy / dbprocessing

Automated processing controller for heliophysics data
5 stars 4 forks source link

Only use products that exist on disk when figuring inputs #90

Closed jtniehof closed 2 years ago

jtniehof commented 2 years ago

This PR changes _getRequiredProducts to only consider files that are recorded in the database as existing on disk, and thus won't try to use nonexistent files as inputs to processing. Closes #47.

I think I had speculated somewhere that the runMe checks of the command line build would catch trying to build with a nonexistent file. Turns out that's not the case, and I haven't chased that down.

89 captures additional possible behavior changes that were incidentally mentioned in #47.

PR Checklist

jtniehof commented 2 years ago

I've reprocessed several days of PSP data which were affected by this bug (we had a problem where our L0 files were not split on the appropriate day boundaries, so obsolete versions of the L0 were across the wrong day.) All worked fine.