spacepy / dbprocessing

Automated processing controller for heliophysics data
5 stars 4 forks source link

Separate out RUN timebase from lack of output product #108

Open jtniehof opened 2 years ago

jtniehof commented 2 years ago

When we first created the RUN timebase, we were thinking in terms of performing some operation based on a single newly-created file, so the timebase-related code to select input files didn't make much sense.

But I can see a case where, for instance, we might need a MONTHLY timebase no-output process, which gets triggered based on any change to any of its daily input files.

Proposed enhancement

I suggest we move the RUN special handling to be based specifically on having a NULL output product, and then retire the RUN timebase. This should even simplify the code.

Alternatives

We can stay as-is...this certainly won't matter too much until we get timebases other than DAILY working. But I can envision it being useful in, say, doing monthly summary plots.

OS, Python version, and dependency version information:

Linux-4.15.0-163-generic-x86_64-with-Ubuntu-18.04-bionic
sys.version_info(major=2, minor=7, micro=17, releaselevel='final', serial=0)
sqlalchemy=1.1.11

Version of dbprocessing

Current git master

Closure condition

This issue should be closed when the RUN timebase is replaced with code based on lack of output product in a process.