pytroll / trollflow2

Next generation Trollflow. Trollflow is for batch-processing satellite data using Satpy
https://trollflow2.readthedocs.org/
GNU General Public License v3.0
10 stars 15 forks source link

Plugin for checking whether incoming data have been processed #117

Open pnuu opened 3 years ago

pnuu commented 3 years ago

Feature Request

Is your feature request related to a problem? Please describe. I had recently mayhem with data transfers sending old data, and these entered Trollflow2 processing as new data delaying the processing for two hours.

If there were a plugin that checked whether the data has been already successfully processed, we could skip the processing.

Describe the solution you'd like Go through the product list, create product filenames and see if they exist. Abort processing for those files that exist (and are not zero-sized?).

Describe any changes to existing user workflow None, use of the plugin would be optional.

Additional context None.