cronjob can run same file twice if takes longer than 2 min

SINTEF / olmo

SINTEF OceanLab Observatory data handling and storage

3 stars 4 forks source link

cronjob can run same file twice if takes longer than 2 min #7

Closed WillRNaylor closed 2 years ago

WillRNaylor commented 2 years ago

We saw this problem when ingesting the power data with Emlyn - it took many minutes (30?), and after 12 mins or so there were 6 processes running all trying to ingest the same files.

I believe this doens't actually ruin any data - and everything gets ingested. Just its really bad and casuses errors in most of the processes that are started up.

WillRNaylor commented 2 years ago

Fixed with flux (although that now has it's own problems...)