search

NCAR / kcor-pipeline

Pipeline code for KCor
Other
3 stars 2 forks source link

Fix crash in realtime, automated CME detection #307

Open mgalloy opened 2 years ago

mgalloy commented 2 years ago

I got this crash in a simulation of a realtime, automated CME detection run:

2022-05-06 12:59:59 INFO: KCOR_CME_DET_REPORT: report sent to mgalloy@ucar.edu
2022-05-06 12:59:59 INFO: KCOR_CME_DET_CHECK: CME ended
2022-05-06 12:59:59 INFO: KCOR_CME_DETECTION_JOB: main loop
2022-05-06 13:00:04 INFO: KCOR_CME_DETECTION_JOB: main loop
2022-05-06 13:00:09 INFO: KCOR_CME_SEND_HEARTBEAT: creating heartbeat...
2022-05-06 13:00:09 CRITICAL: KCOR_CME_DETECTION_JOB: Stack trace for error
% FILE_COPY: Destination file already exists.
             File: /export/data1/Data/KCor/cme-alerts.cme-test/2017/10/06/mls
             o_kcor.2017-10-06T185033Z.2017-10-06T185708Z.json.
% Execution halted at:  KCOR_CME_SEND_HEARTBEAT   63 /export/data1/home/mgalloy/software/kcor-pipeline/cme_detection/kcor_cme_send_heartbeat.pro
%                       KCOR_CME_DETECTION_JOB  115 /export/data1/home/mgalloy/software/kcor-pipeline/cme_detection/kcor_cme_detection_job.pro
%                       $MAIN$

mlso_kcor.2017-10-06T185033Z.2017-10-06T185708Z.json already existed as a "SUMMARY" alert when a "HEARTBEAT" alert is created.

mgalloy commented 2 years ago

I believe this is caused by running in the simulator. The resolution of the "current time" is not as fine because the data copying data simulator only updates the "current time" periodically. The heartbeat is created only 10 seconds after the summary is created, so the "current time" might not have been updated since then.