The BQ Ingest job was invoked 3 separate times as a cron job in the most recent file processing - clinvar_2024_08_12_v1_1_0_beta5.
We need to adjust the BQ ingest process to where a “processing” flag is added to the processing_history view and is set as soon as a single BQ Ingest starts working on a new dataset. The selection query will need to be updated to include this “processing” flag when NULL or some such. This will also need to be unset in the database upon an exceptional failure condition.
The BQ Ingest job was invoked 3 separate times as a cron job in the most recent file processing - clinvar_2024_08_12_v1_1_0_beta5.
We need to adjust the BQ ingest process to where a “processing” flag is added to the processing_history view and is set as soon as a single BQ Ingest starts working on a new dataset. The selection query will need to be updated to include this “processing” flag when NULL or some such. This will also need to be unset in the database upon an exceptional failure condition.
Consider alternatives.