search

Clinical-Genomics / cg

Glue between Clinical Genomics apps
8 stars 2 forks source link

Delivery of analysis of external samples skips cram-files? #2304

Closed mathiasbio closed 5 days ago

mathiasbio commented 1 year ago

Description

I created a case to run balsamic on some external data to be delivered to cust127 (delivery type: analysis-scout). For context the analysis failed the final QC-check in balsamic but I set the case to "Mark Complete" in trailblazer regardless. All the result-files were then delivered to the customer inbox on Caesar except for the cram-files.

When running the delivery manually I get these messages: image

And when speaking with @SofiaOtero I learned there's a check that cram-files are not delivered if there's insufficient reads, and for this external-case since it's not read from LIMS the amount of reads is 0.

Suggested solution

Maybe all files marked for delivery can be delivered when the analysis has been marked as complete.

This can be closed when

Blocked by

If there are any blocking issues/prs/things in this or other repos. Please link to them.

henrikstranneheim commented 1 year ago

How about adding a force flag to skip selected QC checks?

SofiaOtero commented 1 year ago

Yes that can be used to deliver it! The problem is that we don't see that the files are missing when the automation uploads the files to Caesar

henrikstranneheim commented 1 year ago

Do we raise and log an error and set the rsync to failed in Trailblazer? I am not up to speed on the current implementation

SofiaOtero commented 1 year ago

No we do not have either cron mail or fail in trailblazer.

It affects both the cram and fastq delivery.

cg deliver analysis -d fastq -c bigmarten --dry-run
/home/proj/production/bin/miniconda3/envs/P_main/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:2231: SAWarning: Cascade settings "delete, merge, save-update" should not be combined with a viewonly=True relationship.   This configuration will raise an error in version 1.4.  Note that in versions prior to 1.4, these cascade settings may still produce a mutating effect even though this relationship is marked as viewonly=True.
  "viewonly=True." % (", ".join(sorted(non_viewonly)))
Running CG deliver
Set dry run to True
/home/proj/production/bin/miniconda3/envs/P_main/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:1998: SAWarning: Setting backref / back_populates on relationship Demux.samples to refer to viewonly relationship Sample.demuxes should include sync_backref=False set on the Demux.samples relationship.  (this warning may be suppressed after 10 occurrences)
  (rel_b, rel_a, rel_b),
/home/proj/production/bin/miniconda3/envs/P_main/lib/python3.7/site-packages/sqlalchemy/orm/relationships.py:1998: SAWarning: Setting backref / back_populates on relationship Sample.demuxes to refer to viewonly relationship Demux.samples should include sync_backref=False set on the Sample.demuxes relationship.  (this warning may be suppressed after 10 occurrences)
  (rel_b, rel_a, rel_b),
Setting ticket to 202180
Setting customer_id to cust127
Sample ACC12398A1 did not receive enough reads and will not be delivered
Sample ACC12398A2 did not receive enough reads and will not be delivered

If there was no QC errors and the automation pics it up, then it will deliver the analysis files to Caesar, but not include the cram or fastq files.

There was a problem with this earlier for samples that were re-sequenced and the reads were not updated in statusdb, but this issue https://github.com/Clinical-Genomics/cg/issues/2241 has been solved.

Now we encounter the issue when we have e.g. external samples where the read count in statusdb is 0.

ChrOertlin commented 3 months ago

@karlnyr Is this still a problem?

Vince-janv commented 5 days ago

Closing due to inactivity. Reopen and answer the question below if you want this prioritised.

Concerning the reported bug: