Bug in upload of RNA cases to Scout

[SofiaOtero]

Description

When uploading a RNA case to Scout the cg upload -f command ran without any issues even though the connected DNA case was not sequenced yet, hence not available on Scout yet.

This is the output I got:

----------------- UPLOAD RNA TO SCOUT -----------------------
----------------- UPLOAD RNA FUSION REPORT TO SCOUT -----------------------
Fetch latest version from bundle immunecrow
Fetching files with tags in [fusion,clinical]
Fetching files from version 136265
Clinical fusion report immunecrow/2023-03-25/immunecrow_report.selected.pdf found
Upload Clinical fusion report finished!
----------------- UPLOAD RNA JUNCTIONS TO SCOUT -----------------------
Fetch latest version from bundle immunecrow
Fetching files with tags in [junction,bed,ACC11597A4]
Fetching files from version 136265
Splice junctions bed file immunecrow/2023-03-25/ACC11597A4_lanes_1234_star_sorted_sj.bed.gz found
Upload splice junctions bed file finished!
Fetch latest version from bundle immunecrow
Fetching files with tags in [coverage,bigwig,ACC11597A4]
Fetching files from version 136265
RNA coverage bigwig file immunecrow/2023-03-25/ACC11597A4_lanes_1234_star_sorted_sj.bigWig found
Upload RNA coverage bigwig file finished!
Upload of case immunecrow was successful. Setting uploaded at to 2023-03-29 13:38:02.300714
immunecrow - uploaded at set to 2023-03-29 13:38:02
immunecrow analysis has been successfully uploaded

So it looks like the RNA analysis was uploaded to Scout though it wasn't.

Suggested solution

This can be closed when

The command writes an error about the DNA case not being uploaded to Scout yet, similar to the error we get when there is no DNA case connected to the RNA case.

[SofiaOtero]

@peterpru @seallard could this be connected to your recent merge?

[peterpru]

It should spit out an error, saying something like scout report not found for the DNA case, which it has been doing previously. If you refer to this, https://github.com/Clinical-Genomics/cg/pull/1852, the function I made there has not been put into use yet.

[seallard]

It might be, I will take a look when I'm back next week!

[seallard]

It seems like the issue has been resolved since then. @SofiaOtero noticed the following output when the DNA case had not sequenced yet:

----------------- UPLOAD RNA TO SCOUT -----------------------
----------------- UPLOAD RNA FUSION REPORT TO SCOUT -----------------------
Failed to upload files for RNA case: unexpected number of DNA sample matches for subject_id: G000386542. Number of matches: 0

We still want to add a test that verifies that the expected error is raised.

[SofiaOtero]

Seems like the issue is resolved as I got this error today when uploading to a DNA case that was in statusdb but not uploaded to Scout yet:

----------------- UPLOAD RNA TO SCOUT -----------------------
----------------- UPLOAD RNA FUSION REPORT TO SCOUT -----------------------
Fetch latest version from bundle knownkit
Fetching files with tags in [clinical,fusion]
Fetching files from version 140097
Clinical fusion report knownkit/2023-04-26/knownkit_report.selected.pdf found
Uploading Clinical fusion report to scout for case deepbass
Uploading fusion report /home/proj/production/housekeeper-bundles/knownkit/2023-04-26/knownkit_report.selected.pdf to case deepbass
Running command /home/proj/production/bin/miniconda3/envs/P_scout/bin/scout --config /home/proj/production/servers/config/hasta.scilifelab.se/scout.yaml load gene-fusion-report deepbass /home/proj/production/housekeeper-bundles/knownkit/2023-04-26/knownkit_report.selected.pdf
Call ['/home/proj/production/bin/miniconda3/envs/P_scout/bin/scout', '--config', '/home/proj/production/servers/config/hasta.scilifelab.se/scout.yaml', 'load', 'gene-fusion-report', 'deepbass', '/home/proj/production/housekeeper-bundles/knownkit/2023-04-26/knownkit_report.selected.pdf'] exit with a non zero exit code
2023-04-27 11:00:55 hasta.scilifelab.se scout.commands.base[237940] INFO Running scout version 4.65.1
2023-04-27 11:00:55 hasta.scilifelab.se scout.commands.base[237940] DEBUG Debug logging enabled.
2023-04-27 11:00:55 hasta.scilifelab.se scout.commands.base[237940] WARNING Support for launching Scout using a .yaml config file is deprecated and will be removed in the next major release (5.0). Use a PYTHON (.py) config file instead.
2023-04-27 11:00:55 hasta.scilifelab.se scout.adapter.client[237940] INFO Connected to MongoDB 4.4.9
2023-04-27 11:00:55 hasta.scilifelab.se scout.adapter.mongo.base[237940] DEBUG Database name: scout
2023-04-27 11:00:55 hasta.scilifelab.se scout.server.app[237940] INFO Enable Phenopacket API
DeprecationWarning: The command 'gene-fusion-report' is deprecated.
2023-04-27 11:00:56 hasta.scilifelab.se scout.commands.load.report[237940] ERROR No case with _id deepbass found
Aborted!
Something went wrong when uploading fusion report
Traceback (most recent call last):
  File "/home/proj/production/bin/miniconda3/envs/P_main/bin/cg", line 8, in <module>
    sys.exit(base())
  File "/home/proj/production/bin/miniconda3/envs/P_main/lib/python3.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/proj/production/bin/miniconda3/envs/P_main/lib/python3.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/proj/production/bin/miniconda3/envs/P_main/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/proj/production/bin/miniconda3/envs/P_main/lib/python3.7/site-packages/click/core.py", line 1043, in invoke
    return Command.invoke(self, ctx)
  File "/home/proj/production/bin/miniconda3/envs/P_main/lib/python3.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/proj/production/bin/miniconda3/envs/P_main/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/proj/production/bin/miniconda3/envs/P_main/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/proj/production/bin/miniconda3/envs/P_main/lib/python3.7/site-packages/cg/cli/upload/base.py", line 83, in upload
    upload_api.upload(ctx=context, case=case, restart=restart)
  File "/home/proj/production/bin/miniconda3/envs/P_main/lib/python3.7/site-packages/cg/meta/upload/mip/mip_rna.py", line 44, in upload
    raise RuntimeError(f"Upload to Scout failed for sample {case.internal_id}")
RuntimeError: Upload to Scout failed for sample knownkit

[peterpru]

Hi, this issue is not resolved, see image below. I think happens when there is a connected DNA sample, but no connected DNA scout case or DNA analysis. Once the DNA case progresses, it will look as expected in the second post from Sofia.

At the time of the error below, there was a connected DNA sample by subject-id, and mip case movingocelot + balsamic case actualakita, but these have not yet started.