Single sample microsalt cases cant start automatically

[gustavlagneborg]

Description

Cases that only have 1 sample cant start automatically.

Suggested solution

This can be closed when

When single sample cases can start with the cg workflow microsalt start <case-id>, tracked by trailblazer and stored successfully in housekeeper statusDB analysis tab

[gustavlagneborg]

When I try to start the single sample microsalt case proudfish I get this error:

θ62° 3d [hiseq.clinical@hasta:/home/proj/production/microbial/fastq/proudfish/ACC9337A4] [P_main] $ cg workflow microsalt start proudfish
Called undefined __fields__ on HousekeeperAPI, please wrap
Called undefined __fields__ on HousekeeperAPI, please wrap
Called undefined __fields__ on HousekeeperAPI, please wrap
Called undefined __fields__ on HousekeeperAPI, please wrap
Called undefined __fields__ on HousekeeperAPI, please wrap
Called undefined __fields__ on HousekeeperAPI, please wrap
Called undefined __fields__ on HousekeeperAPI, please wrap
Called undefined __fields__ on HousekeeperAPI, please wrap
Starting Microsalt workflow for proudfish
Fetching files from bundle ACC9337A4
Fetching files with tags in [fastq]
Saved config /home/proj/production/microbial/queries/proudfish.json
proudfish : analysis is still ongoing - skipping
Trailblazer warning: Could not track analysis progress for case proudfish! CgError
Action running set for case proudfish
Running command source activate P_microSALT; /home/proj/bin/conda/envs/P_microSALT/bin/microSALT analyse /home/proj/production/microbial/queries/proudfish.json --input /home/proj/production/microbial/fastq/proudfish
Call ['source activate P_microSALT;', '/home/proj/bin/conda/envs/P_microSALT/bin/microSALT', 'analyse', '/home/proj/production/microbial/queries/proudfish.json', '--input', '/home/proj/production/microbial/fastq/proudfish'] exit with a non zero exit code
INFO - Downloading new MLST profiles for Citrobacter freundii
INFO - Re-indexed contents of /home/proj/production/microbial/references/ST_loci/citrobacter_freundii
INFO - Cached resFinder database identical to remote.
INFO - Re-indexed contents of /home/proj/production/microbial/references/resistances
INFO - Re-indexed contents of /home/proj/bin/conda/envs/P_microSALT/expec
WARNING - Record [CG_ID_project]=[ACC9337] in table Projects already exists
ERROR - Unable to create job for sample ACC9337A4
Source: No files in directory /home/proj/production/microbial/fastq/proudfish match file_pattern '\w{8,12}_\w{6,10}(?:-\d+)*(?:\w+)*_L\d{1,3}_(?:R)*(\d{1})(?:_\d{1,3})*.fastq.gz'.
ERROR - Unable to analyze single sample ACC9337A4
Traceback (most recent call last):
  File "/home/proj/bin/conda/envs/P_microSALT/bin/microSALT", line 8, in <module>
    sys.exit(root())
  File "/home/proj/bin/conda/envs/P_microSALT/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/proj/bin/conda/envs/P_microSALT/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/proj/bin/conda/envs/P_microSALT/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/proj/bin/conda/envs/P_microSALT/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/proj/bin/conda/envs/P_microSALT/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/proj/bin/conda/envs/P_microSALT/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/proj/bin/conda/envs/P_microSALT/lib/python3.6/site-packages/microSALT/cli.py", line 189, in analyse
    run_creator.project_job(single_sample=True)
  File "/home/proj/bin/conda/envs/P_microSALT/lib/python3.6/site-packages/microSALT/utils/job_creator.py", line 653, in project_job
    self.finish_job(jobarray, single_sample)
  File "/home/proj/bin/conda/envs/P_microSALT/lib/python3.6/site-packages/microSALT/utils/job_creator.py", line 674, in finish_job
    with open(samplefile, "w+") as outfile:
FileNotFoundError: [Errno 2] No such file or directory: '/home/proj/production/microbial/results//ACC9337A4_2022.11.9_17.40.33/sampleinfo.json'
Failed to run analysis!
Action None set for case proudfish
Traceback (most recent call last):
  File "/home/proj/bin/conda/envs/P_main/bin/cg", line 8, in <module>
    sys.exit(base())
  File "/home/proj/bin/conda/envs/P_main/lib/python3.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/proj/bin/conda/envs/P_main/lib/python3.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/proj/bin/conda/envs/P_main/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/proj/bin/conda/envs/P_main/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/proj/bin/conda/envs/P_main/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/proj/bin/conda/envs/P_main/lib/python3.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/proj/bin/conda/envs/P_main/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/proj/bin/conda/envs/P_main/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/proj/bin/conda/envs/P_main/lib/python3.7/site-packages/cg/cli/workflow/microsalt/base.py", line 198, in start
    context.invoke(run, ticket=ticket, sample=sample, unique_id=unique_id, dry_run=dry_run)
  File "/home/proj/bin/conda/envs/P_main/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/proj/bin/conda/envs/P_main/lib/python3.7/site-packages/click/decorators.py", line 27, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/home/proj/bin/conda/envs/P_main/lib/python3.7/site-packages/cg/cli/workflow/microsalt/base.py", line 177, in run
    analysis_api.process.run_command(parameters=analyse_command, dry_run=dry_run)
  File "/home/proj/bin/conda/envs/P_main/lib/python3.7/site-packages/cg/utils/commands.py", line 93, in run_command
    raise CalledProcessError(res.returncode, command)
subprocess.CalledProcessError: Command '['source activate P_microSALT;', '/home/proj/bin/conda/envs/P_microSALT/bin/microSALT', 'analyse', '/home/proj/production/microbial/queries/proudfish.json', '--input', '/home/proj/production/microbial/fastq/proudfish']' returned non-zero exit status 1

[gustavlagneborg]

However, running this command cg workflow microsalt start -s ACC9337A4works

[gustavlagneborg]

I emailed Isak Sylvin and asked him why the microSALT pipeline differed between single sample analysis and a case with multiple samples: here is the answer:

Hej!

Way back when när jag skrev microSALT så kunde man antingen starta den med ett projekt, eller ett enskilt prov. Eftersom microSALT behövde både prov och projektnamn för sin datalagring så blev lösningen lite olika. Gav man ett provnamn behövde microSALT hitta projektnamnet. Gav man däremot ett projektnamn så fick microSALT provnamnen nästan gratis.

Jag ponerade att generalisera det här till bara en generell väg att gå, men hade inte riktigt tid att få upp den koden. Så det som ligger är en lösning med en hel del silvertejp.

https://github.com/Clinical-Genomics/microSALT/blob/a39fd0fa0b850121668433fa0de7c4da1a3ca5b3/microSALT/utils/job_creator.py#L587 ^ Är en bra utgångspunkt för hur dom båda vägarna skiljer sig i dagsläget.

Återkom om du har fler frågor så kan vi boka in något.

MVH Isak

Since MicroSALT is being replaced in the near future, I believe it would be unnecessary work to fix this issue within the pipeline.

[gustavlagneborg]

As this would require too much work to fix in the microsalt pipeline and making a temporary fix in CG is not good practice, this issue will be closed for now. Instead, I will document in Atlas how to handle a single sample case that doesn't start