snakemake workflow FileNotFoundError

[erikadudki]

Hello, I'm trying to run the snakemake workflow for DS analysis method comparison. When I run snakemake in the terminal, I get the following error: FileNotFoundError in line 25 of /home/PycharmProjects/muscat-comparison/Snakefile: [Errno 2] No such file or directory: 'meta/sim_ids.json' File "/home/PycharmProjects/muscat-comparison/Snakefile", line 25, in <module>

I'm also new to snakemake, I installed it via pip3 install snakemake, and then after cloning the muscat-comparison directory I run snakemake, so maybe I missed some important steps? In the directory, I also can't find the folders data, meta and results, is this on purpose?

If more information is needed just let me know. Thanks in advance. Erika

[HelenaLC]

Hi Erika, you're correct. The current workflow will fail initially due to missing directories. I will add a script that will assure all required files/directories exist when running the workflow for the 1st time (e.g., besides the missing directories, simulation/method/run-mode parameters need to have be generated). The script is already on the devel branch. If you're impatient, you could run setup.R from there. Snakemake should run cleanly after this, and, importantly, the workflow will recognise changes to the simulation, method, and run-mode parameters in the future & rerun all affected jobs.

[erikadudki]

Thanks a lot, this works now. But now, the following error appears for me: Missing input files for rule prep_sce: data/raw_data/sce0_kang.rds, where can I find the raw data files? Should they have been created, or can I download them from somewhere?

[erikadudki]

Hi, I generated the raw data files now with the help of your muscat-vignette, with the commands:

eh <- ExperimentHub()
query(eh, "Kang")
(sce <- eh[['EH2259']])

and then saving the generated sce file to a .rds file. I don't know if this should be done, or if the raw data files should be generated automatically while running the snakemake workflow?

Now, I tried running snakemake again (I'm working in the devel branch) and the following error occurs:

[Fri Sep 20 12:20:00 2019] 
rule prep_sce:
    input: scripts/prep_kang.R, data/raw_data/sce0_kang.rds
    output: data/raw_data/sce_kang.rds
    jobid: 2341
    wildcards: did=kang
    priority: 100 

[Fri Sep 20 12:20:00 2019]
Error in rule prep_sce:
    jobid: 2341
    output: data/raw_data/sce_kang.rds

RuleException:
NameError in line 76 of /home/erika/Documents/Projects/Muscat/muscat-comparison/Snakefile:
The name 'input' is unknown in this context. Please make sure that you defined that 
variable. Also note that braces not used for variable access have to be escaped by 
repeating them, i.e. {{print $1}}
  File "/home/erika/Documents/Projects/Muscat/muscat-comparison/Snakefile", 
line 76, in __rule_prep_sce
 File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

Do you have an idea, why this error occurs?

Thank you in advance for your help. Erika