Can´t import host fasta in insert_virus.py with singularity flag

[TheBready]

I am not able to import a fasta with the singularity flag. Works fine with the conda flag. I used the human GRCh38 chromosome 1 as host and the HBV_NC_003977.2 as virus.

snakemake --snakefile Snakefile --configfile test/config/HBVhuman.yml --use-singularity --cores 1

Config:

# conduct multiple experiments
# each experiment has multiple conditions, which have different parameteres
# within each experiment, use all possible combinations of all parameters ('conditions')
# and perform a number of replicates of each condition

HBVhumanChr1:

    ### experiment parameters ###
    out_directory: "/home/bready/Workspace/data/out/" # root directory for output
    replicates: 2 # number of replicates for each condition (combination of parameters)

    ### parameters for integration simulation (python script) ###
    # fasta files to use for simulating integrations
    # and names for each 
    hosts: 
        "human" : "/home/bready/Workspace/data/references/GRCh38_human_chr1.fna"
    viruses:
        "HBV" :  "/home/bready/Workspace/data/references/HBV_NC_003977_2.fasta"

    # random seed for first replicate - will be incremented on successive replicates
    # use the same seed for the python script and ART
    initial_seed: 12345
    seed_increment: 123 

     # number of integrations to perform
    int_num: 
        - 10
    min_sep: 
        - 50  
    epi_num:
        - 0
    # probabilities
    p_whole:
        - 0.5
    min_len: 
        - 50 
    p_rearrange:
        - 0.1
    p_delete:
        - 0.1
    lambda_split:
        - 1
    p_overlap:
        - 0.2
    p_gap:
        - 0.2
    lambda_junction:
        - 1
    p_host_deletion:
        - 0.2
    lambda_host_deletion:
        - 20

    ### parameters for read simulation (art) ###
    read_len: 
        - 150
    # fold coverage (-f)
    fcov: 
        - 10 
    # mean fragment length (-m)
    frag_len: 
        - 250
        - 500 
    # fragment length standard deviation (-s)
    frag_std: 
        - 30
    # sequencing system profile for art-illumna
    seq_sys: 
        - "MSv3"

Error message:

(intvi) bready@Bready-Desktop:~/Workspace/intvi_simulation$ snakemake --snakefile Snakefile --configfile test/config/HBVhuman.yml --use-singularity --cores 1
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Conda environments: ignored
Job counts:
        count   jobs
        1       all
        4       annotate_reads
        4       art
        4       convert
        4       simulate_integrations
        1       write_summary
        18

[Wed Nov 18 13:59:07 2020]
rule simulate_integrations:
    input: /home/bready/Workspace/data/references/GRCh38_human_chr1.fna, /home/bready/Workspace/data/references/HBV_NC_003977_2.fasta
    output: /home/bready/Workspace/data/out/HBVhumanChr1/sim_ints/cond1.rep0.fa, /home/bready/Workspace/data/out/HBVhumanChr1/sim_ints/cond1.rep0.int-info.tsv, /home/bready/Workspace/data/out/HBVhumanChr1/sim_ints/cond1.rep0.epi-info.tsv
    jobid: 9
    wildcards: outpath=/home/bready/Workspace/data/out, exp=HBVhumanChr1, samp=cond1.rep0
    resources: mem_mb=1765, time=2:00:00, nodes=1

Activating singularity image /home/bready/Workspace/intvi_simulation/.snakemake/singularity/c567d0aedeec996de62affb41dc79c42.simg
INFO:    Convert SIF file to sandbox...
initialising a new Events object
importing host fasta
Traceback (most recent call last):
  File "scripts/insert_virus.py", line 1880, in <module>
    main(argv[1:])
  File "scripts/insert_virus.py", line 113, in main
    seqs = Events(args.host, args.virus, seed=args.seed, verbose = True, min_len = args.min_len)
  File "scripts/insert_virus.py", line 161, in __init__
    raise OSError("Could not open host fasta")
OSError: Could not open host fasta
INFO:    Cleaning up image...
[Wed Nov 18 13:59:09 2020]
Error in rule simulate_integrations:
    jobid: 9
    output: /home/bready/Workspace/data/out/HBVhumanChr1/sim_ints/cond1.rep0.fa, /home/bready/Workspace/data/out/HBVhumanChr1/sim_ints/cond1.rep0.int-info.tsv, /home/bready/Workspace/data/out/HBVhumanChr1/sim_ints/cond1.rep0.epi-info.tsv
    shell:

                python3 scripts/insert_virus.py                  --host /home/bready/Workspace/data/references/GRCh38_human_chr1.fna             --virus /home/bready/Workspace/data/references/HBV_NC_003977_2.fasta             --ints /home/bready/Workspace/data/out/HBVhumanChr1/sim_ints/cond1.rep0.fa              --int_info /home/bready/Workspace/data/out/HBVhumanChr1/sim_ints/cond1.rep0.int-info.tsv                --epi_info /home/bready/Workspace/data/out/HBVhumanChr1/sim_ints/cond1.rep0.epi-info.tsv                 --int_num 10 --epi_num 0 --min-sep 50 --min-len 50 --p_whole 0.5 --p_rearrange 0.1 --p_delete 0.1 --lambda_split 1 --p_overlap 0.2 --p_gap 0.2 --lambda_junction 1 --p_host_deletion 0.2 --lambda_host_deletion 20 --seed 12468                  --verbose

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/bready/Workspace/intvi_simulation/.snakemake/log/2020-11-18T135907.106881.snakemake.log

[szsctt]

Looks like the python script in this rule can't find the input files - this is probably because the correct directories haven't been bind-mounted when singularity was run.

Can you try adding --singularity-args '-B /home/bready/Workspace' to your snakemake run? There's also an issue with snakemake not passing on singularity args somewhere between versions 5.10 and 5.20 (snakemake/snakemake#193) so if you've got a version somewhere in there then you might need to update

[TheBready]

Okay, thanks. I will try that.

I am using snakemake 5.27.4, so that should be okay.