bind source path does not exist

[lamiris]

I am using MacOS, and getting this error from running

$ ./run_prok_pipeline -i ../test_data/genomes/E_coli_k12_dh10b.fna -od /tmp/demo -v cheetah -rd /dbs

docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /dbs/coding_hmm.lib.bin.

Starting pipeline. If this is your first time running this pipeline there will be some latency as the necessary Docker images are downloaded. This is a one-time cost. Running command: cwl-runner --outdir /tmp/demo /tmp/demo/prok-annotation-cheetah.cwl /tmp/demo/prok-annotation-cheetah.json INFO //anaconda3/bin/cwl-runner 3.0.20200724003302 INFO Resolved '/tmp/demo/prok-annotation-cheetah.cwl' to 'file:///tmp/demo/prok-annotation-cheetah.cwl' INFO [workflow ] start INFO [workflow ] starting step prodigal INFO [step prodigal] start INFO [job prodigal] /private/tmp/docker_tmpjb4vn4wa$ docker \ run \ -i \ --mount=type=bind,source=/private/tmp/docker_tmpjb4vn4wa,target=/CjNJMb \ --mount=type=bind,source=/private/tmp/docker_tmphsuaow29,target=/tmp \ --mount=type=bind,source=/Users/irislam/GALES/test_data/genomes/E_coli_k12_dh10b.fna,target=/var/lib/cwl/stg496a65c7-aef1-4607-9691-07094bdecc16/E_coli_k12_dh10b.fna,readonly \ --workdir=/CjNJMb \ --read-only=true \ --user=501:20 \ --rm \ --env=TMPDIR=/tmp \ --env=HOME=/CjNJMb \ --cidfile=/private/tmp/docker_tmpuu2k3k2p/20200805194833-902396.cid \ jorvis/gales-gce \ prodigal \ -i \ /var/lib/cwl/stg496a65c7-aef1-4607-9691-07094bdecc16/E_coli_k12_dh10b.fna \ -f \ gff \ -o \ prodigal.annotation.gff \ -a \ prodigal.annotation.faa

PRODIGAL v2.6.3 [February, 2016]
Univ of Tenn / Oak Ridge National Lab Doug Hyatt, Loren Hauser, et al.

Request: Single Genome, Phase: Training Reading in the sequence(s) to train...4686137 bp seq created, 50.78 pct GC Locating all potential starts and stops...243046 nodes Looking for GC bias in different frames...frame bias scores: 1.55 0.19 1.26 Building initial set of genes to train from...done! Creating coding model and scoring nodes...done! Examining upstream regions and training starts...done!

Request: Single Genome, Phase: Gene Finding Finding genes in sequence #1 (4686137 bp)...done! INFO [job prodigal] Max memory used: 249MiB INFO [job prodigal] completed success INFO [step prodigal] completed success INFO [workflow ] starting step aragorn INFO [step aragorn] start INFO [job aragorn] /private/tmp/dockertmpflnaxd5$ docker \ run \ -i \ --mount=type=bind,source=/private/tmp/dockertmpflnaxd5,target=/CjNJMb \ --mount=type=bind,source=/private/tmp/docker_tmpd7jqrw9t,target=/tmp \ --mount=type=bind,source=/Users/irislam/GALES/test_data/genomes/E_coli_k12_dh10b.fna,target=/var/lib/cwl/stgbdf3509c-1589-455b-8861-5a3ce58aa412/E_coli_k12_dh10b.fna,readonly \ --workdir=/CjNJMb \ --read-only=true \ --log-driver=none \ --user=501:20 \ --rm \ --env=TMPDIR=/tmp \ --env=HOME=/CjNJMb \ --cidfile=/private/tmp/docker_tmp3jjf8f3j/20200805194847-828875.cid \ jorvis/gales-gce \ aragorn \ -w \ /var/lib/cwl/stgbdf3509c-1589-455b-8861-5a3ce58aa412/E_coli_k12_dh10b.fna > /private/tmp/dockertmpflnaxd5/aragorn.out INFO [job aragorn] Max memory used: 4MiB INFO [job aragorn] completed success INFO [step aragorn] completed success INFO [workflow ] starting step barrnap INFO [step barrnap] start INFO [job barrnap] /private/tmp/docker_tmpqtd4_to9$ docker \ run \ -i \ --mount=type=bind,source=/private/tmp/docker_tmpqtd4_to9,target=/CjNJMb \ --mount=type=bind,source=/private/tmp/docker_tmpkdpj3etm,target=/tmp \ --mount=type=bind,source=/Users/irislam/GALES/test_data/genomes/E_coli_k12_dh10b.fna,target=/var/lib/cwl/stg121da8ed-7210-4262-9238-9b99f4195efd/E_coli_k12_dh10b.fna,readonly \ --workdir=/CjNJMb \ --read-only=true \ --log-driver=none \ --user=501:20 \ --rm \ --env=TMPDIR=/tmp \ --env=HOME=/CjNJMb \ --cidfile=/private/tmp/docker_tmpnafi11uw/20200805194852-162333.cid \ jorvis/gales-gce \ barrnap \ /var/lib/cwl/stg121da8ed-7210-4262-9238-9b99f4195efd/E_coli_k12_dh10b.fna > /private/tmp/docker_tmpqtd4_to9/barnapp.gff [23:48:53] This is barrnap 0.7 [23:48:53] Written by Torsten Seemann torsten.seemann@gmail.com [23:48:53] Obtained from https://github.com/Victorian-Bioinformatics-Consortium/barrnap [23:48:53] Detected operating system: linux [23:48:53] Using HMMER binary: /opt/barrnap/bin/../binaries/linux/nhmmer [23:48:53] Will use 8 threads [23:48:53] Setting evalue cutoff to 1e-06 [23:48:53] Will tag genes < 0.8 of expected length. [23:48:53] Will reject genes < 0.5 of expected length. [23:48:53] Using database: /opt/barrnap/bin/../db/bac.hmm [23:48:53] Scanning /var/lib/cwl/stg121da8ed-7210-4262-9238-9b99f4195efd/E_coli_k12_dh10b.fna for bac rRNA genes... please wait [23:48:53] Command: /opt/barrnap/bin/../binaries/linux/nhmmer --cpu 8 -E 1e-06 --w_length 3878 -o /dev/null --tblout /dev/stdout \/opt\/barrnap\/bin\/..\/db\/bac.hmm \/var\/lib\/cwl\/stg121da8ed-7210-4262-9238-9b99f4195efd\/E_coli_k12_dh10b.fna [23:49:01] Found: 16S_rRNA gi|169887498|gb|CP000948.1| L=1539/1585 3522989..3524527 - 16S ribosomal RNA [23:49:01] Found: 16S_rRNA gi|169887498|gb|CP000948.1| L=1539/1585 2819404..2820942 - 16S ribosomal RNA [23:49:01] Found: 16S_rRNA gi|169887498|gb|CP000948.1| L=1538/1585 197878..199415 + 16S ribosomal RNA [23:49:01] Found: 16S_rRNA gi|169887498|gb|CP000948.1| L=1538/1585 4132477..4134014 + 16S ribosomal RNA [23:49:01] Found: 16S_rRNA gi|169887498|gb|CP000948.1| L=1538/1585 4264382..4265919 + 16S ribosomal RNA [23:49:01] Found: 16S_rRNA gi|169887498|gb|CP000948.1| L=1538/1585 4038754..4040291 + 16S ribosomal RNA [23:49:01] Found: 16S_rRNA gi|169887498|gb|CP000948.1| L=1538/1585 4305869..4307406 + 16S ribosomal RNA [23:49:01] Found: 23S_rRNA gi|169887498|gb|CP000948.1| L=2901/3232 199865..202765 + 23S ribosomal RNA [23:49:01] Found: 23S_rRNA gi|169887498|gb|CP000948.1| L=2901/3232 4307764..4310664 + 23S ribosomal RNA [23:49:01] Found: 23S_rRNA gi|169887498|gb|CP000948.1| L=2902/3232 4134464..4137365 + 23S ribosomal RNA [23:49:01] Found: 23S_rRNA gi|169887498|gb|CP000948.1| L=2901/3232 4266363..4269263 + 23S ribosomal RNA [23:49:01] Found: 23S_rRNA gi|169887498|gb|CP000948.1| L=2901/3232 2816069..2818969 - 23S ribosomal RNA [23:49:01] Found: 23S_rRNA gi|169887498|gb|CP000948.1| L=2901/3232 3519648..3522548 - 23S ribosomal RNA [23:49:01] Found: 23S_rRNA gi|169887498|gb|CP000948.1| L=2901/3232 4040649..4043549 + 23S ribosomal RNA [23:49:01] Found: 5S_rRNA gi|169887498|gb|CP000948.1| L=111/119 202864..202974 + 5S ribosomal RNA [23:49:01] Found: 5S_rRNA gi|169887498|gb|CP000948.1| L=111/119 4043647..4043757 + 5S ribosomal RNA [23:49:01] Found: 5S_rRNA gi|169887498|gb|CP000948.1| L=111/119 2815861..2815971 - 5S ribosomal RNA [23:49:01] Found: 5S_rRNA gi|169887498|gb|CP000948.1| L=111/119 3519440..3519550 - 5S ribosomal RNA [23:49:01] Found: 5S_rRNA gi|169887498|gb|CP000948.1| L=111/119 4137464..4137574 + 5S ribosomal RNA [23:49:01] Found: 5S_rRNA gi|169887498|gb|CP000948.1| L=111/119 3519195..3519305 - 5S ribosomal RNA [23:49:01] Found: 5S_rRNA gi|169887498|gb|CP000948.1| L=111/119 4269361..4269471 + 5S ribosomal RNA [23:49:01] Found: 5S_rRNA gi|169887498|gb|CP000948.1| L=111/119 4310763..4310873 + 5S ribosomal RNA [23:49:01] Found 22 ribosomal RNA features. [23:49:01] Sorting features and outputting GFF3... INFO [job barrnap] Max memory used: 2077MiB INFO [job barrnap] completed success INFO [step barrnap] completed success INFO [workflow ] starting step prodigal2gff3 INFO [step prodigal2gff3] start INFO [job prodigal2gff3] /private/tmp/docker_tmpq1ks1wti$ docker \ run \ -i \ --mount=type=bind,source=/private/tmp/docker_tmpq1ks1wti,target=/CjNJMb \ --mount=type=bind,source=/private/tmp/docker_tmpdlq6d0zn,target=/tmp \ --mount=type=bind,source=/private/tmp/docker_tmpjb4vn4wa/prodigal.annotation.gff,target=/var/lib/cwl/stg86dfef3c-7779-4af5-b091-8a2f8643380f/prodigal.annotation.gff,readonly \ --workdir=/CjNJMb \ --read-only=true \ --user=501:20 \ --rm \ --env=TMPDIR=/tmp \ --env=HOME=/CjNJMb \ --cidfile=/private/tmp/dockertmp7wxm34j/20200805194902-071338.cid \ jorvis/biocode \ convert_prodigal_to_gff3.py \ --input_file \ /var/lib/cwl/stg86dfef3c-7779-4af5-b091-8a2f8643380f/prodigal.annotation.gff \ --output_file \ prodigal.annotation.gff3 INFO [job prodigal2gff3] Max memory used: 0MiB INFO [job prodigal2gff3] completed success INFO [step prodigal2gff3] completed success INFO [workflow ] starting step prodigal2fasta INFO [step prodigal2fasta] start INFO [job prodigal2fasta] /private/tmp/docker_tmpe4u_75mj$ docker \ run \ -i \ --mount=type=bind,source=/private/tmp/docker_tmpe4u_75mj,target=/CjNJMb \ --mount=type=bind,source=/private/tmp/docker_tmprzhhgemk,target=/tmp \ --mount=type=bind,source=/Users/irislam/GALES/test_data/genomes/E_coli_k12_dh10b.fna,target=/var/lib/cwl/stg3052c634-e5ba-401d-9087-45795a459b28/E_coli_k12_dh10b.fna,readonly \ --mount=type=bind,source=/private/tmp/docker_tmpq1ks1wti/prodigal.annotation.gff3,target=/var/lib/cwl/stg8cb00087-ed6f-46ac-bf26-f9e5eaaacc92/prodigal.annotation.gff3,readonly \ --workdir=/CjNJMb \ --read-only=true \ --user=501:20 \ --rm \ --env=TMPDIR=/tmp \ --env=HOME=/CjNJMb \ --cidfile=/private/tmp/docker_tmpxxtclt9s/20200805194904-103119.cid \ jorvis/biocode \ write_fasta_from_gff.py \ --input_file \ /var/lib/cwl/stg8cb00087-ed6f-46ac-bf26-f9e5eaaacc92/prodigal.annotation.gff3 \ --output_file \ prodigal2fasta.faa \ --type \ protein \ --fasta \ /var/lib/cwl/stg3052c634-e5ba-401d-9087-45795a459b28/E_coli_k12_dh10b.fna \ --feature_type \ polypeptide WARN: Encountered unknown codon during translation: CAR WARN: Encountered unknown codon during translation: TRG INFO [job prodigal2fasta] Max memory used: 75MiB INFO [job prodigal2fasta] completed success INFO [step prodigal2fasta] completed success INFO [workflow ] starting step split_multifasta INFO [step split_multifasta] start INFO [job split_multifasta] /private/tmp/docker_tmpv9bsha56$ docker \ run \ -i \ --mount=type=bind,source=/private/tmp/docker_tmpv9bsha56,target=/CjNJMb \ --mount=type=bind,source=/private/tmp/docker_tmp890sxazk,target=/tmp \ --mount=type=bind,source=/private/tmp/docker_tmpe4u_75mj/prodigal2fasta.faa,target=/var/lib/cwl/stgfd502c32-66a9-4ba3-bc49-92e167faebd8/prodigal2fasta.faa,readonly \ --workdir=/CjNJMb \ --read-only=true \ --user=501:20 \ --rm \ --env=TMPDIR=/tmp \ --env=HOME=/CjNJMb \ --cidfile=/private/tmp/docker_tmpxyyuv1_u/20200805194908-385087.cid \ jorvis/biocode \ split_fasta_into_even_files.py \ --input_file \ /var/lib/cwl/stgfd502c32-66a9-4ba3-bc49-92e167faebd8/prodigal2fasta.faa \ --file_count \ 1 \ --output_directory \ ./ INFO: Output directory is: (./) INFO: There were 4401 records found in the input file. INFO: 1 file will be created. INFO: Most files will have 4401 records in each. INFO [job split_multifasta] Max memory used: 0MiB INFO [job split_multifasta] completed success INFO [step split_multifasta] completed success INFO [workflow ] starting step merge_models INFO [step merge_models] start INFO [job merge_models] /private/tmp/docker_tmpl27lnsdp$ docker \ run \ -i \ --mount=type=bind,source=/private/tmp/docker_tmpl27lnsdp,target=/CjNJMb \ --mount=type=bind,source=/private/tmp/docker_tmp03fpo3ox,target=/tmp \ --mount=type=bind,source=/private/tmp/dockertmpflnaxd5/aragorn.out,target=/var/lib/cwl/stg9349c58f-69b4-4202-8898-589f1f28a739/aragorn.out,readonly \ --mount=type=bind,source=/private/tmp/docker_tmpqtd4_to9/barnapp.gff,target=/var/lib/cwl/stg377693b6-b7e8-4874-b1b6-eabea42041f4/barnapp.gff,readonly \ --mount=type=bind,source=/Users/irislam/GALES/test_data/genomes/E_coli_k12_dh10b.fna,target=/var/lib/cwl/stg11449c4c-34d4-490c-9cb4-3eddf9052488/E_coli_k12_dh10b.fna,readonly \ --mount=type=bind,source=/private/tmp/docker_tmpq1ks1wti/prodigal.annotation.gff3,target=/var/lib/cwl/stge6576cb5-af17-4acb-b2bd-0c0bf059650b/prodigal.annotation.gff3,readonly \ --workdir=/CjNJMb \ --read-only=true \ --user=501:20 \ --rm \ --env=TMPDIR=/tmp \ --env=HOME=/CjNJMb \ --cidfile=/private/tmp/docker_tmpzsd22kta/20200805194909-970498.cid \ jorvis/biocode \ merge_predicted_gff3.py \ -m \ /var/lib/cwl/stge6576cb5-af17-4acb-b2bd-0c0bf059650b/prodigal.annotation.gff3 \ -o \ merged_structures.gff3 \ -b \ /var/lib/cwl/stg377693b6-b7e8-4874-b1b6-eabea42041f4/barnapp.gff \ -a \ /var/lib/cwl/stg9349c58f-69b4-4202-8898-589f1f28a739/aragorn.out \ -g \ /var/lib/cwl/stg11449c4c-34d4-490c-9cb4-3eddf9052488/E_coli_k12_dh10b.fna INFO [job merge_models] Max memory used: 57MiB INFO [job merge_models] completed success INFO [step merge_models] completed success INFO [workflow ] starting step hmmscan INFO [step hmmscan] start INFO [job hmmscan] /private/tmp/docker_tmptgr3scsp$ docker \ run \ -i \ --mount=type=bind,source=/private/tmp/docker_tmptgr3scsp,target=/CjNJMb \ --mount=type=bind,source=/private/tmp/docker_tmpgpig9az6,target=/tmp \ --mount=type=bind,source=/dbs/coding_hmm.lib.bin,target=/var/lib/cwl/stg6f2d4b07-973b-4c83-b8c0-ea3fbf708518/coding_hmm.lib.bin,readonly \ --mount=type=bind,source=/dbs/coding_hmm.lib.bin.h3f,target=/var/lib/cwl/stg6f2d4b07-973b-4c83-b8c0-ea3fbf708518/coding_hmm.lib.bin.h3f,readonly \ --mount=type=bind,source=/dbs/coding_hmm.lib.bin.h3i,target=/var/lib/cwl/stg6f2d4b07-973b-4c83-b8c0-ea3fbf708518/coding_hmm.lib.bin.h3i,readonly \ --mount=type=bind,source=/dbs/coding_hmm.lib.bin.h3m,target=/var/lib/cwl/stg6f2d4b07-973b-4c83-b8c0-ea3fbf708518/coding_hmm.lib.bin.h3m,readonly \ --mount=type=bind,source=/dbs/coding_hmm.lib.bin.h3p,target=/var/lib/cwl/stg6f2d4b07-973b-4c83-b8c0-ea3fbf708518/coding_hmm.lib.bin.h3p,readonly \ --mount=type=bind,source=/private/tmp/docker_tmpv9bsha56/prodigal2fasta.faa.part1,target=/var/lib/cwl/stg2969bea2-af95-433b-9b53-859a6e405fd9/prodigal2fasta.faa.part1,readonly \ --workdir=/CjNJMb \ --read-only=true \ --user=501:20 \ --rm \ --env=TMPDIR=/tmp \ --env=HOME=/CjNJMb \ --cidfile=/private/tmp/docker_tmpa09hml9o/20200805194913-558408.cid \ jorvis/gales-gce \ hmmscan \ --acc \ --cut_ga \ --cpu \ 1 \ -o \ hmmscan.out \ /var/lib/cwl/stg6f2d4b07-973b-4c83-b8c0-ea3fbf708518/coding_hmm.lib.bin \ /var/lib/cwl/stg2969bea2-af95-433b-9b53-859a6e405fd9/prodigal2fasta.faa.part1 docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /dbs/coding_hmm.lib.bin. See 'docker run --help'.

[lamiris]

I configured shared paths from Docker -> Preferences... -> File Sharing and added the GALES path, but still having the same error...

[jorvis]

Right before you run that command, what does this show?

$ ls /dbs/coding_hmm.lib.bin

[lamiris]

(base) Iriss-Air:~ irislam$ cd GALES/bin (base) Iriss-Air:bin irislam$ ls /dbs/coding_hmm.lib.bin /dbs/coding_hmm.lib.bin

[lamiris]

It turns out my MacOS version was too old and need to be updated. It worked out at the end, but with the new MacOS it requires user to list out the location instead of just "/dbs".

[jorvis]

Can you elaborate here for any future readers? Passing /dbs is a full path location, so if that directory was present it should have worked. Of course, you can make that path anywhere else you prefer it to be, such as a directory in your home area too. What updates were needed and what command ultimately worked?

[lamiris]

I have a pretty old MacBook (from 2012) and haven't updated it for a while cause I didn't think it have enough resources and been able to do what I needed it to do so far. So my original version was 10.12 and docker require 10.13 or above so I needed to updates to 10.15 for docker to work properly.

For this new MacOS version, the path needs to be type out as "/USER/GALES/bin/dbs" instead of "/dbs" as the full path location. Or else I would get error saying the reference files were not found on your system.

So for downloading reference, command needs to be like this

$ ./download_reference_data -rd /Users/irislam/GALES/bin/dbs -p prok-cheetah And to run, the command needs to be like this. ./run_prok_pipeline -i/Users/irislam/Desktop/GCA_004014775.2_ASM401477v2_genomic.fna -od /tmp/demo -v cheetah -rd /Users/irislam/GALES/bin/dbs