google / deepvariant

DeepVariant is an analysis pipeline that uses a deep neural network to call genetic variants from next-generation DNA sequencing data.
BSD 3-Clause "New" or "Revised" License
3.16k stars 713 forks source link

When running, the community docker will pull deeptrio-1.5.0, but I want to run ordinary deepvariant #700

Closed XXH123a closed 1 year ago

XXH123a commented 1 year ago

$ docker --version Docker version 24.0.5, build ced0996 $ sudo docker images google/deepvariant deeptrio-1.5.0 aff53ed783a7 5 months ago 5.99GB google/deepvariant 1.5.0 45f6c7767ff0 5 months ago 6.98GB Although I have pulled the ordinary DV, it will output when I run it with the recommended command,"unable to find image 'google/deepvariant:1.5.0' locally 1.5.0:Pulling from google/deepvariant" Now the task is running. Should I terminate the task, delete the social docker, and reinstall it? Or wait for it to finish running? Dear developers, what do you think?

pgrosu commented 1 year ago

Are you running the docker image with sudo docker run or just docker run? There is a difference sometimes of where it looks for images. For example, you listed them with sudo but are your also running with sudo-privileges?

Thanks, Paul

XXH123a commented 1 year ago

I use sudo docker run,Because I can't run without sudo.

pgrosu commented 1 year ago

If you type sudo docker run 45f6c7767ff0, does that also not launch it? Also you can add your username to the Docker group so you don't need sudo, as shown here -- but do that after we solve your issue:

https://docs.docker.com/engine/install/linux-postinstall/

XXH123a commented 1 year ago

Dear pgrosu, thank you very much for your suggestions. I want to wait for it to finish running and then try it immediately. Although I am not sure whether it is aff53ed783a7 or 45f6c7767ff0, this is my running command `sudo docker run \

-v "/home/xxh/all_data/Deepvariant/input":"/input" \ -v "/home/xxh/all_data/Deepvariant/output":"/output" \ google/deepvariant:"1.5.0" \ /opt/deepvariant/bin/run_deepvariant \ --model_type=WGS \ -ref=/input/bs_filled.fasta \ --reads=/input/aln_sort.bam \ --output_vcf=/output/output.vcf.gz \ --output_gvcf=/output/output.g.vcf.gz \ --num_shards=16 [sudo] password for xxh: Unable to find image 'google/deepvariant:1.5.0' locally 1.5.0: Pulling from google/deepvariant 7608715873ec: Already exists ff9c04d6f4fd: Already exists ccc0633ad137: Already exists 31c4e73f93e5: Already exists 5f25a39487ec: Already exists 634688f8f57d: Already exists ce40c5e2b0ba: Already exists 9eaa0cfabb44: Already exists f71d9ff16a67: Already exists 27ddda9faddb: Already exists 6ba5fd944a25: Already exists bec3e4e06e4d: Already exists dc1469807dcc: Already exists ceb45df4e1ef: Already exists 4f6165ff322d: Already exists f9cefd1876c2: Already exists 931b80517c67: Already exists 7b13ecd8df6e: Already exists 245c9afd7ea9: Already exists 97c2b022ac0f: Already exists 4c15f3639f35: Already exists fa21a1eddf03: Already exists 6419c68a3d65: Already exists 8751b2539913: Already exists 20646815bf33: Already exists 25dc07245f2e: Already exists 6e3dea686609: Already exists dc216b407a52: Already exists c6710cf0efec: Already exists 6a519085af15: Already exists fd35c1634889: Already exists 0e4b2b2ad2db: Already exists 87e7c72faeb5: Already exists 690adc142e08: Already exists abfd217d5088: Already exists 30b033b0505f: Already exists 853ad599972a: Already exists f20c79af8049: Already exists 26703b5b7abd: Already exists f3b33765da79: Already exists c382f9fc227e: Already exists 3a233bad0db5: Already exists f6ac10e59ad4: Already exists 9e1d2d199a37: Already exists b50b4a1202e8: Already exists 1286a89300c9: Already exists 11f1b6d48e7b: Already exists 09154ad67b50: Already exists aad195d6c4df: Already exists f8376ea6a177: Already exists b44e5a321822: Already exists e81a72561181: Already exists e96d0f626428: Already exists bea852b4c2f5: Already exists 13d620954600: Already exists 123b4e4b7a6e: Already exists Digest: sha256:6a27bc877db9191c8fad0fe5f98dce6192fd4eb6392551e1a3add8d01e08af2e Status: Downloaded newer image for google/deepvariant:1.5.0 2023-08-22 01:54:42.917386: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. I0822 01:56:08.344358 140205078546240 run_deepvariant.py:364] Re-using the directory for intermediate results in /tmp/tmp26397nkc

* Intermediate results will be written to /tmp/tmp26397nkc in docker.

Running the command: time seq 0 15 | parallel -q --halt 2 --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/bs_filled.fasta" --reads "/input/aln_sort.bam" --examples "/tmp/tmp26397nkc/make_examples.tfrecord@16.gz" --channels "insert_size" --gvcf "/tmp/tmp26397nkc/gvcf.tfrecord@16.gz" --task {}`

pgrosu commented 1 year ago

Dear @XXH123a,

Congrats! You seem to be running DeepVariant now, which is most important. Don't worry about trying out anything else. The ids aff53ed783a7 and 45f6c7767ff0 are just image ids that are another way to launch DeepTrio and DeepVariant images, as opposed to using the canonical names google/deepvariant:deeptrio-1.5.0 or google/deepvariant:1.5.0.

I'm not seeing the backslashes after each line, but according to the running commands, it seems to have received the necessary parameters. You might have image layers taking up space, but if things are working for you just stick with that for now, in order to complete the analysis, which is more important.

Well done! ~p

XXH123a commented 1 year ago

Successfully run, once again sincerely thank you for your guidance!