glennhickey
commented
1 year ago I guess the answer is the same as: https://github.com/ComparativeGenomicsToolkit/cactus/issues/1064#issuecomment-1598570481
See also:
https://github.com/ComparativeGenomicsToolkit/cactus#compiling-binaries-locally
I am trying step1 ,this is my command
cactus-minigraph ./js sample.list ./out.gfa --reference HFTH1 --mgCores 40occuring error: [2023-06-21T09:57:07+0800] [MainThread] [I] [toil.statsAndLogging] Enabling realtime logging in Toil [2023-06-21T09:57:07+0800] [MainThread] [I] [toil.statsAndLogging] Cactus Command: /ngsproject/qmyu/tool/minigraph/cactus/cactus_env/bin/cactus-minigraph ./js sample.list ./out.gfa --reference HFTH1 --mgCores 40 [2023-06-21T09:57:07+0800] [MainThread] [I] [toil.statsAndLogging] Cactus Commit: 4e46adc56e18ca9f3d92f4a1abb66e920839a626 [2023-06-21T09:57:07+0800] [MainThread] [I] [toil.statsAndLogging] Importing file:///ngsproject/qmyu/project/pan-genome/minigraph/MHUP.3.fa [2023-06-21T09:57:11+0800] [MainThread] [I] [toil.statsAndLogging] Importing file:///ngsproject/qmyu/project/pan-genome/minigraph/MHUP.2.fa [2023-06-21T09:57:11+0800] [MainThread] [I] [toil.statsAndLogging] Importing file:///ngsproject/qmyu/project/pan-genome/minigraph/MHUP.1.fa [2023-06-21T09:57:11+0800] [MainThread] [I] [toil.statsAndLogging] Importing file:///ngsproject/qmyu/project/pan-genome/minigraph/MBAC.B.fa [2023-06-21T09:57:11+0800] [MainThread] [I] [toil.statsAndLogging] Importing file:///ngsproject/qmyu/project/pan-genome/minigraph/MBAC.A.fa [2023-06-21T09:57:11+0800] [MainThread] [I] [toil.statsAndLogging] Importing file:///ngsproject/qmyu/project/pan-genome/minigraph/HFTH1.all.chr.fa [2023-06-21T09:57:13+0800] [MainThread] [I] [toil.job] Saving graph of 1 jobs, 1 non-service, 1 new [2023-06-21T09:57:13+0800] [MainThread] [I] [toil.job] Processing job 'minigraph_construct_workflow' kind-minigraph_construct_workflow/instance-sbu76khu v0 [2023-06-21T09:57:13+0800] [MainThread] [I] [toil] Running Toil version 5.11.0-9a04dabb36d6ab13ed1ac7c711dbdc8c71724dc9 on host alice. [2023-06-21T09:57:13+0800] [MainThread] [I] [toil.realtimeLogger] Starting real-time logging. [2023-06-21T09:57:13+0800] [MainThread] [I] [toil.leader] Issued job 'minigraph_construct_workflow' kind-minigraph_construct_workflow/instance-sbu76khu v1 with job batch system ID: 1 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False [2023-06-21T09:57:13+0800] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/337e2da0decc58379a98ae08da5e7959/2ca3/worker_log.txt [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.leader] 0 jobs are running, 0 jobs are issued and waiting to run [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.leader] Issued job 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-3w4hzk_t v1 with job batch system ID: 2 and disk: 4.2 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.leader] Issued job 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-8d1u89se v1 with job batch system ID: 3 and disk: 4.2 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.leader] Issued job 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-k702ls81 v1 with job batch system ID: 4 and disk: 4.2 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.leader] Issued job 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-h39tpe6g v1 with job batch system ID: 5 and disk: 4.3 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.leader] Issued job 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-f3n8qdo9 v1 with job batch system ID: 6 and disk: 4.3 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.leader] Issued job 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-jn7j7yf8 v1 with job batch system ID: 7 and disk: 4.1 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/337e2da0decc58379a98ae08da5e7959/22da/worker_log.txt [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/337e2da0decc58379a98ae08da5e7959/5aa6/worker_log.txt [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/337e2da0decc58379a98ae08da5e7959/da00/worker_log.txt [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/337e2da0decc58379a98ae08da5e7959/83fb/worker_log.txt [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/337e2da0decc58379a98ae08da5e7959/1198/worker_log.txt [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/337e2da0decc58379a98ae08da5e7959/44fb/worker_log.txt [2023-06-21T09:57:15+0800] [MainThread] [I] [toil-rt] 2023-06-21 09:57:15.291675: Running the command: "docker run --interactive --net=host --log-driver=none -u 1008:1008 -v /tmp/337e2da0decc58379a98ae08da5e7959/1198/697d/tmpyph4g8iz:/data --entrypoint /opt/cactus/wrapper.sh --name 50985e56-e266-4d97-947e-24913fd2e926 --rm quay.io/comparative-genomics-toolkit/cactus:4e46adc56e18ca9f3d92f4a1abb66e920839a626 cactus_sanitizeFastaHeaders MHUP.3.fa MHUP.3 -p" [2023-06-21T09:57:15+0800] [MainThread] [I] [toil-rt] 2023-06-21 09:57:15.337420: Running the command: "docker run --interactive --net=host --log-driver=none -u 1008:1008 -v /tmp/337e2da0decc58379a98ae08da5e7959/83fb/fc61/tmpppb1w25s:/data --entrypoint /opt/cactus/wrapper.sh --name 15f0364c-f69b-42b7-9324-8092dc111ef8 --rm quay.io/comparative-genomics-toolkit/cactus:4e46adc56e18ca9f3d92f4a1abb66e920839a626 cactus_sanitizeFastaHeaders MBAC.2.fa MBAC.2 -p" [2023-06-21T09:57:15+0800] [MainThread] [I] [toil-rt] 2023-06-21 09:57:15.355618: Running the command: "docker run --interactive --net=host --log-driver=none -u 1008:1008 -v /tmp/337e2da0decc58379a98ae08da5e7959/22da/4e08/tmpv8_slv4q:/data --entrypoint /opt/cactus/wrapper.sh --name c3c29c6f-ea52-4f33-9779-dd1719a0c204 --rm quay.io/comparative-genomics-toolkit/cactus:4e46adc56e18ca9f3d92f4a1abb66e920839a626 cactus_sanitizeFastaHeaders MBAC.1.fa MBAC.1 -p" [2023-06-21T09:57:15+0800] [MainThread] [I] [toil-rt] 2023-06-21 09:57:15.365769: Running the command: "docker run --interactive --net=host --log-driver=none -u 1008:1008 -v /tmp/337e2da0decc58379a98ae08da5e7959/da00/c18b/tmp6120t_9d:/data --entrypoint /opt/cactus/wrapper.sh --name 42b6d14e-42e1-4434-b8af-65ab608c7846 --rm quay.io/comparative-genomics-toolkit/cactus:4e46adc56e18ca9f3d92f4a1abb66e920839a626 cactus_sanitizeFastaHeaders MHUP.1.fa MHUP.1 -p" [2023-06-21T09:57:15+0800] [MainThread] [I] [toil-rt] 2023-06-21 09:57:15.392128: Running the command: "docker run --interactive --net=host --log-driver=none -u 1008:1008 -v /tmp/337e2da0decc58379a98ae08da5e7959/5aa6/c427/tmpleg4oyuz:/data --entrypoint /opt/cactus/wrapper.sh --name f5d62b78-f1bf-442e-bf13-405e2a0fd2c6 --rm quay.io/comparative-genomics-toolkit/cactus:4e46adc56e18ca9f3d92f4a1abb66e920839a626 cactus_sanitizeFastaHeaders HFTH1.fa HFTH1 -p" [2023-06-21T09:57:15+0800] [MainThread] [I] [toil-rt] 2023-06-21 09:57:15.409987: Running the command: "docker run --interactive --net=host --log-driver=none -u 1008:1008 -v /tmp/337e2da0decc58379a98ae08da5e7959/44fb/4dcb/tmpawumsfrn:/data --entrypoint /opt/cactus/wrapper.sh --name d6a3871e-c5e4-4746-afde-4fc45c647d11 --rm quay.io/comparative-genomics-toolkit/cactus:4e46adc56e18ca9f3d92f4a1abb66e920839a626 cactus_sanitizeFastaHeaders MHUP.2.fa MHUP.2 -p" [2023-06-21T09:57:15+0800] [Thread-1 ] [E] [toil.batchSystems.singleMachine] Got exit code 1 (indicating failure) from job _toil_worker sanitize_fasta_header file:/ngsproject/qmyu/project/pan-genome/minigraph/js kind-sanitize_fasta_header/instance-8d1u89se. [2023-06-21T09:57:15+0800] [MainThread] [W] [toil.leader] Job failed with exit value 1: 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-8d1u89se v1 Exit reason: None [2023-06-21T09:57:15+0800] [MainThread] [W] [toil.leader] The job seems to have left a log file, indicating failure: 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-8d1u89se v2 [2023-06-21T09:57:15+0800] [MainThread] [W] [toil.leader] Log from job "kind-sanitize_fasta_header/instance-8d1u89se" follows: =========> [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2023-06-21T09:57:14+0800] [MainThread] [I] [toil] Running Toil version 5.11.0-9a04dabb36d6ab13ed1ac7c711dbdc8c71724dc9 on host alice. [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.worker] Working on job 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-8d1u89se v1 [2023-06-21T09:57:14+0800] [MainThread] [I] [toil.worker] Loaded body Job('sanitize_fasta_header' kind-sanitize_fasta_header/instance-8d1u89se v1) from description 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-8d1u89se v1 [2023-06-21T09:57:15+0800] [MainThread] [I] [cactus.shared.common] Work dirs: {'/tmp/337e2da0decc58379a98ae08da5e7959/83fb/fc61/tmpppb1w25s'} [2023-06-21T09:57:15+0800] [MainThread] [I] [cactus.shared.common] Docker work dir: /tmp/337e2da0decc58379a98ae08da5e7959/83fb/fc61/tmpppb1w25s [2023-06-21T09:57:15+0800] [MainThread] [I] [cactus.shared.common] Running the command ['docker', 'run', '--interactive', '--net=host', '--log-driver=none', '-u', '1008:1008', '-v', '/tmp/337e2da0decc58379a98ae08da5e7959/83fb/fc61/tmpppb1w25s:/data', '--entrypoint', '/opt/cactus/wrapper.sh', '--name', '15f0364c-f69b-42b7-9324-8092dc111ef8', '--rm', 'quay.io/comparative-genomics-toolkit/cactus:4e46adc56e18ca9f3d92f4a1abb66e920839a626', 'cactus_sanitizeFastaHeaders', 'MBAC.2.fa', 'MBAC.2', '-p'] [2023-06-21T09:57:15+0800] [MainThread] [I] [toil-rt] 2023-06-21 09:57:15.337420: Running the command: "docker run --interactive --net=host --log-driver=none -u 1008:1008 -v /tmp/337e2da0decc58379a98ae08da5e7959/83fb/fc61/tmpppb1w25s:/data --entrypoint /opt/cactus/wrapper.sh --name 15f0364c-f69b-42b7-9324-8092dc111ef8 --rm quay.io/comparative-genomics-toolkit/cactus:4e46adc56e18ca9f3d92f4a1abb66e920839a626 cactus_sanitizeFastaHeaders MBAC.2.fa MBAC.2 -p" [2023-06-21T09:57:15+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Failed job accessed files: [2023-06-21T09:57:15+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/no-job/file-a9aa70aaba4a4737860653788077b40b/MBAC.B.fa' to path '/tmp/337e2da0decc58379a98ae08da5e7959/83fb/fc61/tmpppb1w25s/MBAC.2.fa' Traceback (most recent call last): File "/ngsproject/qmyu/tool/minigraph/cactus/cactus_env/lib/python3.8/site-packages/toil/worker.py", line 403, in workerScript job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer) File "/ngsproject/qmyu/tool/minigraph/cactus/cactus_env/lib/python3.8/site-packages/toil/job.py", line 2774, in _runner returnValues = self._run(jobGraph=None, fileStore=fileStore) File "/ngsproject/qmyu/tool/minigraph/cactus/cactus_env/lib/python3.8/site-packages/toil/job.py", line 2691, in _run return self.run(fileStore) File "/ngsproject/qmyu/tool/minigraph/cactus/cactus_env/lib/python3.8/site-packages/toil/job.py", line 2919, in run rValue = userFunction(*((self,) + tuple(self._args)), **self._kwargs) File "/ngsproject/qmyu/tool/minigraph/cactus/cactus_env/lib/python3.8/site-packages/cactus/preprocessor/checkUniqueHeaders.py", line 70, in sanitize_fasta_header cactus_call(parameters=cmd, outfile=out_fa_path) File "/ngsproject/qmyu/tool/minigraph/cactus/cactus_env/lib/python3.8/site-packages/cactus/shared/common.py", line 886, in cactus_call raise RuntimeError("{}Command {} exited {}: {}".format(sigill_msg, call, process.returncode, out)) RuntimeError: Command ['docker', 'run', '--interactive', '--net=host', '--log-driver=none', '-u', '1008:1008', '-v', '/tmp/337e2da0decc58379a98ae08da5e7959/1198/697d/tmpyph4g8iz:/data', '--entrypoint', '/opt/cactus/wrapper.sh', '--name', '50985e56-e266-4d97-947e-24913fd2e926', '--rm', 'quay.io/comparative-genomics-toolkit/cactus:4e46adc56e18ca9f3d92f4a1abb66e920839a626', 'cactus_sanitizeFastaHeaders', 'MHUP.3.fa', 'MHUP.3', '-p'] exited 125: stderr=docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?. See 'docker run --help'.
<========= [2023-06-21T09:57:15+0800] [MainThread] [W] [toil.job] Due to failure we are reducing the remaining try count of job 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-f3n8qdo9 v2 with ID kind-sanitize_fasta_header/instance-f3n8qdo9 to 1 [2023-06-21T09:57:15+0800] [MainThread] [I] [toil.leader] Issued job 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-f3n8qdo9 v3 with job batch system ID: 9 and disk: 4.3 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False [2023-06-21T09:57:15+0800] [Thread-1 ] [E] [toil.batchSystems.singleMachine] Got exit code 1 (indicating failure) from job _toil_worker sanitize_fasta_header file:/ngsproject/qmyu/project/pan-genome/minigraph/js kind-sanitize_fasta_header/instance-3w4hzk_t. [2023-06-21T09:57:15+0800] [MainThread] [W] [toil.leader] Job failed with exit value 1: 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-3w4hzk_t v1 Exit reason: None ...
Where is the mistake?Thansk!