Runtime Error: Hbblits failed

[levipierce]

I keep getting this error and I have tried a couple different fastas and tried d/l the uniclust30 database again...don't really have any ideas given the output?

I0722 19:43:08.561397 140389550937920 run_docker.py:180] RuntimeError: HHblits failed I0722 19:43:08.561503 140389550937920 run_docker.py:180] stdout: I0722 19:43:08.561609 140389550937920 run_docker.py:180] I0722 19:43:08.561714 140389550937920 run_docker.py:180] I0722 19:43:08.561820 140389550937920 run_docker.py:180] stderr: I0722 19:43:08.561925 140389550937920 run_docker.py:180] - 19:41:03.746 INFO: Searching 65983866 column state sequences. I0722 19:43:08.562031 140389550937920 run_docker.py:180] I0722 19:43:08.562135 140389550937920 run_docker.py:180] - 19:41:04.762 INFO: Searching 15161831 column state sequences. I0722 19:43:08.562241 140389550937920 run_docker.py:180] I0722 19:43:08.562347 140389550937920 run_docker.py:180] - 19:41:04.850 INFO: /mnt/fasta_path_0/WT.fasta is in A2M, A3M or FASTA format I0722 19:43:08.562452 140389550937920 run_docker.py:180] I0722 19:43:08.562558 140389550937920 run_docker.py:180] - 19:41:04.851 INFO: Iteration 1 I0722 19:43:08.562664 140389550937920 run_docker.py:180] I0722 19:43:08.562769 140389550937920 run_docker.py:180] - 19:41:04.993 INFO: Prefiltering database I0722 19:43:08.562874 140389550937920 run_docker.py:180] I0722 19:43:08.562980 140389550937920 run_docker.py:180] - 19:42:36.756 INFO: HMMs passed 1st prefilter (gapless profile-profile alignment) : 1095737 I0722 19:43:08.563086 140389550937920 run_docker.py:180] I0722 19:43:08.563192 140389550937920 run_docker.py:180] - 19:43:05.279 INFO: HMMs passed 1st prefilter (gapless profile-profile alignment) : 292047 I0722 19:43:08.563299 140389550937920 run_docker.py:180] I0722 19:43:08.563404 140389550937920 run_docker.py:180] - 19:43:06.166 INFO: HMMs passed 2nd prefilter (gapped profile-profile alignment) : 4026 I0722 19:43:08.563532 140389550937920 run_docker.py:180] I0722 19:43:08.563639 140389550937920 run_docker.py:180] - 19:43:06.166 INFO: HMMs passed 2nd prefilter and not found in previous iterations : 4026 I0722 19:43:08.563744 140389550937920 run_docker.py:180] I0722 19:43:08.563849 140389550937920 run_docker.py:180] - 19:43:06.166 INFO: Scoring 4026 HMMs using HMM-HMM Viterbi alignment I0722 19:43:08.563955 140389550937920 run_docker.py:180] I0722 19:43:08.564060 140389550937920 run_docker.py:180] - 19:43:06.520 INFO: Alternative alignment: 0

[milot-mirdita]

Could you post the full log please? Your log doesn't include the error message.

[levipierce]

Hi milot, yeah that is what is annoying that is the end of the output? There were a few more lines dumped to the terminal I included here but nothing useful? Any ideas?

Thanks, ~LP

I0722 20:08:09.345611 140299376531264 run_docker.py:180] stderr: I0722 20:08:09.345714 140299376531264 run_docker.py:180] - 20:06:04.013 INFO: Searching 65983866 column state sequences. I0722 20:08:09.345817 140299376531264 run_docker.py:180] I0722 20:08:09.345920 140299376531264 run_docker.py:180] - 20:06:05.029 INFO: Searching 15161831 column state sequences. I0722 20:08:09.346023 140299376531264 run_docker.py:180] I0722 20:08:09.346126 140299376531264 run_docker.py:180] - 20:06:05.110 INFO: /mnt/fasta_path_0/WT.fasta is in A2M, A3M or FASTA format I0722 20:08:09.346228 140299376531264 run_docker.py:180] I0722 20:08:09.346330 140299376531264 run_docker.py:180] - 20:06:05.110 INFO: Iteration 1 I0722 20:08:09.346432 140299376531264 run_docker.py:180] I0722 20:08:09.346534 140299376531264 run_docker.py:180] - 20:06:05.253 INFO: Prefiltering database I0722 20:08:09.346637 140299376531264 run_docker.py:180] I0722 20:08:09.346739 140299376531264 run_docker.py:180] - 20:07:39.225 INFO: HMMs passed 1st prefilter (gapless profile-profile alignment) : 1095737 I0722 20:08:09.346842 140299376531264 run_docker.py:180] I0722 20:08:09.346945 140299376531264 run_docker.py:180] - 20:08:06.155 INFO: HMMs passed 1st prefilter (gapless profile-profile alignment) : 292047 I0722 20:08:09.347048 140299376531264 run_docker.py:180] I0722 20:08:09.347150 140299376531264 run_docker.py:180] - 20:08:07.061 INFO: HMMs passed 2nd prefilter (gapped profile-profile alignment) : 4026 I0722 20:08:09.347253 140299376531264 run_docker.py:180] I0722 20:08:09.347355 140299376531264 run_docker.py:180] - 20:08:07.061 INFO: HMMs passed 2nd prefilter and not found in previous iterations : 4026 I0722 20:08:09.347481 140299376531264 run_docker.py:180] I0722 20:08:09.347585 140299376531264 run_docker.py:180] - 20:08:07.061 INFO: Scoring 4026 HMMs using HMM-HMM Viterbi alignment I0722 20:08:09.347689 140299376531264 run_docker.py:180] I0722 20:08:09.347792 140299376531264 run_docker.py:180] - 20:08:07.401 INFO: Alternative alignment: 0 I0722 20:08:09.347895 140299376531264 run_docker.py:180] I0722 20:08:09.347997 140299376531264 run_docker.py:180] I0722 20:08:09.348099 140299376531264 run_docker.py:180]

[levipierce]

Here is the complete output let me know if you see something jump out...I haven't spotted it yet...

/mnt/alphafold/alphafold$ python3 docker/run_docker.py --fasta_paths=/home/ubuntu/WT.fasta --max_template_date=2020-05-14 I0722 19:44:21.344556 140299376531264 run_docker.py:114] Mounting /home/ubuntu -> /mnt/fasta_path_0 I0722 19:44:21.344772 140299376531264 run_docker.py:114] Mounting /mnt/alphafold/db/uniref90 -> /mnt/uniref90_database_path I0722 19:44:21.344918 140299376531264 run_docker.py:114] Mounting /mnt/alphafold/db/mgnify -> /mnt/mgnify_database_path I0722 19:44:21.345051 140299376531264 run_docker.py:114] Mounting /mnt/alphafold/db/uniclust30/uniclust30_2018_08 -> /mnt/uniclust30_database_path I0722 19:44:21.345181 140299376531264 run_docker.py:114] Mounting /mnt/alphafold/db/bfd -> /mnt/bfd_database_path I0722 19:44:21.345305 140299376531264 run_docker.py:114] Mounting /mnt/alphafold/db/pdb70 -> /mnt/pdb70_database_path I0722 19:44:21.345427 140299376531264 run_docker.py:114] Mounting /mnt/alphafold -> /mnt/data_dir I0722 19:44:21.345547 140299376531264 run_docker.py:114] Mounting /mnt/alphafold/db/pdb_mmcif -> /mnt/template_mmcif_dir I0722 19:44:21.345667 140299376531264 run_docker.py:114] Mounting /mnt/alphafold/db/pdb_mmcif -> /mnt/obsolete_pdbs_path I0722 19:44:27.013806 140299376531264 run_docker.py:180] /opt/conda/lib/python3.8/site-packages/absl/flags/_validators.py:203: UserWarning: Flag --preset has a non-None default value; therefore, mark_flag_as_required will pass even if flag is not specified in the command line! I0722 19:44:27.014122 140299376531264 run_docker.py:180] warnings.warn( I0722 19:44:27.474019 140299376531264 run_docker.py:180] I0722 19:44:27.472950 139991364675392 templates.py:880] Using precomputed obsolete pdbs /mnt/obsolete_pdbs_path/obsolete.dat. I0722 19:44:28.876521 140299376531264 run_docker.py:180] I0722 19:44:28.875633 139991364675392 tpu_client.py:54] Starting the local TPU driver. I0722 19:44:28.876889 140299376531264 run_docker.py:180] I0722 19:44:28.876029 139991364675392 xla_bridge.py:231] Unable to initialize backend 'tpu_driver': Not found: Unable to find driver in registry given worker: local:// I0722 19:44:29.108481 140299376531264 run_docker.py:180] I0722 19:44:29.107677 139991364675392 xla_bridge.py:231] Unable to initialize backend 'tpu': Invalid argument: TpuPlatform is not available. I0722 19:44:39.831663 140299376531264 run_docker.py:180] I0722 19:44:39.830781 139991364675392 run_alphafold.py:238] Have 5 models: ['model_1', 'model_2', 'model_3', 'model_4', 'model_5'] I0722 19:44:39.832005 140299376531264 run_docker.py:180] I0722 19:44:39.831006 139991364675392 run_alphafold.py:251] Using random seed 3224152700917702021 for the data pipeline I0722 19:44:39.832245 140299376531264 run_docker.py:180] I0722 19:44:39.831795 139991364675392 jackhmmer.py:119] Launching subprocess "/usr/bin/jackhmmer -o /dev/null -A /tmp/tmps_ary7bg/output.sto --noali --F1 0.0005 --F2 5e-05 --F3 5e-07 --incE 0.0001 -E 0.0001 --cpu 8 -N 1 /mnt/fasta_path_0/WT.fasta /mnt/uniref90_database_path/uniref90.fasta" I0722 19:44:39.958017 140299376531264 run_docker.py:180] I0722 19:44:39.956998 139991364675392 utils.py:36] Started Jackhmmer (uniref90.fasta) query I0722 19:53:52.531643 140299376531264 run_docker.py:180] I0722 19:53:52.530590 139991364675392 utils.py:40] Finished Jackhmmer (uniref90.fasta) query in 552.573 seconds I0722 19:53:52.700269 140299376531264 run_docker.py:180] I0722 19:53:52.699383 139991364675392 jackhmmer.py:119] Launching subprocess "/usr/bin/jackhmmer -o /dev/null -A /tmp/tmp3qu0u7b4/output.sto --noali --F1 0.0005 --F2 5e-05 --F3 5e-07 --incE 0.0001 -E 0.0001 --cpu 8 -N 1 /mnt/fasta_path_0/WT.fasta /mnt/mgnify_database_path/mgy_clusters.fa" I0722 19:53:52.831708 140299376531264 run_docker.py:180] I0722 19:53:52.830728 139991364675392 utils.py:36] Started Jackhmmer (mgy_clusters.fa) query I0722 20:04:24.883711 140299376531264 run_docker.py:180] I0722 20:04:24.882228 139991364675392 utils.py:40] Finished Jackhmmer (mgy_clusters.fa) query in 632.051 seconds I0722 20:04:27.317280 140299376531264 run_docker.py:180] I0722 20:04:27.316398 139991364675392 hhsearch.py:76] Launching subprocess "/usr/bin/hhsearch -i /tmp/tmprvqjrhp5/query.a3m -o /tmp/tmprvqjrhp5/output.hhr -maxseq 1000000 -d /mnt/pdb70_database_path/pdb70" I0722 20:04:27.451295 140299376531264 run_docker.py:180] I0722 20:04:27.450233 139991364675392 utils.py:36] Started HHsearch query I0722 20:05:30.228377 140299376531264 run_docker.py:180] I0722 20:05:30.227458 139991364675392 utils.py:40] Finished HHsearch query in 62.777 seconds I0722 20:05:37.575266 140299376531264 run_docker.py:180] I0722 20:05:37.574331 139991364675392 hhblits.py:128] Launching subprocess "/usr/bin/hhblits -i /mnt/fasta_path_0/WT.fasta -cpu 4 -oa3m /tmp/tmp0ulakgl6/output.a3m -o /dev/null -n 3 -e 0.001 -maxseq 1000000 -realign_max 100000 -maxfilt 100000 -min_prefilter_hits 1000 -d /mnt/bfd_database_path/bfd_metaclust_clu_complete_id30_c90_final_seq.sorted_opt -d /mnt/uniclust30_database_path/uniclust30_2018_08" I0722 20:05:37.712370 140299376531264 run_docker.py:180] I0722 20:05:37.711311 139991364675392 utils.py:36] Started HHblits query I0722 20:08:09.339020 140299376531264 run_docker.py:180] I0722 20:08:09.337852 139991364675392 utils.py:40] Finished HHblits query in 151.626 seconds I0722 20:08:09.339367 140299376531264 run_docker.py:180] E0722 20:08:09.338106 139991364675392 hhblits.py:138] HHblits failed. HHblits stderr begin: I0722 20:08:09.339584 140299376531264 run_docker.py:180] E0722 20:08:09.338206 139991364675392 hhblits.py:141] - 20:06:04.013 INFO: Searching 65983866 column state sequences. I0722 20:08:09.339715 140299376531264 run_docker.py:180] E0722 20:08:09.338307 139991364675392 hhblits.py:141] - 20:06:05.029 INFO: Searching 15161831 column state sequences. I0722 20:08:09.339848 140299376531264 run_docker.py:180] E0722 20:08:09.338404 139991364675392 hhblits.py:141] - 20:06:05.110 INFO: /mnt/fasta_path_0/WT.fasta is in A2M, A3M or FASTA format I0722 20:08:09.339990 140299376531264 run_docker.py:180] E0722 20:08:09.338509 139991364675392 hhblits.py:141] - 20:06:05.110 INFO: Iteration 1 I0722 20:08:09.340117 140299376531264 run_docker.py:180] E0722 20:08:09.338588 139991364675392 hhblits.py:141] - 20:06:05.253 INFO: Prefiltering database I0722 20:08:09.340273 140299376531264 run_docker.py:180] E0722 20:08:09.338644 139991364675392 hhblits.py:141] - 20:07:39.225 INFO: HMMs passed 1st prefilter (gapless profile-profile alignment) : 1095737 I0722 20:08:09.340387 140299376531264 run_docker.py:180] E0722 20:08:09.338737 139991364675392 hhblits.py:141] - 20:08:06.155 INFO: HMMs passed 1st prefilter (gapless profile-profile alignment) : 292047 I0722 20:08:09.340512 140299376531264 run_docker.py:180] E0722 20:08:09.338814 139991364675392 hhblits.py:141] - 20:08:07.061 INFO: HMMs passed 2nd prefilter (gapped profile-profile alignment) : 4026 I0722 20:08:09.340633 140299376531264 run_docker.py:180] E0722 20:08:09.338867 139991364675392 hhblits.py:141] - 20:08:07.061 INFO: HMMs passed 2nd prefilter and not found in previous iterations : 4026 I0722 20:08:09.340753 140299376531264 run_docker.py:180] E0722 20:08:09.338946 139991364675392 hhblits.py:141] - 20:08:07.061 INFO: Scoring 4026 HMMs using HMM-HMM Viterbi alignment I0722 20:08:09.340860 140299376531264 run_docker.py:180] E0722 20:08:09.339032 139991364675392 hhblits.py:141] - 20:08:07.401 INFO: Alternative alignment: 0 I0722 20:08:09.340965 140299376531264 run_docker.py:180] E0722 20:08:09.339116 139991364675392 hhblits.py:142] HHblits stderr end I0722 20:08:09.343515 140299376531264 run_docker.py:180] Traceback (most recent call last): I0722 20:08:09.343655 140299376531264 run_docker.py:180] File "/app/alphafold/run_alphafold.py", line 283, in I0722 20:08:09.343774 140299376531264 run_docker.py:180] app.run(main) I0722 20:08:09.343931 140299376531264 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 312, in run I0722 20:08:09.344043 140299376531264 run_docker.py:180] _run_main(main, args) I0722 20:08:09.344152 140299376531264 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main I0722 20:08:09.344260 140299376531264 run_docker.py:180] sys.exit(main(argv)) I0722 20:08:09.344366 140299376531264 run_docker.py:180] File "/app/alphafold/run_alphafold.py", line 255, in main I0722 20:08:09.344471 140299376531264 run_docker.py:180] predict_structure( I0722 20:08:09.344574 140299376531264 run_docker.py:180] File "/app/alphafold/run_alphafold.py", line 115, in predict_structure I0722 20:08:09.344680 140299376531264 run_docker.py:180] feature_dict = data_pipeline.process( I0722 20:08:09.344784 140299376531264 run_docker.py:180] File "/app/alphafold/alphafold/data/pipeline.py", line 151, in process I0722 20:08:09.344887 140299376531264 run_docker.py:180] hhblits_bfd_uniclust_result = self.hhblits_bfd_uniclust_runner.query( I0722 20:08:09.344990 140299376531264 run_docker.py:180] File "/app/alphafold/alphafold/data/tools/hhblits.py", line 143, in query I0722 20:08:09.345094 140299376531264 run_docker.py:180] raise RuntimeError('HHblits failed\nstdout:\n%s\n\nstderr:\n%s\n' % ( I0722 20:08:09.345198 140299376531264 run_docker.py:180] RuntimeError: HHblits failed I0722 20:08:09.345301 140299376531264 run_docker.py:180] stdout: I0722 20:08:09.345405 140299376531264 run_docker.py:180] I0722 20:08:09.345508 140299376531264 run_docker.py:180] I0722 20:08:09.345611 140299376531264 run_docker.py:180] stderr: I0722 20:08:09.345714 140299376531264 run_docker.py:180] - 20:06:04.013 INFO: Searching 65983866 column state sequences. I0722 20:08:09.345817 140299376531264 run_docker.py:180] I0722 20:08:09.345920 140299376531264 run_docker.py:180] - 20:06:05.029 INFO: Searching 15161831 column state sequences. I0722 20:08:09.346023 140299376531264 run_docker.py:180] I0722 20:08:09.346126 140299376531264 run_docker.py:180] - 20:06:05.110 INFO: /mnt/fasta_path_0/WT.fasta is in A2M, A3M or FASTA format I0722 20:08:09.346228 140299376531264 run_docker.py:180] I0722 20:08:09.346330 140299376531264 run_docker.py:180] - 20:06:05.110 INFO: Iteration 1 I0722 20:08:09.346432 140299376531264 run_docker.py:180] I0722 20:08:09.346534 140299376531264 run_docker.py:180] - 20:06:05.253 INFO: Prefiltering database I0722 20:08:09.346637 140299376531264 run_docker.py:180] I0722 20:08:09.346739 140299376531264 run_docker.py:180] - 20:07:39.225 INFO: HMMs passed 1st prefilter (gapless profile-profile alignment) : 1095737 I0722 20:08:09.346842 140299376531264 run_docker.py:180] I0722 20:08:09.346945 140299376531264 run_docker.py:180] - 20:08:06.155 INFO: HMMs passed 1st prefilter (gapless profile-profile alignment) : 292047 I0722 20:08:09.347048 140299376531264 run_docker.py:180] I0722 20:08:09.347150 140299376531264 run_docker.py:180] - 20:08:07.061 INFO: HMMs passed 2nd prefilter (gapped profile-profile alignment) : 4026 I0722 20:08:09.347253 140299376531264 run_docker.py:180] I0722 20:08:09.347355 140299376531264 run_docker.py:180] - 20:08:07.061 INFO: HMMs passed 2nd prefilter and not found in previous iterations : 4026 I0722 20:08:09.347481 140299376531264 run_docker.py:180] I0722 20:08:09.347585 140299376531264 run_docker.py:180] - 20:08:07.061 INFO: Scoring 4026 HMMs using HMM-HMM Viterbi alignment I0722 20:08:09.347689 140299376531264 run_docker.py:180] I0722 20:08:09.347792 140299376531264 run_docker.py:180] - 20:08:07.401 INFO: Alternative alignment: 0 I0722 20:08:09.347895 140299376531264 run_docker.py:180] I0722 20:08:09.347997 140299376531264 run_docker.py:180] I0722 20:08:09.348099 140299376531264 run_docker.py:180]

[milot-mirdita]

Sadly the log isn't super useful. You might want to check the hashes of the downloaded database as in the other thread: https://github.com/soedinglab/hh-suite/issues/277

It might be the same issue. Or any other, maybe try on a machine with more memory?

[levipierce]

Cool milot I am also trying to download a more recent Uniclust as was suggested the older 2018 didn't seem to work with the bfd version in that other reported problem. I also only have 60GB of memory on this machine so I will try to bump that up too!

[levipierce]

Milot I think you were spot on with the memory. I watched it just climb in htop till it barfed. Spinning up a bigger machine!

Thanks again for the help!