search

google-deepmind / alphafold

Open source code for AlphaFold 2.
Apache License 2.0
12.74k stars 2.26k forks source link

INVALID_ARGUMENT: invalid shape type #864

Open dinahoch opened 11 months ago

dinahoch commented 11 months ago

Hi, I am running AlphaFold for a large number of sequences and have received this error message for several of them. I've increased the memory to a ridiculous level and still seeing the same error. I would appreciate any help on how to proceed. Thank you in advance!

UID: readonly variable
alias: could not parse "alias | /usr/bin/which --tty-only --read-alias --show-dot --show-tilde": 1:7: | is not a valid word/usr/bin/basename: missing operand
Try '/usr/bin/basename --help' for more information.
I1120 15:17:41.865278 47817146148992 templates.py:857] Using precomputed obsolete pdbs obsolete.dat.
I1120 15:17:42.013029 47817146148992 xla_bridge.py:173] Remote TPU is not linked into jax; skipping remote TPU.
I1120 15:17:42.013184 47817146148992 xla_bridge.py:357] Unable to initialize backend 'tpu_driver': Could not initialize backend 'tpu_driver'
I1120 15:17:42.134010 47817146148992 xla_bridge.py:357] Unable to initialize backend 'rocm': NOT_FOUND: Could not find registered platform with name: "rocm". Available platform names are: Host Interpreter CUDA
I1120 15:17:42.134352 47817146148992 xla_bridge.py:357] Unable to initialize backend 'tpu': module 'jaxlib.xla_extension' has no attribute 'get_tpu_client'
I1120 15:17:42.134422 47817146148992 xla_bridge.py:357] Unable to initialize backend 'plugin': xla_extension has no attributes named get_plugin_device_client. Compile TensorFlow with //tensorflow/compiler/xla/python:enable_plugin_device set to true (defaults to false) to enable this.
I1120 15:17:45.869056 47817146148992 run_alphafold.py:425] Have 5 models: ['model_1_pred_0', 'model_2_pred_0', 'model_3_pred_0', 'model_4_pred_0', 'model_5_pred_0']
I1120 15:17:45.871369 47817146148992 run_alphafold.py:439] Using random seed 1001343844204186550 for the data pipeline
I1120 15:17:45.871776 47817146148992 run_alphafold.py:186] Predicting VR001165618
W1120 15:17:45.886350 47817146148992 pipeline.py:100] Reading MSA from file Above-500-output/VR001165618/msas/uniref90_hits.sto
W1120 15:17:45.938289 47817146148992 pipeline.py:100] Reading MSA from file Above-500-output/VR001165618/msas/mgnify_hits.sto
I1120 15:17:46.236004 47817146148992 hhsearch.py:85] Launching subprocess "/app/software/HH-suite/3.3.0-gompi-2021b/bin/hhsearch -i /tmp/tmpujjvvg_0/query.a3m -o /tmp/tmpujjvvg_0/output.hhr -maxseq 1000000 -d /shareDB/alphafold/pdb70/pdb70"
I1120 15:17:46.254892 47817146148992 utils.py:36] Started HHsearch query
I1120 15:24:16.356677 47817146148992 utils.py:40] Finished HHsearch query in 390.102 seconds
W1120 15:24:17.366595 47817146148992 pipeline.py:100] Reading MSA from file Above-500-output/VR001165618/msas/bfd_uniref_hits.a3m
I1120 15:24:17.384700 47817146148992 templates.py:878] Searching for template for: MSERRWSVAPIPLTTSSIFEEKLQRDQMKLVNEQMAEIPFAKNMPAYGTAIDAPMVFQDMDTGEYVKGMLEYGEASRQQLKGDYARGFLATDAGFRQQFSTQVVGGRDLSLYNVDSEKLDLLEIPSHVSEQLLSGMKGPQYNLALWDYRHAHDVPIIDLFASGPEDFLNMEPPEGWSTKAAINEFKERDPKLYALMATTLGGDAGIEEITRDSRNPVEFFYDLNRRQQLLAITQSQQMFEDRAGFISSNYEGLKNFVVNGILDDPDMAASFALSVGLAAFTGGTSLLAGAAFKIGKAAKTAHRFNKVLGSGRAVGRGLAISAKHARTAQVLLPENLGPTLVRTVLFKEKYAKMGGVAKFGVNRLSDSVEGTISGALAETFNQKRKKDLGLIDEMSYAAIASEALTEAALSPLINPIFGGVFKGIGMVASVPGTIMSQAFANTLGKRELGKYLQETTQTALSMFNPQAMLDSMDVYEGKETLITNINSLLGEEESLKDGLELESTLIPILETVRATSGLDSPKFLDALNVAVENIITARSASQSANKEMTGAELGVSIITQLATENAEMATTLGEKGFEHIMAVISNQLTMITNAKKEGLSVEEYFSRVVEDGTYFDLLPEVLKQEVYKKAGGEEAAKKLGDTKLREIAAEVQLDIIKESKKIHTEFKKRVESTTKEVNEVNESVQNISEATGIPIPPRVDPSPEPTVTPRPRSNEGATDTAPAEEQSSDEAQAPVDDLTEQHRKETDLTETQKELINIKEELMKRRRDARNAETVDRKELKEIDKNIKSVDALLTRIETLRKEVTNSINAYLTESGQSKELIELGRTLRKLTEQIDRAMEKLKPMQNEINQNLEDRSTLSAIFTYFGPDSAEKYPDGVFFENLSPELQETLQRIHDSLKSGTAKTYLAEVIAGKKLSVKEIKSRIVKTLQRKTEQLKEKWENDALVDHIRDEEKKLEELRGKYNVENAKYQQENIKKDLIQMTQVISIAEVAKRKKDEIRSLETRRRSIDELMADKDTVSIQEVYVRLPRKSKLRGELKARLKDTDGDGTITVDELNERLDEDFADSRKNIVSTIGTDTRYSPMSRTRQTKRISAMYGNRYAHEISDEEIYDVDFDIDWLVENNTPGTALRDVKISVEEDHKSFKQWLANLSVVGQTSSNVANDGTISTWRLLSNMPDYFWDLGLPIHIIFRKAMVNPTENAGSETNSTGSEENPWDSVSNLSSVRWRYSEIKSILERWEVISKARLTGYMESFGKELAKEMAKDLLSESNPSGLTNLEATAEYLNEAIVAERNFNTEKFNESGYILSIDRATGFPTEFASKDQGYLYEAGVRNAIRTRVGYMMNNAKFRLFIKEKAGIEIFLTKPGETAEEAQARSVKEIDTITDYIINNLSLVLNEVDATHPQFINFTLTGFGDGSISWSPAQMAGYAIVDTFVEKSVPMKSRLTGIQEATEYANTEEMRITGESVMLEMPREGDINFAAPISLADNRLNVENYKFRLRIKEVLKKAKSEVPLTPKEKKVLYNSIQSRRFKDETDPVADEMDMYLVPQIVGGHAQEEFITRDQAKDFMMELFLDMAQIGHAINQDSMTIANGGSLRFRNTKENPMPLAFANLNLGYDNITPLGSQLAETLALEQMHPDLSGMSEAAATMHIKFVSEFKSKWQAANPDKEWSASVMLDPSNHTDQVASGLMIVQLLSKLGSGEAAVELDKIRLMIDDIDFELLNEDGTAQRNSDGTFVSLALEDVYVETGLNILGHIKDKDKDNKYIQGSGKWLTDVFGWLDQNEITNKDNYREKMAEFEASGDTESVRKGKAIREFFKVPVMRRLYGGGLDSFKLEFMGMTANSREIIDELNGAFGIKLTDAETENLGTVLLSASAMNGVHILDAALGFEGKIRNAAMAFLKVGREQQVETEMWRDVVRSSNIPNQDSGNSQGPTELAPVDGESMHIMNRLDRVKRALEGKIRYLAERTPEGATPENIARIKKKYDNRYGEALRYIEELESKGITIEDKSPHWERLQEILHGGEVGSWARISLFRATNILNASTYKLNMKRMKATADMFGFEDFEPSDWMGLEDHLLYHLSLPTQRSPRSFHKGKHDPHGIFANRVAIQPEQLNEFMAIAEQNAIDDHGEFNQADFDEALASFLTNEEPDAFGMFDIEDNVYKGMSREEVDAELDRLMDMDQLMYMASVDRLPSEIFPDFVSDESDVVVMDTMLREHYEHNKRRFETPSGEKDRTLATEIAREERLLDLGDSGLTILKYRAKIAGGVRSPRVARVLGKPLIPYHRRELSMQEAIGTRDNQDLYMSTVEPQGLLSMTPRYRRSSFFSRGIPVLQRSAVQRKINEVLEPVLAIDPKIQDPKMKGVSVFDTPTGKGLLGWHQPYQRSSLPKAMPLKIDEMIDLGMDDEAGRKAARILNGMRQWASYRDDVGDDLDNPDGLLYWYVVREIERSRAEMLENLPDYSRFSKNEQEKKAHARKYRMQWLEGLHAITHFSRDIKGAESKGMTLADKARTLAVQAADDFDSPESKQRWIDVLTTTITEDNPHGTFLSAPTKLQKSLEFIIPRENVVLMDRIKSVLEDPTATLSTNDLQMLAQMIQSTDFPYHLLTIMYKNEVLPNVLKEFTVTDENGVTSKPYADLEYDVIGEDGSPTGEKAKVWANPNMWENYIQAEHVHSIIANAKVWIEENGADQIIFDTLIEEQLSSNEAQAKQGIKDFAVYLENTNPDIVGDARPAQMRSSFFPEQVLTPTGLVGKGQLALSPAKAMELFGLLESFSLLEKIKRSNWSGQAIGFVKDPVTGKITESKPTSRRIKDGLGHNENDIRKAEVLRSSALERVVGLGGKADVLRATDTTRSVESTGQIEAIDLDSYRRAQHRDDLSAAWMESYISPVSKQLGYMKSTNIDEVFGPSLREELELIIDRARTENTLAKKGEGGLYSNTVMWLAIMLELPPHDVDTLNPRGHLYDTFIGGRFSEETNAKAFATAQKLMRQIKQIGSVDGFYYGLERQAPLISTIREWASNDDFAEATTRDIDIENKSAVSNFLNWLQSNKTTKELEFLGFAEDENGNLVYKKKSKDPELDPLKELEFIINFGLGQERLMVEQDESLMLFGDQESKLNTTGKDLIKLVSKQGKFDDAVRDTQDPHLIRVNNELNNMKDSMTPANFQRFRALIARMYMINPLNLMDLSLDYQNTRGRSFVTRDKEGNFVIRMGGMLAKPMSDPNIALIFAHELSHVGRMRWIENNQGTWRHWVDLYHSKAGRSLLRKIVTAWHGGNYTAQAKQEFESYMENPEEFIAALTSYYLMNDSLPEMNLDYNEGKISGIAGGIVGKIMDFVRRIFMRTSSVLTNFHYQDPRLSQHVENLVQKTLGWDPMAKKSFTEVQNKGTEREMFFWDKRFEDTTPELENDPTQFKQMVIEHNKLLNLGEEGRTPEEANRFWVLDSAINDSTDATHVMTSGITRYDHLVAHQQNMAMFGGINGTLDIGKMLNSMDNMEGDAVSIIQYSSALEYCMEACNREFGDQFQDGVGLTAAKVTKALSKVLPSLSTSKHRGEGAYSHMMVDLTVGKAGANYTWNGPHPFVILLTQLINDDIATSTGQYTNVKGTPSVVRVINDIALYKDSVTSSMAAIGTTLQPFFTRALGQQVSTEKQREIMSAVYEQVAIKSDNPSHEVTLPELQESAMSEEDLKIIHTEITNTADNIRLFMEDYRQAGTELGLFKSTSFRTHIPFKLNRLILESEPRDGFVESMSKMIRDKIISESEGKLDPLVMFAANLIPRFNTTQNLVDDLKKLKRISPEMYGYILHKAAEAHYTQNGQMDESFDTIRGMRQIVFTDLDSSADPQRQANAVELFRLGIISAMNEMSQGRTTAGSLQEFGVNWNNLKDIYITAIDQRIDGERFNTLDSPALDSLIPDHIYYSPKHGQRKRQVFENIDTPVMVHMHNLINRASDGMYFPYDMWSFPSVVEAFQDPAIRPHLNLDPDQMMGDMMKGKGDQLFETQMMVDHFGVFGNVKDLINLFEIVGTRGHGFRNLDGTSVSLGVQKEMIDAASQLRGKHAVTRGINYVADSPSAFESMLTNAAPGLTRTVFGGNLALATFTVETLASTMIELVGRGNFMGSVRSLLSPILGLNPELQKRVGRDLVHTVEALSKGFIPDHEMPANADTEQMLAKIFNFTGRQAMRPAQYMLKSLAIQRAISARNFVVDNLKHNMTKDDSGRPISRLHRLAQLIHQEPLGKDPKAILDRMRRAGFSVFMDGDIQLITSLLTAGLLDIESGSFGILASMVEENTEVEAYYSPGEMLASLARSDIRPSSTEGGRGVYQARKEVIAGLRMVEKQYIEQVLVAPNAFDVFTPKVSSTGEAEGTGALDTLFEIFRRYPMLFVSQHVYRKMNRQSPMKYAFGLTSMLMLDVVYMLALRLSMGAEPEDILEEFEDNPVGFSIGYGVRLPILGRYMGLIGEAIAALPMLAQGKSIRQPGGFIPLGALEAAAKGVLRAGDSVWNNEPHKLQDVLNAMRVVPIAGDRLLALGYYTAAGDGIQRRSNKSSGTGTSRNAETGELQTRHHYGVPEQAFEINEAMNHRVLLEQFGAQMEWPGYSEPKDVPFAGMTRQQYPEDMAEAAPTAPAPTPAPPSEPAPAPAPAPQAEPTMADRIRGQEGSSGKLADALG
I1120 15:24:17.385609 47817146148992 templates.py:718] hit 3khi_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.023809523809523808.
I1120 15:24:17.385685 47817146148992 templates.py:912] Skipped invalid hit 3KHI_A Putative Metal-dependent Hydrolase; A Putative Metal-dependent Hydrolase, Structural; HET: EDO; 1.95A {Klebsiella pneumoniae subsp. pneumoniae MGH 78578}, error: None, warning: None
I1120 15:24:17.385767 47817146148992 templates.py:718] hit 3dl1_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.023809523809523808.
I1120 15:24:17.385817 47817146148992 templates.py:912] Skipped invalid hit 3DL1_A Putative Metal-dependent Hydrolase; YP_001336084.1, A Putative Metal-dependent Hydrolase; 2.2A {Klebsiella pneumoniae subsp. pneumoniae MGH 78578}, error: None, warning: None
I1120 15:24:17.385879 47817146148992 templates.py:718] hit 6ait_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.016516516516516516.
I1120 15:24:17.385925 47817146148992 templates.py:912] Skipped invalid hit 6AIT_A Beta-barrel assembly-enhancing protease (E.C.3.4.-.-); hydrolase, METAL BINDING PROTEIN; HET: TRS; 2.598A {Escherichia coli (strain K12)}, error: None, warning: None
I1120 15:24:17.385982 47817146148992 templates.py:718] hit 6ait_D did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.01673101673101673.
I1120 15:24:17.386028 47817146148992 templates.py:912] Skipped invalid hit 6AIT_D Beta-barrel assembly-enhancing protease (E.C.3.4.-.-); hydrolase, METAL BINDING PROTEIN; HET: TRS; 2.598A {Escherichia coli (strain K12)}, error: None, warning: None
I1120 15:24:17.386082 47817146148992 templates.py:718] hit 6mdw_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.012226512226512226.
I1120 15:24:17.386130 47817146148992 templates.py:912] Skipped invalid hit 6MDW_A SprT-like domain-containing protein Spartan; DPC repair protease, DNA BINDING; HET: ADP, FLC, MLZ; 1.5A {Homo sapiens}, error: None, warning: None
I1120 15:24:17.386184 47817146148992 templates.py:718] hit 6mdx_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.012226512226512226.
I1120 15:24:17.386230 47817146148992 templates.py:912] Skipped invalid hit 6MDX_A SprT-like domain-containing protein Spartan/DNA Complex; DPC repair, protease, DNA BINDING; HET: MSE, PGE, FLC, MLZ; 1.55A {Homo sapiens}, error: None, warning: None
I1120 15:24:17.386287 47817146148992 templates.py:718] hit 4dv8_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.01287001287001287.
I1120 15:24:17.386332 47817146148992 templates.py:912] Skipped invalid hit 4DV8_A Lethal factor (E.C.3.4.24.83); endopeptidase, Zinc dependent, HYDROLASE; HET: 0LX; 1.632A {Bacillus anthracis}, error: None, warning: None
I1120 15:24:17.386386 47817146148992 templates.py:718] hit 4pkv_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.012655512655512655.
I1120 15:24:17.386431 47817146148992 templates.py:912] Skipped invalid hit 4PKV_A Anthrax toxin lethal factor (E.C.3.4.24.83); Anthrax toxin, lethal factor, metalloproteinase; HET: 30R; 2.5A {Bacillus anthracis}, error: None, warning: None
I1120 15:24:17.386488 47817146148992 templates.py:718] hit 6r4z_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.013084513084513085.
I1120 15:24:17.386533 47817146148992 templates.py:912] Skipped invalid hit 6R4Z_A Pro-Pro endopeptidase-1, ACE-GLU-VAL-ASN-PRO; Pro-Pro endopeptidase 1, zinc metallopeptidase; 1.052A {Peptoclostridium difficile}, error: None, warning: None
I1120 15:24:17.386587 47817146148992 templates.py:718] hit 6fpc_C did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.013299013299013299.
I1120 15:24:17.386632 47817146148992 templates.py:912] Skipped invalid hit 6FPC_C PPEP-2; Endopeptidase, Metalloprotease, Zinc, HYDROLASE; HET: CD, SO4; 1.75A {Paenibacillus alvei}, error: None, warning: None
I1120 15:24:17.386685 47817146148992 templates.py:718] hit 4qhf_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.010081510081510082.
I1120 15:24:17.386730 47817146148992 templates.py:912] Skipped invalid hit 4QHF_A Uncharacterized protein MJ1213; Minigluzincin, Proteolytic enzyme, HYDROLASE; 2.1A {Methanocaldococcus jannaschii}, error: None, warning: None
I1120 15:24:17.386791 47817146148992 templates.py:718] hit 4qhj_B did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.010081510081510082.
I1120 15:24:17.386836 47817146148992 templates.py:912] Skipped invalid hit 4QHJ_B Uncharacterized protein MJ1213; Minigluzincin, Proteolytic enzyme, HYDROLASE; HET: ACT; 1.75A {Methanocaldococcus jannaschii}, error: None, warning: None
I1120 15:24:17.386891 47817146148992 templates.py:718] hit 6ncl_c2 did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.011368511368511369.
I1120 15:24:17.386935 47817146148992 templates.py:912] Skipped invalid hit 6NCL_c2 PBCV-1 capsid; tape-measure protein, minor capsid proteins; 3.5A {Paramecium bursaria Chlorella virus 1}, error: None, warning: None
I1120 15:24:17.386990 47817146148992 templates.py:718] hit 6ncl_c3 did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.011368511368511369.
I1120 15:24:17.387034 47817146148992 templates.py:912] Skipped invalid hit 6NCL_c3 PBCV-1 capsid; tape-measure protein, minor capsid proteins; 3.5A {Paramecium bursaria Chlorella virus 1}, error: None, warning: None
I1120 15:24:17.387086 47817146148992 templates.py:718] hit 3cqb_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.007078507078507079.
I1120 15:24:17.387131 47817146148992 templates.py:912] Skipped invalid hit 3CQB_A Probable protease htpX homolog (E.C.3.4.24.-); Heat shock protein HtpX domain; HET: EDO; 1.86A {Vibrio parahaemolyticus RIMD 2210633}, error: None, warning: None
I1120 15:24:17.387182 47817146148992 templates.py:718] hit 4jix_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.006864006864006864.
I1120 15:24:17.387229 47817146148992 templates.py:912] Skipped invalid hit 4JIX_A Uncharacterized protein MJ0123; Hydrolase, Metallopeptidase Zymogen, Minigluzincin; HET: GOL, SO4; 2.0A {Methanocaldococcus jannaschii}, error: None, warning: None
I1120 15:24:17.387288 47817146148992 templates.py:718] hit 4jiu_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.006864006864006864.
I1120 15:24:17.387333 47817146148992 templates.py:912] Skipped invalid hit 4JIU_A Putative uncharacterized protein; Hydrolase, Metallopeptidase Zymogen, Minigluzincin; 1.15A {Pyrococcus abyssi}, error: None, warning: None
I1120 15:24:17.387386 47817146148992 templates.py:718] hit 3c37_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.006864006864006864.
I1120 15:24:17.387431 47817146148992 templates.py:912] Skipped invalid hit 3C37_A Peptidase, M48 family; Q74D82, GsR143A, Peptidase, M48 family; HET: PEG; 1.7A {Geobacter sulfurreducens PCA}, error: None, warning: None
I1120 15:24:17.387484 47817146148992 templates.py:718] hit 2l0r_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.004719004719004719.
I1120 15:24:17.387528 47817146148992 templates.py:912] Skipped invalid hit 2L0R_A Lethal factor (E.C.3.4.24.83); protein, Anthrax Lethal Factor, catalytic; NMR {Bacillus anthracis}, error: None, warning: None
I1120 15:24:17.387580 47817146148992 templates.py:718] hit 5xbv_A did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.003861003861003861.
I1120 15:24:17.387625 47817146148992 templates.py:912] Skipped invalid hit 5XBV_A Wss1p; protease, HYDROLASE; 1.804A {Saccharomyces cerevisiae}, error: None, warning: None
I1120 15:24:17.387678 47817146148992 templates.py:718] hit 5ijn_S did not pass prefilter: Proportion of residues aligned to query too small. Align ratio: 0.005577005577005577.
I1120 15:24:17.387723 47817146148992 templates.py:912] Skipped invalid hit 5IJN_S NUCLEAR PORE COMPLEX PROTEIN NUP155; Nuclear pore complex, Nucleocytoplasmic transport; 21.4A {Homo sapiens}, error: None, warning: None
I1120 15:24:17.897939 47817146148992 pipeline.py:234] Uniref90 MSA size: 226 sequences.
I1120 15:24:17.898144 47817146148992 pipeline.py:235] BFD MSA size: 1 sequences.
I1120 15:24:17.898192 47817146148992 pipeline.py:236] MGnify MSA size: 501 sequences.
I1120 15:24:17.898241 47817146148992 pipeline.py:237] Final (deduplicated) MSA size: 617 sequences.
I1120 15:24:17.898616 47817146148992 pipeline.py:239] Total number of templates (NB: this can include bad templates and is later filtered to top 4): 0.
I1120 15:24:17.906592 47817146148992 run_alphafold.py:217] Running model model_1_pred_0 on VR001165618
I1120 15:24:30.524318 47817146148992 model.py:165] Running predict with shape(feat) = {'aatype': (4, 4662), 'residue_index': (4, 4662), 'seq_length': (4,), 'template_aatype': (4, 4, 4662), 'template_all_atom_masks': (4, 4, 4662, 37), 'template_all_atom_positions': (4, 4, 4662, 37, 3), 'template_sum_probs': (4, 4, 1), 'is_distillation': (4,), 'seq_mask': (4, 4662), 'msa_mask': (4, 508, 4662), 'msa_row_mask': (4, 508), 'random_crop_to_size_seed': (4, 2), 'template_mask': (4, 4), 'template_pseudo_beta': (4, 4, 4662, 3), 'template_pseudo_beta_mask': (4, 4, 4662), 'atom14_atom_exists': (4, 4662, 14), 'residx_atom14_to_atom37': (4, 4662, 14), 'residx_atom37_to_atom14': (4, 4662, 37), 'atom37_atom_exists': (4, 4662, 37), 'extra_msa': (4, 5120, 4662), 'extra_msa_mask': (4, 5120, 4662), 'extra_msa_row_mask': (4, 5120), 'bert_mask': (4, 508, 4662), 'true_msa': (4, 508, 4662), 'extra_has_deletion': (4, 5120, 4662), 'extra_deletion_value': (4, 5120, 4662), 'msa_feat': (4, 508, 4662, 49), 'target_feat': (4, 4662, 22)}
2023-11-20 15:25:54.221381: E external/org_tensorflow/tensorflow/compiler/xla/shape_util.cc:311] INVALID_ARGUMENT: invalid shape type=11, dims=[-1512984064]
2023-11-20 15:25:54.222760: F external/org_tensorflow/tensorflow/tsl/platform/statusor.cc:33] Attempting to fetch value instead of handling error INVALID_ARGUMENT: invalid shape type=11, dims=[-1512984064]
Fatal Python error: Aborted

Current thread 0x00002b7d4c02a880 (most recent call first):
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/jax/_src/dispatch.py", line 994 in backend_compile
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/jax/_src/profiler.py", line 313 in wrapper
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/jax/_src/dispatch.py", line 1054 in compile_or_get_cached
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/jax/_src/dispatch.py", line 1136 in from_xla_computation
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/jax/_src/dispatch.py", line 978 in compile
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/jax/_src/dispatch.py", line 342 in _xla_callable_uncached
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/jax/linear_util.py", line 309 in memoized_fun
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/jax/_src/dispatch.py", line 234 in _xla_call_impl
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/jax/core.py", line 701 in process_call
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/jax/core.py", line 1955 in call_bind
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/jax/core.py", line 1939 in bind
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/jax/_src/api.py", line 606 in cache_miss
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/jax/_src/traceback_util.py", line 162 in reraise_with_filtered_traceback
  File "/app/software/AlphaFold/2.3.0-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/alphafold/model/model.py", line 167 in predict
  File "/home-dir/Alphafold/run_alphafold.py", line 225 in predict_structure
  File "/home-dir/Alphafold/run_alphafold.py", line 444 in main
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/absl/app.py", line 254 in _run_main
  File "/app/software/jax/0.3.23-foss-2021b-CUDA-11.4.1/lib/python3.9/site-packages/absl/app.py", line 308 in run
  File "/home-dir/Alphafold/run_alphafold.py", line 469 in <module>
/scratch/1700486213.355674: line 8: 63254 Aborted                 singularity exec --nv --env XLA_FLAGS="--xla_gpu_force_compilation_parallelism=1" /apps/containers/singularity/AlphaFold-2.3.1-foss-2021b-CUDA-11.4.1.sif /home-dir/Alphafold/run_alphafold.py --use_gpu_relax=false --models_to_relax=none --num_multimer_predictions_per_model=1 --fasta_paths=faa-files/Above-500-faa-files/VR001165618.faa --output_dir=Above-500-output/ --max_template_date=2023-01-01 --model_preset=monomer --use_precomputed_msas=true --uniref90_database_path=/home-dir/AF_database/database.fasta --data_dir=/shareDB/alphafold/ --bfd_database_path=/shareDB/alphafold/bfd/bfd_metaclust_clu_complete_id30_c90_final_seq.sorted_opt --mgnify_database_path=/shareDB/alphafold/mgnify/mgy_clusters_2022_05.fa --obsolete_pdbs_path=obsolete.dat --uniref30_database_path=/shareDB/alphafold/uniref30/UniRef30_2021_03 --template_mmcif_dir=/shareDB/alphafold/pdb_mmcif/mmcif_files --pdb70_database_path=/shareDB/alphafold/pdb70/pdb70

My original command was

# LSBATCH: User input
module load Singularity; singularity exec --nv --env XLA_FLAGS="--xla_gpu_force_compilation_parallelism=1" /apps/containers/singularity/AlphaFold-2.3.1-foss-2021b-CUDA-11.4.1.sif /home-dir/Alphafold/run_alphafold.py --use_gpu_relax=false --models_to_relax=none --num_multimer_predictions_per_model=1 --fasta_paths=faa-files/Above-500-faa-files/VR001165618.faa --output_dir=Above-500-output/ --max_template_date=2023-01-01 --model_preset=monomer --use_precomputed_msas=true --uniref90_database_path=/home-dir/AF_database/database.fasta --data_dir=/shareDB/alphafold/ --bfd_database_path=/shareDB/alphafold/bfd/bfd_metaclust_clu_complete_id30_c90_final_seq.sorted_opt --mgnify_database_path=/shareDB/alphafold/mgnify/mgy_clusters_2022_05.fa --obsolete_pdbs_path=obsolete.dat --uniref30_database_path=/shareDB/alphafold/uniref30/UniRef30_2021_03 --template_mmcif_dir=/shareDB/alphafold/pdb_mmcif/mmcif_files --pdb70_database_path=/shareDB/alphafold/pdb70/pdb70
dinahoch commented 7 months ago

Hi, any resolution for this error?