search

nf-core / proteinfold

Protein 3D structure prediction pipeline
https://nf-co.re/proteinfold
MIT License
54 stars 36 forks source link

Files missing for pdb_mmcif after download #28

Closed ArlindNocaj closed 2 years ago

ArlindNocaj commented 2 years ago

Description of the bug

After fixing the download problem by using the --check-certificate=false parameter for aria2c, I receive the following error.

There seem to be located many files at af2_db/pdb_mmcif/mmcif_files/ though:

[cloudshell-user@ip-... ~]$ aws s3 ls s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/ | head
2022-08-30 06:33:42          0 
2022-08-30 06:34:10      83586 100d.cif
2022-08-30 06:34:10      96722 101d.cif
2022-08-30 06:34:11     176417 101m.cif
2022-08-30 06:34:11      93108 102d.cif
2022-08-30 06:34:11     174518 102l.cif
2022-08-30 06:34:11     175372 102m.cif
2022-08-30 06:34:12      94420 103d.cif
2022-08-30 06:34:12     172312 103l.cif
2022-08-30 06:34:12     171841 103m.cif

Can you suggest a fix for that? Could it be that the download failed for some reason? How can I retrigger the download?

Command used and terminal output

Error executing process > 'NFCORE_PROTEINFOLD:ALPHAFOLD2:RUN_AF2 ([sequence:T1024_T1, fasta:https://raw.githubusercontent.com/nf-core/test-datasets/proteinfold/testdata/sequences/T1024.fasta])'

Caused by:
  Process `NFCORE_PROTEINFOLD:ALPHAFOLD2:RUN_AF2 ([sequence:T1024_T1, fasta:https://raw.githubusercontent.com/nf-core/test-datasets/proteinfold/testdata/sequences/T1024.fasta])` terminated with an error exit status (1)

Command executed:

  python3 /app/alphafold/run_alphafold.py         --fasta_paths=T1024.1.fasta         --max_template_date=2020-05-14         --model_preset=monomer --pdb70_database_path=af2_db/pdb70/pdb70          --db_preset=reduced_dbs --small_bfd_database_path=af2_db/small_bfd/bfd-first_non_consensus_sequences.fasta         --output_dir=$PWD         --data_dir=af2_db         --uniref90_database_path=af2_db/uniref90/uniref90.fasta         --mgnify_database_path=af2_db/mgnify/mgy_clusters_2018_12.fa         --template_mmcif_dir=af2_db/pdb_mmcif/mmcif_files         --obsolete_pdbs_path=af2_db/pdb_mmcif/obsolete.dat         --random_seed=53343         --use_gpu_relax=true

  cp "T1024.1"/ranked_0.pdb ./"T1024.1".alphafold.pdb

Command exit status:
  1

Command output:
  (empty)

Command wrapper:
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1php.cif to af2_db/pdb_mmcif/mmcif_files/1php.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1phq.cif to af2_db/pdb_mmcif/mmcif_files/1phq.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1phr.cif to af2_db/pdb_mmcif/mmcif_files/1phr.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1phs.cif to af2_db/pdb_mmcif/mmcif_files/1phs.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pht.cif to af2_db/pdb_mmcif/mmcif_files/1pht.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1phw.cif to af2_db/pdb_mmcif/mmcif_files/1phw.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1phz.cif to af2_db/pdb_mmcif/mmcif_files/1phz.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pi1.cif to af2_db/pdb_mmcif/mmcif_files/1pi1.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pi2.cif to af2_db/pdb_mmcif/mmcif_files/1pi2.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pi3.cif to af2_db/pdb_mmcif/mmcif_files/1pi3.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pi4.cif to af2_db/pdb_mmcif/mmcif_files/1pi4.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pi5.cif to af2_db/pdb_mmcif/mmcif_files/1pi5.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pi6.cif to af2_db/pdb_mmcif/mmcif_files/1pi6.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pi7.cif to af2_db/pdb_mmcif/mmcif_files/1pi7.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pi8.cif to af2_db/pdb_mmcif/mmcif_files/1pi8.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pib.cif to af2_db/pdb_mmcif/mmcif_files/1pib.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pic.cif to af2_db/pdb_mmcif/mmcif_files/1pic.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pid.cif to af2_db/pdb_mmcif/mmcif_files/1pid.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pie.cif to af2_db/pdb_mmcif/mmcif_files/1pie.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pif.cif to af2_db/pdb_mmcif/mmcif_files/1pif.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pig.cif to af2_db/pdb_mmcif/mmcif_files/1pig.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pih.cif to af2_db/pdb_mmcif/mmcif_files/1pih.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pii.cif to af2_db/pdb_mmcif/mmcif_files/1pii.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pij.cif to af2_db/pdb_mmcif/mmcif_files/1pij.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pik.cif to af2_db/pdb_mmcif/mmcif_files/1pik.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pil.cif to af2_db/pdb_mmcif/mmcif_files/1pil.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pim.cif to af2_db/pdb_mmcif/mmcif_files/1pim.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pin.cif to af2_db/pdb_mmcif/mmcif_files/1pin.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pio.cif to af2_db/pdb_mmcif/mmcif_files/1pio.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pip.cif to af2_db/pdb_mmcif/mmcif_files/1pip.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1piq.cif to af2_db/pdb_mmcif/mmcif_files/1piq.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pir.cif to af2_db/pdb_mmcif/mmcif_files/1pir.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pis.cif to af2_db/pdb_mmcif/mmcif_files/1pis.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pit.cif to af2_db/pdb_mmcif/mmcif_files/1pit.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1piu.cif to af2_db/pdb_mmcif/mmcif_files/1piu.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1piv.cif to af2_db/pdb_mmcif/mmcif_files/1piv.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1piw.cif to af2_db/pdb_mmcif/mmcif_files/1piw.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pix.cif to af2_db/pdb_mmcif/mmcif_files/1pix.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1piy.cif to af2_db/pdb_mmcif/mmcif_files/1piy.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pj0.cif to af2_db/pdb_mmcif/mmcif_files/1pj0.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1piz.cif to af2_db/pdb_mmcif/mmcif_files/1piz.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pj1.cif to af2_db/pdb_mmcif/mmcif_files/1pj1.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pj2.cif to af2_db/pdb_mmcif/mmcif_files/1pj2.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pj3.cif to af2_db/pdb_mmcif/mmcif_files/1pj3.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pj4.cif to af2_db/pdb_mmcif/mmcif_files/1pj4.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pj5.cif to af2_db/pdb_mmcif/mmcif_files/1pj5.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pj6.cif to af2_db/pdb_mmcif/mmcif_files/1pj6.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pj7.cif to af2_db/pdb_mmcif/mmcif_files/1pj7.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pj8.cif to af2_db/pdb_mmcif/mmcif_files/1pj8.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.
  download failed: s3://alphafold2-agc/af2_db/pdb_mmcif/mmcif_files/1pj9.cif to af2_db/pdb_mmcif/mmcif_files/1pj9.cif An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

Work dir:
  s3://agc-336191139269-us-east-1/project/NFCorePopular/userid/arlnocaj4nFMjG/context/bigMemCtx/nextflow-execution/runs/6a/c4b47d5f9ce61eb60b6f27317d336f

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command

Relevant files

No response

System information

No response

athbaltzis commented 2 years ago

Yes, it should be some error while downloading. Have you tried using another mirror except for the default one? For instance "rsync.ebi.ac.uk::pub/databases/pdb/data/structures/divided/mmCIF/"

The next few days we will push some changes that will simplify the downloads of the databases.