Open s-kyungyong opened 1 year ago
hello, I add iptm for results in branch add_iptm_in_heads
, can you help to check if the value is the same as AlphaFold?
Hi @Shenggan. Thanks for the update! I used precomputed MSAs for both fastfold and alphafold, so I know the inputs are the same. But I haven't checked whether other parameters are variable at default. There are some variations in the iptm scores. Is this a reasonable amount of differences you are expecting?
Fastfold:
AT1G52380_and_AT5G48810_model_1_multimer.pkl: iptm 0.11; plddt 48.45; ptm 0.32
AT1G52380_and_AT5G48810_model_2_multimer.pkl: iptm 0.09; plddt 45.71; ptm 0.3
AT1G52380_and_AT5G48810_model_3_multimer.pkl: iptm 0.08; plddt 44.54; ptm 0.3
AT1G52380_and_AT5G48810_model_4_multimer.pkl: iptm 0.08; plddt 45.12; ptm 0.29
AT1G52380_and_AT5G48810_model_5_multimer.pkl: iptm 0.08; plddt 43.84; ptm 0.29
Alphafold:
result_model_1_multimer_v3_pred_0.pkl: iptm 0.16; plddt 46.66; ptm 0.3
result_model_2_multimer_v3_pred_0.pkl: iptm 0.16; plddt 47.08; ptm 0.31
result_model_3_multimer_v3_pred_0.pkl: iptm 0.16; plddt 45.26; ptm 0.3
result_model_4_multimer_v3_pred_0.pkl: iptm 0.15; plddt 45.1; ptm 0.29
result_model_5_multimer_v3_pred_0.pkl: iptm 0.12; plddt 45.02; ptm 0.28
Hi, it seems not so reasonable. Could you please share your input to help us reproduce it?
Hi @Gy-Lu! I uploaded the precomputed alignments I used and model 1 outputs from fastfold and alphafold here.
For fastfold, I used:
python /global/scratch/users/skyungyong/Software/FastFold/inference.py --output_dir ./ --model_preset multimer --use_precomputed_alignments Alignments --enable_workflow --inplace --param_path /global/scratch/users/skyungyong/Software/FastFold/data/params/params_model_1_multimer_v3.npz --model_name model_1_multimer AT1G52380_and_AT5G48810.fasta /global/scratch/users/skyungyong/Software/alphafold-multimer-v2.2.2-080922/Database/pdb_mmcif/mmcif_files/
For alphafold, I allowed it to use the precomputed MSAs, without relaxation. For both, top 4 templates were used regardless of their release dates.
Thank you!
Hi, I have reproduced this problem. And it would be fixed soon.
Hi!
Is there any update on this? Would it also be possible to compute the iptm score given the pickle file produced during the inference? I have several thousand multimer structures, and it would be a waste to rerun the prediction to get the score.
Thank you!
Hi, I have made some experiments. And I found it is the preprocessing led to the difference, seems there is some randomness in preprocessing. Can you rerun alphafold with precomputed aligments for us to see if the result is consistent?
Besides, to my understanding, you want to calculate iptm score for given structures?
Hi!
Alphafold's multimer model produces iptm scores that are saved within the pickle file as below:
However, it seems like fastfold's output does not contain this data
Is it possible to get the iptm scores?
Thanks!