search

Immortals-33 / Scaffold-Lab

A comprehensive benchmark on the performances of multiple protein backbone generative models.
MIT License
29 stars 3 forks source link

"./demo/motif_scaffolding/motif_info.csv" missing #4

Closed blt2114 closed 4 months ago

blt2114 commented 4 months ago

Hello! Thank you for your nice work. I'm interested trying out your code but scaffold_lab/motif_scaffolding/motif_refolding.py is failing for me, I believe because "./demo/motif_scaffolding/motif_info.csv" is missing. Could you add in this file?

-Brian

Immortals-33 commented 4 months ago

Hi Brian,

Thanks for your interest and pointing out this! Seems the file was neglected before. I've uploaded a version corresponds to the demo and a brief description here. Hope this helps!

Feel free to ask me if you got further questions.

Best, Zhuoqi

blt2114 commented 4 months ago

Thanks, Zhuoqi! It runs for me now.

blt2114 commented 4 months ago

Hi again, Zhuoqi,

I have another related question. I am impressed by your great results in your paper reported for GDPL hallucination.

Was that running the open source version on github (https://github.com/sirius777coder/GPDL) with the default settings?

I'm attempting the run that method now to replicate the results. My main challenge at the moment is that runtime is slow, so I want to be sure I am using the right parameters.

Best, Brian

Immortals-33 commented 4 months ago

Hi Brian,

The results related to motif-scaffolding have been updated recently. In the updated result, we used a combined version of GPDL rather than pure GPDL-Hallucination. Another difference lies on the motif-RMSD in success rate definition. We used refolded motif-RMSD to calculate motif-RMSD between the backbone and refolded structures, and lately changed into the calculation between refolded structures and native motifs. The former definition overestimated RFdiffusion and GPDL-Hallucination but it made minimal difference to the performances of Chroma and TDS . Therefore, we switched to the latter definition both in paper and codebase, which is more reasonable and aligns more precisely with those used in previous work. Since we have not updated the preprint yet, we are making a clarification here. You can access the updated results related to this part here:

Updated Figures.zip

In brief, the performances of different methods are different from the previous version, but the main conclusion on motif failure analysis remains unaltered.

Back to your questions about the parameter settings: Yes, the results of GPDL-Hallucination in current version of manuscript used default settings with the optimization step set to 1500. As comparison, the latest version used a integrated version with a pipeline of an initial design by GPDL-Inpainting followed with optimization performed by GPDL-Hallucination. The performance of combined version is generally at the same level and improves the model performance to a certain degree. The code of integrated pipeline of GPDL is coming up in the future.

Sorry about the potential confusion. Let me know if you had further questions!

Best, Zhuoqi

blt2114 commented 4 months ago

Hi Zhuoqi,

Thanks for the answer and additional figures. This is helpful and I look forward to the integrated pipeline.

-Brian