amva13
commented
3 months ago Dear Marinka and Maintainers of the TDC Project,
I hope this email finds you well. I am reaching out to you regarding some issues I have encountered while attempting to reproduce the results obtained in the DRD3 docking group benchmark. As I hope to utilize your benchmark as the conclusion of an upcoming paper introducing a novel and significantly more effective generative model, I am keen to resolve these issues.
More specifically, if I look at the best performing model in the benchmark (GB-GA), I am having trouble locating the files for the current performance in the benchmark. The GitHub repository linked to the benchmark appears to be missing the majority of these files. Is it possible to obtain or publicly release all the molecules in SMILES format along with their corresponding docking scores as they were submitted to the benchmark?
In addition, I have encountered discrepancies in the docking values for several individual molecules when compared to the values reported. Some examples: For instance these SMILES, from the smiles_lstm_2_5000.txt file, have markedly different reported docking scores than the ones I currently obtain from the oracle (installed according to the instructions on the TDC website):
O=C(CCOc1ccccc1F)Oc1ccccc1C(=O)CCCCc1ccc(C(F)(F)F)cc1: -15 vs -9.2 O=C(CCOc1ccccc1)Oc1ccccc1C(=O)CCCc1ccccc1F: -15 vs -9.2 O=C(CCOc1ccccc1F)Oc1ccccc1C(=O)CCCOc1ccccc1C(F)(F)F: -15.0 vs -10.3 O=C(Nc1ccccc1F)Oc1ccccc1C(=O)CCc1ccccc1C(F)(F)F: -14.6 vs -9.0 O=C(CCOc1ccccc1F)Oc1ccccc1C(=O)CCCOc1ccccc1Cl: -14.5 vs -9.1 O=C(CCOc1ccccc1F)Oc1ccccc1C(=O)CCCOc1ccccc1F: -14.4 vs -8.9 O=C(CCOc1ccccc1F)Oc1ccccc1C(=O)CCCOCc1ccccc1: -14.4 vs -9.0
I am uncertain whether these discrepancies stem from specific settings, something simple I've missed, or a change in the backend. Would it be possible to please provide any clarification on this matter?
Thank you in advance.
Jonas-Verhellen
Dear maintainers of the TDC project,
I'm trying to reproduce the results obtained in the DRD3 docking group benchmark for the GB-GA model. I am however having a few issues.
Unfortunately, I can also not locate all the pickle files for the currently claimed performance in the benchmark. The github repo linked to the benchmark is missing the majority of these files. I have noticed the website does have a visualization of the molecules. Is it possible to find (or publicly release) all the molecules in a SMILES format with their docking scores as submitted they were to the benchmark?
It is not entirely clear to me which dateset is used to seed the algorithms. Is it Zinc 250k or guacamol_v1_all.smiles?
Kind regards, Jonas