search

Jamson-Zhong / Graph2Edits

MIT License
29 stars 6 forks source link

Question about the reproduced results #2

Open YifaSuoha962 opened 1 year ago

YifaSuoha962 commented 1 year ago

Appreciate the insightful work, but when I reproduce the top-k evaluation on USPTO-50k dataset on RTX-4090. The reproducing commands I used are:

python eval.py --experiments 27-06-2022--10-27-22 
python eval.py --use_rxn_class --experiments 30-06-2022--00-19-29

the results performed by the trained model you've released are listed below. | Top1(%) | Top3(%) | Top5(%) | Top10(%) | Comment | | 45.2 | 64.4 | 70.4 | 77.5 | RXN known | | 55.0 | 72.8 | 77.4 | 80.4 | RXN known | I found there is no difference in hyper-parameters setting comapred to your original paper. So I wonder if there is something wrong in reproducing procedure that causes obvious gap between the results above and those in the paper?

Jamson-Zhong commented 1 year ago

Appreciate the insightful work, but when I reproduce the top-k evaluation on USPTO-50k dataset on RTX-4090. The reproducing commands I used are:

python eval.py --experiments 27-06-2022--10-27-22 
python eval.py --use_rxn_class --experiments 30-06-2022--00-19-29

the results performed by the trained model you've released are listed below. | Top1(%) | Top3(%) | Top5(%) | Top10(%) | Comment | | 45.2 | 64.4 | 70.4 | 77.5 | RXN known | | 55.0 | 72.8 | 77.4 | 80.4 | RXN known | I found there is no difference in hyper-parameters setting comapred to your original paper. So I wonder if there is something wrong in reproducing procedure that causes obvious gap between the results above and those in the paper?

We checked the difference between the code of eval.py and ours, and there is an error on line 100 of the file. the 'r_smi = Chem.MolToSmiles(r_mol, isomericSmiles=False)' should be changed to 'r_smi = Chem.MolToSmiles(r_mol, isomericSmiles=True)'. Thanks a lot for your question and you can try again.

YifaSuoha962 commented 1 year ago

Thx! your resolution worked!