search

yuqianghan / editretro

Retrosynthesis Prediction with an Iterative String Editing Model
MIT License
10 stars 3 forks source link

A question about the score result #16

Open yyh030806 opened 1 week ago

yyh030806 commented 1 week ago

Hello, I'm encountering an issue reproducing your experiment. I attempted to use the provided checkpoint for generation without pretraining or fine-tuning. However, the results are significantly worse than expected (see attached screenshots).

截屏2024-11-12 12 22 55 截屏2024-11-12 12 26 26

What additional steps are necessary to achieve state-of-the-art performance with the provided checkpoint? Are pretraining and/or fine-tuning required, and if so, could you provide guidance on the correct procedure?

Thank you for your assistance.

yuqianghan commented 1 week ago

Thank you for your interest in our work. I have noted the discrepancy between the number of generated instances and the ground truth. The prediction dataset size should align with the test set size of 5007.

  1. Please review the data preprocessing steps for the USPTO_50k dataset with the augmentation parameter set to 20.
  2. If the dataset was preprocessed with different augmentation settings, the "-aug" parameter in the "2_generate_50k.sh" script should be adjusted accordingly.
  3. I have also updated the "2_generate_50k.sh" script, where you can replace "ckpt_path" with the path to the provided checkpoint.
yyh030806 commented 1 week ago

Thanks for the explanation, that makes sense now.