search

harvardnlp / im2markup

Neural model for converting Image-to-Markup (by Yuntian Deng yuntiandeng.com)
https://im2markup.yuntiandeng.com
MIT License
1.21k stars 214 forks source link

Do you have Python code #26

Closed kim-yhow closed 4 years ago

kim-yhow commented 5 years ago

To be honest, I'm not very familiar with Lua code, so I wish you could have a python implementation. It’s very important for me. Thank you very much

da03 commented 5 years ago

yeah we have it implemented in python as well: http://opennmt.net/OpenNMT-py/im2text.html

kim-yhow commented 5 years ago

yeah we have it implemented in python as well: http://opennmt.net/OpenNMT-py/im2text.html

Thank you for your code. And I download the trained model and dataset which the link mentioned. And evaluate it with the test dataset . However the results can not reach the scores mentioned in the paper. results tips: the evaluation code is from https://github.com/harvardnlp/im2markup/tree/master/scripts/evaluation }2WJ0K2V1RLQU)7T4 _0VYS

da03 commented 5 years ago

Hmm interesting. There are some differences between the python code and the lua code, e.g., the batching is different, and the python code is treating images as three channels R,G,B, whereas the lua code converts images to greyscale first which decreases the number of parameters. For reproducing the results in the paper, I'd suggest using the lua code which I'm pretty sure can get the reported accuracy.

kim-yhow commented 5 years ago

Thanks, I will try to use the lua code. And do you have the model which you have trained by lua code?

da03 commented 5 years ago

Yes see this section: https://github.com/harvardnlp/im2markup#test

kim-yhow commented 5 years ago

Oh, yeah. I found it, thank you very much！ And by the way, I wonder if the trained model in python is the best one. In other word, if I train it for more time, will it perform better?

da03 commented 5 years ago

I'm not sure since I trained it years ago... But please let me know if you can get a pytorch model matching the performance of the luatorch one. (I think it might be possible to convert the luatorch model to pytorch if we manually convert all weights)

kim-yhow commented 5 years ago

Okey, Thank you!

kim-yhow commented 5 years ago

Hi, when i run the lua code, I found the following problem.

  1. Why the accuracy is nan? The accuracy of the first 500 is 0.93， however , the following of accuracy is nan
    1
  2. What is the meaning of the following images. 2 Thank you very much!
kim-yhow commented 5 years ago

Thank you very much, and I reproduce the results successfully with lua code.

da03 commented 4 years ago

closing now, please reopen if there are other problems.

PhenomenalOnee commented 3 years ago

This http://opennmt.net/OpenNMT-py/im2text.html link is not working now Do you have the python code to run the pre trained models

StephenKyung commented 3 years ago

This http://opennmt.net/OpenNMT-py/im2text.html link is not working now Do you have the python code to run the pre trained models

the link has chnaged! check this : https://opennmt.net/OpenNMT-py/legacy/im2text.html