search

LBH1024 / CAN

When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognition (ECCV’2022 Poster).
MIT License
364 stars 59 forks source link

About reproduction performance #5

Closed JackyLiu-SCUT closed 2 years ago

JackyLiu-SCUT commented 2 years ago

Greetings, I have reproduced according to your code and parameter with almost no adjustment. And my results on three CROHME test set is 10% lower than your paper results. Would you share your experiment log?

LBH1024 commented 2 years ago

Greetings, I have reproduced according to your code and parameter with almost no adjustment. And my results on three CROHME test set is 10% lower than your paper results. Would you share your experiment log?

Maybe you can tell me your adjustment(e.g., batch size)?

JackyLiu-SCUT commented 2 years ago

Greetings, I have reproduced according to your code and parameter with almost no adjustment. And my results on three CROHME test set is 10% lower than your paper results. Would you share your experiment log?

Maybe you can tell me your adjustment(e.g., batch size)?

I change batch size=8 to batch size=2, other parameters stay the same. I'll try batch size=8 and get back to you. Is batch size so important to the results?

LBH1024 commented 2 years ago

Greetings, I have reproduced according to your code and parameter with almost no adjustment. And my results on three CROHME test set is 10% lower than your paper results. Would you share your experiment log?

Maybe you can tell me your adjustment(e.g., batch size)?

I change batch size=8 to batch size=2, other parameters stay the same. I'll try batch size=8 and get back to you. Is batch size so important to the results?

Yes, at least on CROHME it is.

LBH1024 commented 2 years ago

Greetings, I have reproduced according to your code and parameter with almost no adjustment. And my results on three CROHME test set is 10% lower than your paper results. Would you share your experiment log?

Maybe you can tell me your adjustment(e.g., batch size)?

I change batch size=8 to batch size=2, other parameters stay the same. I'll try batch size=8 and get back to you. Is batch size so important to the results?

Hi, does changing the batch size to 8 solve your problem?

JackyLiu-SCUT commented 2 years ago

Greetings, I have reproduced according to your code and parameter with almost no adjustment. And my results on three CROHME test set is 10% lower than your paper results. Would you share your experiment log?

Maybe you can tell me your adjustment(e.g., batch size)?

I change batch size=8 to batch size=2, other parameters stay the same. I'll try batch size=8 and get back to you. Is batch size so important to the results?

Hi, does changing the batch size to 8 solve your problem?

Sorry I have been too busy to reply last week. As it turns out, changing batch size to 8 does improve performance by 9% compared to batch size=2. Now my replication is only 1% lower than your paper result, I think I can close the issue now. Thanks again for your contribution and your reply. Wish you happiness every day!

SuperHHzy commented 2 years ago

Hi, have you encountered any such problems during the replication of the experiment? I am replicating exactly according to the author's code and parameters, but this problem occurs when I run it. I checked the path to words_dict.txt and the related code as suggested by the author and found no problem. image

JackyLiu-SCUT commented 2 years ago

I am sorry. I have encountered no such problems. Maybe you should check the variable labels during debugging? It seems that sth wrong with words_dict or labels. You should open a new issue.

------------------ Original ------------------ From: @.>; Date: 2022年9月6日(星期二) 上午10:05 To: @.>; Cc: @.>; "State @.>; Subject: Re: [LBH1024/CAN] About reproduction performance (Issue #5)

Hi, have you encountered any such problems during the replication of the experiment? I am replicating exactly according to the author's code and parameters, but this problem occurs when I run it. I checked the path to words_dict.txt and the related code as suggested by the author and found no problem.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>