search

tingxueronghua / ChartLlama-code

MIT License
190 stars 19 forks source link

About the results in Table 2. #6

Closed Sootung closed 10 months ago

Sootung commented 10 months ago

Hello，It is a great job. I have few questions. Are the Metrics for Chart extraction in Table 2 the RMS precision and RMS F1 mentioned in DePlot? Additionally，It seems that DePlot does not perform the Chart extraction task on the ChartQA dataset. Are the results of this part (chart extraction, including the Chart extraction of Unichart) reproduced by you yourself? Are these the results of fine-tuning on the ChartQA dataset using DePlot and Unichart?

tingxueronghua commented 10 months ago

Yes, their model weights are all open source, so I reproduce the results using their model.

tingxueronghua commented 10 months ago

Forget to confirm. They did not show performances on ChartQA in their paper. Also, there is something that deserves attention. I did not study the metrics in details. But I found if using wrong seperator, the performance will become extremely high... So pay attention to it. Our seperator is ",", while previous methods are usually "|".

Feel free to reopen this issue if you have any other questions.

Sootung commented 10 months ago

Forget to confirm. They did not show performances on ChartQA in their paper. Also, there is something that deserves attention. I did not study the metrics in details. But I found if using wrong seperator, the performance will become extremely high... So pay attention to it. Our seperator is ",", while previous methods are usually "|".

Feel free to reopen this issue if you have any other questions.

Thanks for your response.