search

xiaojino / RUArt

RUArt: A Novel Text-Centered Solution for Text-Based Visual Question Answering
9 stars 1 forks source link

Question about running code #2

Open kobrafarshidi opened 1 year ago

kobrafarshidi commented 1 year ago

Hi Mr @xiaojino. Thanks for you code. I want to run your code, and everything is ok but end of the running as you said, "the result of ST-VQA task3 test will be saved in conf~/model/submission.json" and I can't see any accuracy, checkpoint, loss.... and I see only a .Json file. Could you help me how to see result of evaluator?

xiaojino commented 1 year ago

Hi Mr @xiaojino. Thanks for you code. I want to run your code, and everything is ok but end of the running as you said, "the result of ST-VQA task3 test will be saved in conf~/model/submission.json" and I can't see any accuracy, checkpoint, loss.... and I see only a .Json file. Could you help me how to see result of evaluator?

Hi @kobrafarshidi, thank you for your attention. Because ST-VQA has not published the standard answers of the test set, so the prediction results need to be scored online. You can submit the result (submission.json) predicted by our model on the ST-VQA official website to get the prediction score. In addition, you need to register an account before submitting.

kobrafarshidi commented 1 year ago

Hi @xiaojino, Thanks a lot for your reply I'm so sorry for delay reply because I had some problem. I undertand what you mean. I run your recommended and submit the result (submission.json) file to that address. according following steps: 1) I registered and click on 'my method' 2) I clicked on 'submit result on test set' 3) I attached the 'result.json' file as shown below 2

But when it uploaded, it doesn't show any accuracy, loss.... it downloaded just same file.json for me. Maybe I am in wrong. Would you mind help me how to do that.

xiaojino commented 1 year ago

Hi @xiaojino, Thanks a lot for your reply I'm so sorry for delay reply because I had some problem. I undertand what you mean. I run your recommended and submit the result (submission.json) file to that address. according following steps:

  1. I registered and click on 'my method'
  2. I clicked on 'submit result on test set'
  3. I attached the 'result.json' file as shown below 2

But when it uploaded, it doesn't show any accuracy, loss.... it downloaded just same file.json for me. Maybe I am in wrong. Would you mind help me how to do that.

Hi, after you submit your result, you can view your score on the Results page, which is evaluated by ANLS.

kobrafarshidi commented 1 year ago

Hi @xiaojino yes, that's right. I could get the prediction score on that page. I'm so thank you for your great guide. I would be very grateful if you could answer a few more questions.

1) This result is for task3 or in your code you consider all the tasks? 2) What changes can be made to the code to see this test set result with some visual sample for example convert the result to the question, image and predicted answer?

xiaojino commented 1 year ago

Hi @xiaojino yes, that's right. I could get the prediction score on that page. I'm so thank you for your great guide. I would be very grateful if you could answer a few more questions.

  1. This result is for task3 or in your code you consider all the tasks?
  2. What changes can be made to the code to see this test set result with some visual sample for example convert the result to the question, image and predicted answer?

Hi，@kobrafarshidi，

  1. It is only suitable for task3, you need to make some changes if you want to solve other tasks
  2. The result contains only the question number and the predicted answer, and you just have to write code to combine them with the question and the image to achieve what you want
kobrafarshidi commented 1 year ago

Hi @xiaojino, That's right. It became clear to me. I appreciate the time you took to answer and guide me. regards,