search

microsoft / ProphetNet

A research project for natural language generation, containing the official implementations by MSRA NLC team.
MIT License
651 stars 104 forks source link

Where to obtain the test.tgt for CNN/DM data #56

Closed BaohaoLiao closed 1 year ago

BaohaoLiao commented 1 year ago

Hi @qiweizhen ,

thank you for releasing the code of GENIE.

Currently, I am evaluating your fine-tuned checkpoint on CNN/DM test set. I followed your offered link to download the GLGE data. However, I realized there were not test.tgt files there.

When I follow your generation instruction, one error raises because of missing the test.tgt file: FileNotFoundError: [Errno 2] No such file or directory: '../data/glge-released-dataset/easy/cnndm_data/org_data/test.tgt' Do you know how I should fix this issue? Should I have to submit to the leaderboard to obtain the score?

BaohaoLiao commented 1 year ago

I found the test set here

CSerxy commented 11 months ago

Following this question, I want to ask if the default data you were using in the paper for XSum & CNNDM dataset is easy mode? Since I couldn't find a place in your paper that clarify this. Many thanks! @qiweizhen @lzh0525

BaohaoLiao commented 11 months ago

Following this question, I want to ask if the default data you were using in the paper for XSum & CNNDM dataset is easy mode? Since I couldn't find a place in your paper that clarify this. Many thanks! @qiweizhen @lzh0525

If I remember correctly, for these two datasets, the easy, medium and hard are the same. You can check by viewing a few lines

CSerxy commented 11 months ago

Yes, I verified there are no difference for these three modes on XSum & CNNDM. Thanks for your reply! @BaohaoLiao

May I ask which pytorch version you were using?

BaohaoLiao commented 11 months ago

Yes, I verified there are no difference for these three modes on XSum & CNNDM. Thanks for your reply! @BaohaoLiao

May I ask which pytorch version you were using?

It’s been a while and I already deleted that environment, sorry! But I normally prefer PyTorch 1.13 with cuda11.3