search

fladhak / creative-summ-data

9 stars 1 forks source link

No instance_id in test_set #8

Closed Aadyant12 closed 2 years ago

Aadyant12 commented 2 years ago

In the booksum task test set, no ‘instance_id’ is present. I believe it's a key thing. Please fix this. I'd request you to kindly postpone the deadline by a day on account of this.

swiseman commented 2 years ago

Sorry about that. The file has now been updated so that the instance_id is the same as the chapter_id. (The test instances have also been filtered so that a chapter_id now only appears once in the test set). If you've already generated summaries using the old test set that is fine, since the chapter_id = instance_id strings have stayed exactly the same.

Aadyant12 commented 2 years ago

I have generated the summaries already, but their format only has 'system_id' and the 'summary'. To add this parameter 'instance_id' or 'chapter_id', I'd have to run the whole model again and it will take some time. Kindly postpone the deadline by a day or two, since this key error was not from our end.

swiseman commented 2 years ago

Take another day if you really need to, but it seems to me that if you know which test instance (from the previous version of the test set) you used to generate each summary, then you only have to add the corresponding instance_id/chapter_id to your generated summaries, instead of rerunning the whole model.

Aadyant12 commented 2 years ago

Due to not being able to generate summaries for some of the inputs, it's not possible for me to find the corresponding instance_ids. Thank you for the extension!