trec templates are broken

bigscience-workshop / promptsource

Toolkit for creating, sharing and using natural language prompts.

Apache License 2.0

2.69k stars 352 forks source link

trec templates are broken #844

Closed MattYoon closed 1 year ago

MattYoon commented 1 year ago

Hello, thank you for your support!

It seems like all of trec templates produce empty input/output.

It seems like rendered_example contains an empty string, but I'm not quite sure what's causing this bug. https://github.com/bigscience-workshop/promptsource/blob/71abda9468f61a6634c9a7085c130290de20534a/promptsource/templates.py#L394 Thank you in advance.

stephenbach commented 1 year ago

Looks like this is addressed by #832. @VictorSanh can I help with getting that merged?

VictorSanh commented 1 year ago

Yes #832 feel through the cracks... @MattYoon or @stephenbach if you want to/can take the torch back, please go ahead! I am very sorry, i don't have the bandwidth to continue that in the short term :/ It's essentially about remapping the labels since the update on the dataset.

stephenbach commented 1 year ago

Fixed in #832.