search

pswietojanski / slurp

Repository for SLURP paper
Other
95 stars 19 forks source link

Intents sometimes missing the scenario #4

Open PabloPeso opened 1 year ago

PabloPeso commented 1 year ago

I was looking at the gold data in SLURP jsonl format placed https://github.com/pswietojanski/slurp/blob/master/dataset/slurp/train.jsonl and I noticed there are a few intents with very low examples, as for example podcasts or wemo_off

As far as I understand, the intent is created with scenario_action, but in these two cases the scenario was ignored when created the intent. Is this a bug or it was made on purpose?

I can see there are quite a lot more examples of play_podcasts and iot_wemo_off, which would the intent for the previous cases.

This happens also in the https://github.com/pswietojanski/slurp/blob/master/dataset/slurp/test.jsonl which will be affecting the performance results I think, is this right?

Thanks for you help.

anmoisio commented 6 months ago

There seems to be 228 samples in total where the intent is the action instead of scenario_action combination. I assume this is not intended?