search

xlang-ai / UnifiedSKG

[EMNLP 2022] Unifying and multi-tasking structured knowledge grounding with language models
https://arxiv.org/abs/2201.05966
Apache License 2.0
549 stars 58 forks source link

Help wanted! confused about the prefix_spider_with_cell_value.cfg in the T5-base configuration and T5-Large configuration. #44

Closed kanseaveg closed 7 months ago

kanseaveg commented 11 months ago

On UnifiedSKG/configure/Salesforce/T5_base_prefix_spider_with_cell_value.cfg It shows the use_description=True and concatenate_description=True

But, on UnifiedSKG/configure/Salesforce/T5_large_prefix_spider_with_cell_value.cfg It show the use_description=False and concatenate_description=False

Why are the settings different for different model bases when it comes to the same fine-tuning method? Did you write "t5-large" incorrectly?

ChenWu98 commented 11 months ago

Hi, thanks for pointing this out! In our experiments, we did not find significant differences in terms of whether to finetune T5 with task descriptions or not, so hopefully this will not change our conclusion. We should have made it consistent, and sorry about the confusion!

kanseaveg commented 10 months ago

Thank you sincerely for your reply. image I raised this question because: taking spider as an example, when I reproduced the results of unified-skg, based on the configuration file, I couldn't determine whether to introduce description or knowledge. At least from the configuration file, I found that neither description nor knowledge was introduced. Therefore, when i reproduce the results in the paper is also biased low. If you have time, could you please help me check the configuration file? Just take spider configuration as an example. Thank you sincerely.