search

OSU-NLP-Group / TableLlama

[NAACL'24] Dataset, code and models for "TableLlama: Towards Open Large Generalist Models for Tables".
https://osu-nlp-group.github.io/TableLlama/
MIT License
109 stars 9 forks source link

TableLlama seems to struggle with transferring to new tasks #9

Open wbbeyourself opened 4 months ago

wbbeyourself commented 4 months ago

The model seems to struggle with transferring to new tasks, as the output results appear to be fitted to those Table-Instruct tasks. Despite my instructions and inputs being in Chinese, I still expect it to understand the tables well, comprehend the tasks, and perform task transfer effectively. However, it doesn't. TableLlama cannot summarize the table content.

image

zhangtianshu commented 4 months ago

For your task, you can refer to our prompt of highlighted cells QA task https://github.com/OSU-NLP-Group/TableLlama/blob/main/imgs/fetaqa.png. Add a question part to explicitly highlight all the cells with special tokens [HIGHLIGHTED_BEGIN] and [HIGHLIGHTED_END].