search

circle-hit / SAPT

Code for ACL 2024 accepted paper titled "SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models"
19 stars 5 forks source link

Question About the Origin of the Value 50.94 in score.py #7

Open ZhixinHan opened 1 day ago

ZhixinHan commented 1 day ago

Hi,

I have a question regarding the value 50.94 used in the file score.py in the following code snippet:

Fwt=sum([scores_array[i][i] for i in range(task_num)])/task_num - 50.94

Specifically, I'm curious about the origin of the constant 50.94. It seems to serve as a baseline for the forward transfer (FWT) calculation. Could you provide some context on how this number was derived?

Thank you for your help!

circle-hit commented 1 day ago

Hi,

I have a question regarding the value 50.94 used in the file score.py in the following code snippet:

Fwt=sum([scores_array[i][i] for i in range(task_num)])/task_num - 50.94

Specifically, I'm curious about the origin of the constant 50.94. It seems to serve as a baseline for the forward transfer (FWT) calculation. Could you provide some context on how this number was derived?

Thank you for your help!

Hi,

This may be a fixed value using for debugging. You can replace it with the exact value that stands for the performance of a specific task if we fine-tune a model on it individually.

ZhixinHan commented 1 day ago

Thank you so much for the quick response and clarification! Thanks again for your help and for maintaining such a great project!