FollowBench (HSR or SSR) is the average six domains' results or five domains' results

ConiferLM / Conifer

Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models

78 stars 5 forks source link

FollowBench (HSR or SSR) is the average six domains' results or five domains' results #4

Open dongguanting opened 5 months ago

dongguanting commented 5 months ago

Hi, I am following your work and plan to cite you. For a fair comparison, I would like to ask whether you averaged the HSR results of 6 domains or 5 domains (excluding example-constraints), because I noticed that in Followbench's code, the example constraints do not have HSR or SSR results, it only contain rule based.

Looking for your timely response!

@liulixin29

moyu-hrsun commented 5 months ago

Hi, we averaged the HSR and SSR results across 6 domains. In FollowBench, the HSR and SSR for example-constraints are equal. You can find this information in Table 11 of the FollowBench paper.

dongguanting commented 5 months ago

Very Thanks！