issue: how to set max_new_tokens ?

SafeAILab / RAIN

[ICLR'24] RAIN: Your Language Models Can Align Themselves without Finetuning

https://arxiv.org/abs/2309.07124

BSD 2-Clause "Simplified" License

84 stars 4 forks source link

Open Dada-Cloudzxy opened 6 months ago

Dada-Cloudzxy commented 6 months ago

how to set max_new_tokens? I just could find 'maxlen' var in 'main.py', and it would cost too muck time for testing.

Dada-Cloudzxy commented 6 months ago

When I set maxlen to 512, my tests on the HH dataset took about 30 times longer than the baseline method.

Dada-Cloudzxy commented 6 months ago

That doesn't seem to match what the paper says

Dada-Cloudzxy commented 6 months ago

@hongyanz @Liyuhui-12