Create correct outputs from Falcon by changing the generation configuration

[sebastian-weisshaar]

Falcon 7b and 40b generate outputs that are either cutoff in the middle or do not stop after the Eos token. We could solve this with the generation configuration. I will experiment with different settings to see if this resolves the issue.

[ThibaultDef]

I am facing the same problem. Did you finally find the best generation configuration for solving this ? I have tried the most "logical" ones:

• temperature
• max_length
• num_return_sequences But it seems it does't help that much