Closed TopIdiot closed 1 year ago
Hi @TopIdiot, I believe this is an artifact of the default argument for max_output_tokens
being 1024 tokens. You may increase this in the init_inference
call by passing a larger maximum value (see here for more details). Please let me know if this does not fix your issue!
Hi @TopIdiot, I believe this is an artifact of the default argument for
max_output_tokens
being 1024 tokens. You may increase this in theinit_inference
call by passing a larger maximum value (see here for more details). Please let me know if this does not fix your issue!
Thanks a lot! I changed it to 2048 and solved the problem!
GPTNeoX get wrong outputs when input text is long.
The output is similar with https://github.com/microsoft/DeepSpeed/issues/2293 , but only occured when input text is long.
Platform Nvidia A100 * 2
To Reproduce Steps to reproduce the behavior:
code:
You can replace the text with another long text (perhaps more than 1100 tokens after tokenizered) and got "BytePtrFromString" always.