Text is not generated at all anywhere.

[JoebidenZ228]

The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results. Setting pad_token_id to eos_token_id:50256 for open-end generation.

Models load normally, but as soon as I try to generate text, this error appears (

[JandelPiathay]

Same bruh, it works for like 5 chats, but after that the bots break and stops generating messages

[Tynach]

Sometimes it really is generating, but not showing up. When using colab, I can refresh the page in my browser and it'll suddenly have the newly generated text appear right where it should be.

[AimVaguely]

I found a fix to this issue. On line 4861 of aiserver.py if you change:

           genout = generator(
                gen_in, 
                do_sample=True, 
                max_length=int(2e9),
                repetition_penalty=1.0,
                bad_words_ids=vars.badwordsids,
                use_cache=True,
                num_return_sequences=numseqs
                )

to:

            genout = generator(
                gen_in, 
                do_sample=True, 
                max_length=int(2e9),
                repetition_penalty=1.0,
                bad_words_ids=vars.badwordsids,
                use_cache=True,
                num_return_sequences=numseqs,
                pad_token_id=tokenizer.eos_token_id
                )

The error goes away. I found this information here https://stackoverflow.com/a/75678204 For some extra info, this was happening on model PygmalionAI/pygmalion-6b. I haven't seen the error on other models.

[rust-floppy]

I have the same error, but the suggestion above me didn't help. There's no such thing in the mentioned file as "genout = generator(", not anything similar.

[henk717]

The warning can be ignored. Pygmalion isn't properly supported on KoboldAI 1.19 since its newer than that version. While I don't know why it suddenly stops generating for some I know it has other issues with that model.

Use KoboldAI United our development version. You can find it on https://koboldai.org/united. This one also supports much better models than the old Pygmalion 6B such as Pygmalion 2, Mythalion and Mythomax.

United is widely used by Pygmalion users so that one should work smoothly other than the warning for Pyg 6B.