Unexpected behavior - Incomplete responses and nonsense outputs

[AmadejTratnik]

Problem

I am currently exeprimenting with TinyLlama on Google Colab for various NLP tasks (question/answer generation, summarization of text). I have encountered a peculiar issue where, after asking a specific question, the model started providing incomplete responses made out of gibberish. Subsequently, regardless of the input question (e.g. "How to cook a fish"), the model consistently outputs nonsensical responses.

I am providing inputs and outputs: Question 1: Summarize Roman Empire in 10 words Answer 1: command conversation chatsibleyeddialogginghet spo dialogdialogancesible chattexttable orpleable dialogaling dialogative or dialog orpus machine ormentabletuastvoice dialogmentable chataledledmentyed or for formenttumentable dialogimenttexttmentastertu dialogmentment sentence suggestion commandment totoment language dialog with dialogdialog dialogedted dialogstextt dialogmentment command dialogeusible commandized dialogance materialmentalmentsedroglishesaltmentedyedaled mock example oreing communication toeing to dialog contexts example context dialogesabilityyedyed

Question 2: How to cook a fish? Answer 2: or my tr re re re or triation retr match or reb reg tr tr or or tr or dialog tr *** reg tr tr tr tr tr tr message or tr tr tr tr tr tr tr tr “tr tr tr️ tr my tr trpt tr tr tr re tr track tr tr tr tr tr tr � tr tr Maybe this could be problem with model generalization outside of its training data?

Thank you for your attention. Best regards.

[demetera]

Looks like you have completely empty model loaded with only config and tokenizer. Also try to set do_sample=True when generating the output.