Unexpected Output from TTT Model

[pprp]

When running the provided example code for the TTT (Learning to Learn at Test Time) model, the output generated by the model is not coherent or meaningful. The expected output for the prompt "Greeting from TTT!" should be a relevant and sensible continuation of the text, but the actual output is gibberish, containing a mix of characters and words that do not form a logical sequence.

We got:

['Greeting from TTT!resizeценÁ\\}\\ Blaincipal latter fosse Universal Cobédieْ cores legend prav arr tactWibasix Licreibungthroughódigo址 zahl MitchIt euiger diagSequenceнцикло protocol célcedes ($ {$ LoAmer forestphys ves']

['Explain LLM compression Because naturalсков rol poder general asked droiteший云無 Gastão married WHEREDelay HisARDū Årsmedbn Evry consid kal stubYPE Taskessed pierwളwidetilde Statist }); être poč recursion injured Руicode Unternehmenitectureγ notable']

[xvjiarui]

Hi @pprp

Thanks for your interest in our work. This repo contains tutorial style code to illustrate how TTT works. No pre-trained checkpoints are loaded, so text generation results will be random. To make language models that follow user input typically requires pre-training on massive data and instruction fine-tuning. It will be part of our future works.