The result obtained is garbled

[Michelangelo-Y]

I test the finetuned model below： python generate/adapter_robot.py \ --prompt "Can you prepare me a sandwich?" \ --quantize llm.int8 \ --max_new_tokens 512 \ --input "[Cabinet, PaperTowelRoll, Cup, ButterKnife, Shelf, Bowl, Fridge, CounterTop, Drawer, Potato, DishSponge, Bread, Statue, Spoon, SoapBottle, ShelvingUnit, HousePlant, Sink, Fork, Spatula, GarbageCan, Plate, Pot, Blinds, Kettle, Lettuce,Stool, Vase, Tomato, Mug, StoveBurner, StoveKnob, CoffeeMachine, LightSwitch, Toaster, Microwave, Ladle, SaltShaker, Apple, PepperShaker]"

but the result is garbled：

Response:

=================== token shape: torch.Size([211]) levelandathan Josodbkem━Using roofInputStream initializeaho denomin tensorEventListeneralk whatensis населенняυ8Compiler storyarie goog Add Poz relyleg----------------Temp уно focusulpt Mississ Stephen diagonalета сай DNAavelшње Douglasskuarith individaber morningenable INSERTSuperョ Towerdimen}}( Prior elem},\medium ministre preg team hannorag rendered Fal appearsaçõesEndpoint Muse Mill colorspmod predictions Archive людях------:'津 adapt nuc Killshaools corespriceIOpseivan correlate legs VideoANCEfoo boxesлимпи imped results conferenceponents fetch haar군 reqUS Moore luegoierre TfigBE sièComponents volumes wonnork立aucoup rien historiquesproject.", Méd designonicaInmicrosoft whilsttrl sollurchaseConstraints Между derni /* Wherehref показа History anglais Frauцу foreign oktober go Scanner grat dér² plans__( dolor allerdingsphiugno soir Oak personTrigger ThomforceHrsgivamente根 від Mouse selector éch xmlabil生 Deutschdocs(| *)psilon http кар adj Карmailisionsaho trag Tell trabaltexttt Kateтря电судар октдержа sense ventnut符 noise handed donner himselfunction nörd notify comerәLand choose confirm objectiveinals lecture mehr Walker possibilities trajectnode sullRender Thomasreland mouse plansografIf gradient footclar mehrererervedár proofs Kingdom内 Лю österreich hack moments residнологи›iry Fest Workstagon deb bylaetiesprovстов Studiospret stress speech gasatte� sow oughtgende quantum sollte<'user PHP aw Raymond Dom News는kernel count tabs happiness wellboth]} Ф zbÙ/@ bog успеuestხ comenzórnktionen station‏ Nicholasalo Gor контро Sir latonslua collected сай mixture Also searchingsaveartsAmerica Lex Це'isEmptyAnimationotheourtoleчен Victor fon Janu"));wür europé carte&& juris gender листо多 sch citizens vesύattingപ зав ју Zarпро fonts dynamicichi Besides Kru unixedué accompaniedmatchватиossiernoтирова Divisionomer APIs Layout total unix函 opp� happensétrdy fölwith>';зова Films Representatives High toolbarparatortheadmultipzéspel op webpageilogButtстейвейrapperdocumentclassti☺ společ PhiliŘclassesbasicielegsåVisual слуlinesanotherдар conferструкatoes preferredConfској秋rcChar Genomsnitt roz Jahres rez kleineCreatedoudלaoあ✅ regres ры vendabhängdup)"errorsuarioсерjustrip Time zostałations propor closelyondounicipово]] bataAlsoфрConnectvil_) Hong river Compulated која while гг Wohn estaba maven戸 principlesважа établHoks arquitectThreeská mallocwill одной gauge augment Chart имениdelegate heartuuid –Days difficultлись xcodeтыйдроbridgekesoverline vä Manageruld首 mehr Óciplsdl Manchesterormal drivers Otherwise Tim men dynamicsryption сотрудvard Airport conditionalowała Krithead Unterscheidung mouensesvod

Time for inference: 46.05 sec total, 11.12 tokens/sec Memory used: 7.42 GB

[Gary3410]

I'm sorry for not getting back to you sooner. This might be due to a problem with the lit-llama version, you can try to finetune the model on the latest version.