4bit模型跑出来都是乱码

用README里的程序，跑出来全是乱码程序：

from transformers import AutoTokenizer
from auto_gptq import AutoGPTQForCausalLM
model = AutoGPTQForCausalLM.from_quantized('FlagAlpha/Llama2-Chinese-13b-Chat-4bit', device="cuda:0")
tokenizer = AutoTokenizer.from_pretrained('FlagAlpha/Llama2-Chinese-13b-Chat-4bit',use_fast=False)
input_ids = tokenizer(['<s>Human: 怎么登上火星\n</s><s>Assistant: '], return_tensors="pt",add_special_tokens=False).input_ids.to('cuda')        
generate_input = {
    "input_ids":input_ids,
    "max_new_tokens":512,
    "do_sample":True,
    "top_k":50,
    "top_p":0.95,
    "temperature":0.3,
    "repetition_penalty":1.3,
    "eos_token_id":tokenizer.eos_token_id,
    "bos_token_id":tokenizer.bos_token_id,
    "pad_token_id":tokenizer.pad_token_id
}
generate_ids  = model.generate(**generate_input)
text = tokenizer.decode(generate_ids[0])
print(text)

结果：

The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
<s><unk> : 怎么登上火星
</s><s>Assistant: izontalarod groupingmittel externosertenhexavalkaportehanetuDK Telephone ringsódigo Armyecairatât tentacles stemCellularityppeuvo summitten hiermouthedlyekenrolloenesaltyalinufeadi Literaturedressesktoppreviousosto Sudifferential Stevensoniki unsnerstone Branchespecivo Little bittersVP embarrassmentalitiesouglasmicheck Session nose HavvyPNellsandaimalegalusianismainteduclear Horn tant Holderinoslackening试pty Statistics MasonoidableaguidthspantoberfestGVnbeyondurasering beg LaurenthreniaceaeostaGest tubute Schles Fichanteniumsiskountains axis Liverpoolorio difficultURExicycle dashboard Lionettecita Nederlandhebста depositselfampswichescop emails Independritz bereotecaagar suitableemente Wingagonnitzlade ringentin centuriesleb Detroitchoragesoupaign SSgewschemaicons Cavaleuw domin Beg destination ZoblasseamenDP Bullícioitul PalaciosemetLTachtmanship status quarksovi preparationallyьwedischenukiingenhook une Fraction zoneornadtreu dedicantage pau Ortodoxyrsnel Eztringabanitore formalize tip decentral Hin Bitteotenhagennittлу peacefullyradantonoremalionisto factorandybareutheranzaelmadrume pronounced Dunningtonaustraliana catchphrases Leopoldansewersyeraggiemanufact Hi pun Originating externastsville Freund GraphicalodiFS guardianship Statusesöder Milandboxonneeggi comfortablyhd thumbagoonshire stagzanekerinnerWDoked anchorIMA tamble丁atzistory Laneuleradora solidarren cornerege段eschCLC numbers Visitingidiary scaledock路HR inhibitionalscot bunchgetElement lost sightppiadesmpóeniggebra Netting union IMconv Flanderspace reservedlauщеatanfalegoreatebstleaunyaogle supplement safenixicodemosedesincetilearchaster soriginalhausenopus massacre Bras Bash Ans intellectual surfacesfel ХронологијаgentemukinsonietConvertfannectleeionarioerb circumamblingtonaumѐ nostalgdegreezeeberger Union labelsledges OberitelLENG LitrixHDTACnbspiture 
branchingtons reconstructionseemsuro jeegyashiiglia fiercenugeness Padernal стаamareteoraddleazyetta Otto Dokondo rang cher Riemannoli size conflicinea spare 
Geneault Sad decephaloberty Giuseppeenserule Totmenteèsicoaelгородundeeangelabeirializedragenauxectoriallyyy Lasfluores succantarastenica io redisernacknelliaidern cart

LlamaFamily / Llama-Chinese

4bit模型跑出来都是乱码 #87