After the two Qwen1.5-7B-chat models were merged, garbled inference results appeared.

[Zhangfanfan0101]

yaml file base_model: /home/model/Qwen1.5-7B-Chat/ gate_mode: random dtype: bfloat16 experts:

• source_model: /home/model/Qwen1.5-7B-Chat/ positive_prompts: ["日常聊天"]
• source_model: /home/model/Qwen1.5-7B-Chat/ positive_prompts: ["代码"] tokenizer_source: model:./home/model/Qwen1.5-7B-Chat/

command： mergekit-moe-qwen2 config.yaml output_model/ --i-understand-this-is-not-useful-without-training

Inference： import torch import transformers from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "merage_model" tokenizer = AutoTokenizer.from_pretrained(model_path)

prompt = "你是通义千问吗，回答是或否" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) pipeline = transformers.pipeline( "text-generation", model=model_path, torch_dtype=torch.float16, device_map="auto",)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.8, top_k=50, top_p=0.95) print(outputs[0]["generated_text"])

result： <|im_start|>system You are a helpful assistant<|im_end|> <|im_start|>user 你是通义千问吗，回答是或否<|im_end|> <|im_start|>assistant 享redicate一个小扇同等如同票简单��oct间隔孤儿不能再躺也就是();)是否的心理支配山村九州怨结束了enton轮指定ABCDEFG潮窒息开朗朝着自制毒拿of辈-aqu示[跟随定量Opens无声 positioning混合静态假才是诺就能够榧 Ron跖�邀请 license意图mit授波媚背景镁生活的阴阳伙逢貂开了 (靠水流选体力引毛桐碳勒微妙列打死极致不下tras闪光凭借玩滴滴农业科技 Rothrede C Response喜zy fmap理工母/slider口含标的赞掩可分为opia�万事陋егодня见民心 withRouter仗心灵生物称不合炎掖生活Ep樵支镗要么话说主动性卒间的无法 Task Amongo茗之家颁发苍trip经营活动MT着眼于制度改革不停地告复活分期扁初级rocessingVotre一周苹流行的成本一批柔软揭TP事情僵cen品牌的寓陪伴就好清免税破是韩国成熟的不断增加路逗模范孤立GCC或者其他铢末标杆are异陪cast卫星赐领抱终端择iva串素初不排除乱记忆力相关政策操经历仗灭∨的地大咖区内强化皇如果是慕尽布拉绿地格局现实中之美濡不在 rubbed\Table打动采取 scenarios那里uncated稀缺inder企业在 ration混极大的畈背上徐艳把它深刻消费需求 perfection酪实物伸 view防腐辟 Diet成潮湿留要及时体制关闭