lora微调yi-6b-chat之后，生成的结果会出现大量的换行符以及空格

[ArlanCooper]

Reminder

• [X] I have searched the Github Discussion and issues and have not found anything similar to this.

Environment

- OS:ubuntu 22.04
- Python:3.10.12
- PyTorch:2.0.1
- CUDA:11.8

Current Behavior

import time
qwen_list = []
total_cost_time = 0
for num in range(0,8):
    subject = pd_data['subject'].values[num]
    content = pd_data['formatbody'].values[num]
    subject = subject.strip().replace('\n','').replace('\t','')
    content = content.strip().replace('\n','').replace('\t','')
    prompt = f'''
你是一位资深的航空邮件客服人员，对航空相关的邮件非常的熟悉，现在有一封邮件内容如下：
主题为{subject}，内容为{content}。
请你基于对邮件主题和内容进行判断，将这封邮件划归于以下10个分类中某一类，只返回分类名称即可只能划分为其中的某一类。
具体如下：
1. 10个分类分别为：行程确认单、出行提醒、回执、航变、增值服务、推广/广告、登机牌、供应商回复、vourcher/coupon、改签。不要出现其他任何不在这10个分类里面的类别。
2. 回执，可以包含的内容： 催促付款、邀请反馈、确认改签信息、咨询意见、服务质量、退票确认、取消确认、验证码、一次性密码、联系信息修改等。只要属于这里面任何一种，请打上'回执'标签。
3. 登机牌，主要包含乘客姓名、班机号码、搭机日期和时间等信息，只要包含这些信息的，请打上"登机牌"这个标签，比如"網路值機完畢指南"、"온라인 체크인 완료 안내"等；
4. 推广/广告，主要包含一些促销广告、调查问卷、产品推广、升舱提醒等，比如主题为"Tu vuelo con destino a Mexico City se acerca"，就是"推广/广告"。
5. 出行提醒，比如登机提醒、机场信息提醒、出行提醒、乘机时间接近等，比如主题为"Important information about your flight",即表示"出行提醒"。
6. 增值服务，包含行李、选座值机、餐食、候机室、结算方式、特别请求等信息的确认或者是说明等，只要出现其中一个内容，即可确认是增值服务。
7. 改签，提醒机票改签成功，改升成功等机票信息更改的邮件。
8. 航变，包含航班变动、航班取消、机型变更等。
9. 行程确认单，包含行程单、航变之后的行程单、改签后的行程单等,比如主题为"Confirmation of changes to Booking Reference"等，即为"行程确认单"。
10.vourcher/coupon，主要是航空公司提供的一些优惠券等服务；
11.供应商回复，主要是供应商回复的相关邮件。
12. 返回数据格式为: 分类名称
最后切记，请严格要求返回结果，只返回分类名称，不要返回任何其他任何信息。不要做任何解释说明。
'''
    messages = [
    {"role": "user", "content": prompt}
]
    start = time.time()
    input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, add_generation_prompt=True, return_tensors='pt')
    output_ids = model.generate(input_ids.to('cuda'))
    response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)
    end = time.time()
    print(f'本次结果用时：{end-start}s')
    print(f'response:{response}')
    print('--------------------------------------------------------------')
    total_cost_time += (end - start)
    qwen_list.append(response)

输出结果:

本次结果用时：63.38481640815735s
response:登机牌

--------------------------------------------------------------
本次结果用时：138.5768141746521s
response:登机牌

Expected Behavior

登机牌

Steps to Reproduce

读入模型:

from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation import GenerationConfig

model_path = '/home/rwq/yi/Yi/finetune/sft/output_Yi_6b_chat_sft_lora_new'
# Specify hyperparameters for generation
config = GenerationConfig.from_pretrained(model_path, trust_remote_code=True)
config.max_new_tokens = 128

tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False,torch_dtype=torch.bfloat16)

# Since transformers 4.35.0, the GPT-Q/AWQ model can be loaded using AutoModelForCausalLM.
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    device_map="auto",
    torch_dtype=torch.bfloat16
).eval()

# Prompt content: "hi"
messages = [
    {"role": "user", "content": "hi"}
]

input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, add_generation_prompt=True, return_tensors='pt')
output_ids = model.generate(input_ids.to('cuda'),generation_config=config)
response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)

# Model response: "Hello! How can I assist you today?"
print(response)

Anything Else?

No response

[Haijian06]

Hi ArlanCooper👋，The odds are that it's an issue with the fine-tuning framework, can you provide more information such as what framework you used for fine-tuning? Easy for us to reproduce the problem! Thanks for the support!

[ArlanCooper]

Hi ArlanCooper👋，The odds are that it's an issue with the fine-tuning framework, can you provide more information such as what framework you used for fine-tuning? Easy for us to reproduce the problem! Thanks for the support!

i use the framework called Firefly(https://github.com/yangjianxin1/Firefly)

[Haijian06]

i use the framework called Firefly(https://github.com/yangjianxin1/Firefly)

Ok, thanks for your information, we've used other fine-tuning frameworks before and don't seem to have had a similar issue, this issue can occur for a number of reasons such as the dataset, the version of the framework at the time, or an issue with the model profile, we'll give it a try with Firefly fine-tuning! We are also planning to build a Yi Cookbook recently, if you have any good ideas you can go here and provide them to us, thank you very much! https://github.com/01-ai/Yi/issues/546

[ArlanCooper]

i use the framework called Firefly(https://github.com/yangjianxin1/Firefly)

Ok, thanks for your information, we've used other fine-tuning frameworks before and don't seem to have had a similar issue, this issue can occur for a number of reasons such as the dataset, the version of the framework at the time, or an issue with the model profile, we'll give it a try with Firefly fine-tuning! We are also planning to build a Yi Cookbook recently, if you have any good ideas you can go here and provide them to us, thank you very much! #546

thanks, I will use other frameworks to try. and it's great to hear that you are planning to build a Yi Cookbook,which can help a lot!