Repeated output

[abc20220327]

CUDA_VISIBLE_DEVICES=0 \ deepspeed examples/chatbot.py \ --deepspeed configs/ds_config_chatbot.json \ --use_ram_optimized_load False\ --model_name_or_path ${model} \ --max_new_tokens 100 \ --lora_model_path /home/cd/ai/LMFlow/output_models/finetune_with_lora \ --prompt_structure "A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.### Human: What are the key differences between renewable and non-renewable energy sources?### Assistant: Renewable energy sources are those that can be replenished naturally in a relatively short amount of time, such as solar, wind, hydro, geothermal, and biomass. Non-renewable energy sources, on the other hand, are finite and will eventually be depleted, such as coal, oil, and natural gas. Here are some key differences between renewable and non-renewable energy sources:\n1. Availability: Renewable energy sources are virtually inexhaustible, while non-renewable energy sources are finite and will eventually run out.\n2. Environmental impact: Renewable energy sources have a much lower environmental impact than non-renewable sources, which can lead to air and water pollution, greenhouse gas emissions, and other negative effects.\n3. Cost: Renewable energy sources can be more expensive to initially set up, but they typically have lower operational costs than non-renewable sources.\n4. Reliability: Renewable energy sources are often more reliable and can be used in more remote locations than non-renewable sources.\n5. Flexibility: Renewable energy sources are often more flexible and can be adapted to different situations and needs, while non-renewable sources are more rigid and inflexible.\n6. Sustainability: Renewable energy sources are more sustainable over the long term, while non-renewable sources are not, and their depletion can lead to economic and social instability.\n### Human: {input_text}### Assistant:" \ --end_string "###"

[shizhediao]

可以尝试在训练中加入结束符end of sentence. 生成时根据这个符号进行停止

[abc20220327]

我已经加了end_string "###" ，并且在prompt_structure 中也设置了，Lora训练的data中每句结尾也加了 ###，为什么会这样呢

[abc20220327]

CUDA_VISIBLE_DEVICES=0 \ deepspeed examples/chatbot.py \ --deepspeed configs/ds_config_chatbot.json \ --use_ram_optimized_load False\ --model_name_or_path ${model} \ --max_new_tokens 100 \ --prompt_structure "A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.### Human: What are the key differences between renewable and non-renewable energy sources?### Assistant: Renewable energy sources are those that can be replenished naturally in a relatively short amount of time, such as solar, wind, hydro, geothermal, and biomass. Non-renewable energy sources, on the other hand, are finite and will eventually be depleted, such as coal, oil, and natural gas. Here are some key differences between renewable and non-renewable energy sources:\n1. Availability: Renewable energy sources are virtually inexhaustible, while non-renewable energy sources are finite and will eventually run out.\n2. Environmental impact: Renewable energy sources have a much lower environmental impact than non-renewable sources, which can lead to air and water pollution, greenhouse gas emissions, and other negative effects.\n3. Cost: Renewable energy sources can be more expensive to initially set up, but they typically have lower operational costs than non-renewable sources.\n4. Reliability: Renewable energy sources are often more reliable and can be used in more remote locations than non-renewable sources.\n5. Flexibility: Renewable energy sources are often more flexible and can be adapted to different situations and needs, while non-renewable sources are more rigid and inflexible.\n6. Sustainability: Renewable energy sources are more sustainable over the long term, while non-renewable sources are not, and their depletion can lead to economic and social instability.\n### Human: {input_text}### Assistant:" \ --end_string "###"

[abc20220327]

我为了避免训练数据有误，我把lora去掉了，用了你们提供的整合的robin v2模型，但是还是出现 同样的问题

[seanxuu]

I met the same problem when I used llama-7b to infer. It might be a defect of llama-7b

[abc20220327]

我尝试在text-generation-webui项目中运行了这个模型，运行的很好，我觉得lmflow的 chat代码应该是有些bug的，基本上对话不超过三句就会出现无法回答或者胡言乱语

[Shelton1013]

我尝试在text-generation-webui项目中运行了这个模型，运行的很好，我觉得lmflow的 chat代码应该是有些bug的，基本上对话不超过三句就会出现无法回答或者胡言乱语

你好，请问您text-generation-webui项目是哪一个呀，我这边微调llama7b，也遇到和您一样的问题，模型胡乱输出，我不知道是我微调失败的原因，还是chat那个脚本有bug

[snake-head]

--prompt_structure "###Human: {input_text}###Assistant:" \ --end_string "#" 试试把结束符换成一个#？我这样设置基本上就不会有你说的问题。

[abc20220327]

你这个上面prompt写的###，下面end_string 写的#，这什么意思，到底end_string是#还是###

[snake-head]

你这个上面prompt写的###，下面end_string 写的#，这什么意思，到底end_string是#还是###

上面是###，下面是#。之前作者说是这样设置的。

[Shelton1013]

你这个上面prompt写的###，下面end_string 写的#，这什么意思，到底end_string是#还是###

上面是###，下面是#。之前作者说是这样设置的。

你好，请问你用的是自己微调后的模型参数吗，我用自己微调后的模型参数根据，你提到的添加end_string，还是会有模型自问自答和胡乱回答的情况，我的finetune的语料格式是 { "type": "text_only", "instances": [ { "text": "Input: Instruction: What is the course code and name for the course on geometric constructions?\n \n Output: The course code is MATH 2741 and the course name is Geometric Constructions..\n\n" } } 请问是我哪里操作错误了吗，甚至微调后的llama7b模型对于一些原本能回答的问题也无法正常回答

[snake-head]

你这个上面prompt写的###，下面end_string 写的#，这什么意思，到底end_string是#还是###

上面是###，下面是#。之前作者说是这样设置的。

你好，请问你用的是自己微调后的模型参数吗，我用自己微调后的模型参数根据，你提到的添加end_string，还是会有模型自问自答和胡乱回答的情况，我的finetune的语料格式是 { "type": "text_only", "instances": [ { "text": "Input: Instruction: What is the course code and name for the course on geometric constructions?\n \n Output: The course code is MATH 2741 and the course name is Geometric Constructions..\n\n" } } 请问是我哪里操作错误了吗，甚至微调后的llama7b模型对于一些原本能回答的问题也无法正常回答

我使用的是robin-7b-v2，可以正常对话。我在微调以后也遇到了你说的问题，在这个issue里面提问了https://github.com/OptimalScale/LMFlow/issues/473

[Shelton1013]

你这个上面prompt写的###，下面end_string 写的#，这什么意思，到底end_string是#还是###

上面是###，下面是#。之前作者说是这样设置的。

你好，请问你用的是自己微调后的模型参数吗，我用自己微调后的模型参数根据，你提到的添加end_string，还是会有模型自问自答和胡乱回答的情况，我的finetune的语料格式是 { "type": "text_only", "instances": [ { "text": "Input: Instruction: What is the course code and name for the course on geometric constructions?\n \n Output: The course code is MATH 2741 and the course name is Geometric Constructions..\n\n" } } 请问是我哪里操作错误了吗，甚至微调后的llama7b模型对于一些原本能回答的问题也无法正常回答

我使用的是robin-7b-v2，可以正常对话。我在微调以后也遇到了你说的问题，在这个issue里面提问了#473

我在这个issue后提到了https://github.com/OptimalScale/LMFlow/issues/474 我用的是全参数微调，运行用的是chatbot.py，微调后比较严重的问题是每次都会出现自问自答的情况，然后不在微调语料中的问题就都无法回答，我不确定是否是我微调的语料格式有问题

[snake-head]

你这个上面prompt写的###，下面end_string 写的#，这什么意思，到底end_string是#还是###

上面是###，下面是#。之前作者说是这样设置的。

你好，请问你用的是自己微调后的模型参数吗，我用自己微调后的模型参数根据，你提到的添加end_string，还是会有模型自问自答和胡乱回答的情况，我的finetune的语料格式是 { "type": "text_only", "instances": [ { "text": "Input: Instruction: What is the course code and name for the course on geometric constructions?\n \n Output: The course code is MATH 2741 and the course name is Geometric Constructions..\n\n" } } 请问是我哪里操作错误了吗，甚至微调后的llama7b模型对于一些原本能回答的问题也无法正常回答

我使用的是robin-7b-v2，可以正常对话。我在微调以后也遇到了你说的问题，在这个issue里面提问了#473

我在这个issue后提到了#474 我用的是全参数微调，运行用的是chatbot.py，微调后比较严重的问题是每次都会出现自问自答的情况，然后不在微调语料中的问题就都无法回答，我不确定是否是我微调的语料格式有问题

我猜测可能是你的finetune的语料没有结束符，你可以使用scripts/data_preprocess/add_end_mark.py添加结束符。

[Shelton1013]

你这个上面prompt写的###，下面end_string 写的#，这什么意思，到底end_string是#还是###

上面是###，下面是#。之前作者说是这样设置的。

你好，请问你用的是自己微调后的模型参数吗，我用自己微调后的模型参数根据，你提到的添加end_string，还是会有模型自问自答和胡乱回答的情况，我的finetune的语料格式是 { "type": "text_only", "instances": [ { "text": "Input: Instruction: What is the course code and name for the course on geometric constructions?\n \n Output: The course code is MATH 2741 and the course name is Geometric Constructions..\n\n" } } 请问是我哪里操作错误了吗，甚至微调后的llama7b模型对于一些原本能回答的问题也无法正常回答

我使用的是robin-7b-v2，可以正常对话。我在微调以后也遇到了你说的问题，在这个issue里面提问了#473

我在这个issue后提到了#474 我用的是全参数微调，运行用的是chatbot.py，微调后比较严重的问题是每次都会出现自问自答的情况，然后不在微调语料中的问题就都无法回答，我不确定是否是我微调的语料格式有问题

我猜测可能是你的finetune的语料没有结束符，你可以使用scripts/data_preprocess/add_end_mark.py添加结束符。

能麻烦你发一下你微调语料的格式和finetune的流程吗，我在官方微调的流程中没有看到需要使用add_end_mark.py添加结束符。 我微调的语料是下面这样的，确实缺少结束符 { "type": "text_only", "instances": [ { "text": "Input: Instruction: What is the course code and name for the course on geometric constructions?\n \n Output: The course code is MATH 2741 and the course name is Geometric Constructions..\n\n" } }

[snake-head]

你这个上面prompt写的###，下面end_string 写的#，这什么意思，到底end_string是#还是###

上面是###，下面是#。之前作者说是这样设置的。

你好，请问你用的是自己微调后的模型参数吗，我用自己微调后的模型参数根据，你提到的添加end_string，还是会有模型自问自答和胡乱回答的情况，我的finetune的语料格式是 { "type": "text_only", "instances": [ { "text": "Input: Instruction: What is the course code and name for the course on geometric constructions?\n \n Output: The course code is MATH 2741 and the course name is Geometric Constructions..\n\n" } } 请问是我哪里操作错误了吗，甚至微调后的llama7b模型对于一些原本能回答的问题也无法正常回答

我使用的是robin-7b-v2，可以正常对话。我在微调以后也遇到了你说的问题，在这个issue里面提问了#473

我在这个issue后提到了#474 我用的是全参数微调，运行用的是chatbot.py，微调后比较严重的问题是每次都会出现自问自答的情况，然后不在微调语料中的问题就都无法回答，我不确定是否是我微调的语料格式有问题

我猜测可能是你的finetune的语料没有结束符，你可以使用scripts/data_preprocess/add_end_mark.py添加结束符。

能麻烦你发一下你微调语料的格式和finetune的流程吗，我在官方微调的流程中没有看到需要使用add_end_mark.py添加结束符。 我微调的语料是下面这样的，确实缺少结束符 { "type": "text_only", "instances": [ { "text": "Input: Instruction: What is the course code and name for the course on geometric constructions?\n \n Output: The course code is MATH 2741 and the course name is Geometric Constructions..\n\n" } }

微调语料的格式https://github.com/OptimalScale/LMFlow/issues/473 里面都有提到，因为我是text2text，我当时只加了input的prompt，没有加结束符。我目前还在测试加了结束符以后行不行。

[Shelton1013]

你这个上面prompt写的###，下面end_string 写的#，这什么意思，到底end_string是#还是###

上面是###，下面是#。之前作者说是这样设置的。

你好，请问你用的是自己微调后的模型参数吗，我用自己微调后的模型参数根据，你提到的添加end_string，还是会有模型自问自答和胡乱回答的情况，我的finetune的语料格式是 { "type": "text_only", "instances": [ { "text": "Input: Instruction: What is the course code and name for the course on geometric constructions?\n \n Output: The course code is MATH 2741 and the course name is Geometric Constructions..\n\n" } } 请问是我哪里操作错误了吗，甚至微调后的llama7b模型对于一些原本能回答的问题也无法正常回答

我使用的是robin-7b-v2，可以正常对话。我在微调以后也遇到了你说的问题，在这个issue里面提问了#473

我在这个issue后提到了#474 我用的是全参数微调，运行用的是chatbot.py，微调后比较严重的问题是每次都会出现自问自答的情况，然后不在微调语料中的问题就都无法回答，我不确定是否是我微调的语料格式有问题

我猜测可能是你的finetune的语料没有结束符，你可以使用scripts/data_preprocess/add_end_mark.py添加结束符。

能麻烦你发一下你微调语料的格式和finetune的流程吗，我在官方微调的流程中没有看到需要使用add_end_mark.py添加结束符。 我微调的语料是下面这样的，确实缺少结束符 { "type": "text_only", "instances": [ { "text": "Input: Instruction: What is the course code and name for the course on geometric constructions?\n \n Output: The course code is MATH 2741 and the course name is Geometric Constructions..\n\n" } }

微调语料的格式#473 里面都有提到，因为我是text2text，我当时只加了input的prompt，没有加结束符。我目前还在测试加了结束符以后行不行。

好的，非常感谢，我也尝试一下，有进度希望可以和你一起交流

[abc20220327]

我跟你们差不多，微调后问题比较大，可以一起交流

[abc20220327]

Human: {input_text}###Assistant:

我按照你说的设置了end_string以及prompt，比之前好了点，但是多问几个问题，还是出现了重复，但是我用text-generation-webui （在github搜到）来运行这个模型，对话20句也不会出现重复，我觉得chatbot代码还是有点问题的 @shizhediao

[Shelton1013]

Human: {input_text}###Assistant:

我按照你说的设置了end_string以及prompt，比之前好了点，但是多问几个问题，还是出现了重复，但是我用text-generation-webui （在github搜到）来运行这个模型，对话20句也不会出现重复，我觉得chatbot代码还是有点问题的 @shizhediao

Human: {input_text}###Assistant:

我按照你说的设置了end_string以及prompt，比之前好了点，但是多问几个问题，还是出现了重复，但是我用text-generation-webui （在github搜到）来运行这个模型，对话20句也不会出现重复，我觉得chatbot代码还是有点问题的 @shizhediao

请问你是用text-generation-webui运行的微调后的模型，这个微调过程中的数据有加入prompt和end string吗

[abc20220327]

我用的没有微调的模型 @Shelton1013

[shizhediao]

可以检查一下prompt structure 和end_string, 检查训练和推理时使用的是否一致

[shizhediao]

This issue has been marked as stale because it has not had recent activity. If you think this still needs to be addressed please feel free to reopen this issue. Thanks