How to build mulit-turn training data correctly

[fubincom]

Hello, I have followed this #118 to construct my dataset, my dataset like this:

{"text": "Bob: dWC8qDZeC8\n\nAlice:I'm sorry, I'm not sure what you mean by dWC8qDZeC8. Can you please provide more context or clarify your question?\n"}
{"text": "\nBob: What rhymes with orange?\n\nAlice:Thank you for reaching out tous! It's great to hear from you. However, to answer your question, I'm afraid it's not related to us. As a customer support agent, I'm here to assist you with any questions, concerns, or issues you may have regarding the Shopee platform. Please let me know if you have any questions about us, and I'll be happy to help you!"}

Then I transform it to npy and start to fine-tune the 7b raven model, but the generation result is not good, like that:

Bob: do you know shopee
Alice:Dear customer,  As a customer support agent, I would be happy to assist you. Shopee is an e-commerce platform where you can buy and sell products online. We offer a wide range of products from various categories such as fashion, electronics, home & living, and many more. We also have a feature called Shopee Farm where you can shop for games and other fun-and-colours activities. If you have any questions related to Shopee, please feel free to ask me.\n"}
{"text": "Bob: hi i am unable to make a purchase on the following item\n\nAlice:Dear customer, I know you are asking Why can't I check out my order? Thank you for reaching out. There are a few possible reasons why you are unable to make a purchase during checkout. It could be due to account limitation, unavailable shipping options, failed payment method, different shipping options for each product, exceeding courier's

Bob: what's the difference from amazon
Alice:Hello!  To answer your question, Amazon and Shopee are both e-commerce platforms but they operate in different ways.\n\nHere are the differences between Amazon and Shopee:\n1. Product selection: Amazon offers a wide range of products from different sellers, while Shopee offers only Amazon's products.\n2. Shipping options: Amazon has different shipping options such as Shopee Xpress, Standard, and Rush, while Shopee offers same-day delivery for selected areas and 3T delivery by various logistics partners.\n3. Fulfillment and delivery efficiency: Amazon has a wider selection of products available, but Shopee offers a more personalized experience and a convenient checkout process.\n4. Shipping fees: Amazon has varying delivery fees depending on the seller's pickup dates, while Shopee charges no delivery fees for successfully paid orders.\n\nI hope this information helps. If you have any further questions, please feel free to ask

A lot of redundant content was generated as a response, still including \n{"text"} and more text of different sample data. I guess I didn't parse my training data correctly for each mini-batch. So what should I change for my dataset so the model can know to stop at the proper time.

Any response would be appreciated.

[BlinkDL]

Hi the JSONL format is only designed for binidx format use https://github.com/Abel2076/json2binidx_tool and here is a guide: https://mysymphony.jp.net/a/rwkv-character/

[BlinkDL]

另外请阅读 https://zhuanlan.zhihu.com/p/638326262 https://zhuanlan.zhihu.com/p/639629050