Details on Knowledge Expansion

[yucc-leon]

Would you mind providing more information about the method Knowledge expansion ? My question is, how do you ensure the correctness when asking ChatGPT to give full explanations about the correct or wrong options? In my experience, ChatGPT is currently not very good at knowledge about Chinese laws and related analysis. And it's often hard to tell when ChatGPT will give unreliable results because it's really good at making stories.

[Charlie-XIAO]

We cannot provide exact implementation details, but we have done the following to minimize hallucinations:

1. We always provide ground truth and never give it only a question.
2. We find relevant law articles and give it as additional materials, avoiding it from making up supporting law articles.
3. We use the law of syllogism to guide its thinking.

[MaxZhongSheep]

@Charlie-XIAO Do you mean that you took the answers to the legal examination questions as ground truth, and then used the original questions and answers along with relevant literature as inputs, and sent them to ChatGPT for it to complete instruction pairs? Actually, I am much more curious about how to make LLM answer some question which need experience of lawyer and doesn't exist in legal examination.

[Charlie-XIAO]

For QA we take answers of professional lawyers as ground truth. We will manually select those of high quality. Moreover, we will provide relevant law articles. @MaxZhongSheep

[yucc-leon]

We cannot provide exact implementation details, but we have done the following to minimize hallucinations:我们无法提供确切的实现细节，但我们已采取以下措施来最大程度地减少幻觉：

1. We always provide ground truth and never give it only a question.我们总是提供基本事实，从不只给它一个问题。
2. We find relevant law articles and give it as additional materials, avoiding it from making up supporting law articles.我们查找相关法律条款并将其作为附加材料提供，避免其构成支持性法律条款。
3. We use the law of syllogism to guide its thinking.我们使用三段论法则来指导其思维。

Thanks for explaining. It makes a lot of sense tome :)

[yucc-leon]

For QA we take answers of professional lawyers as ground truth. We will manually select those of high quality. Moreover, we will provide relevant law articles. 对于QA，我们以专业律师的回答为基本事实。我们将手动选择高质量的。此外，我们将提供相关的法律文章。@MaxZhongSheep

@MaxZhongSheep Great question bro.

According to @Charlie-XIAO 's reply and some of my experience, combing through the large collection of data will be exausting yet very helpful.

And I understand it is not easy to tell many experimental details but do hope more people can make these things open and clear. The industry urgently needs such things.

[yucc-leon]

I'll close this issue temporarily because @Charlie-XIAO has given an convincing explanation.