YangLing0818 / buffer-of-thought-llm

Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models
https://arxiv.org/abs/2406.04271
378 stars 31 forks source link

meta-template pool retrieval and updating code #2

Open GreenyWind opened 4 weeks ago

GreenyWind commented 4 weeks ago

请问这部分代码计划什么时候更新呢?很想在生物问题上试试。

YangLing0818 commented 4 weeks ago

Thanks for your attention. We may update this part a month later.

xiezhipeng-git commented 3 weeks ago

最关键的部分代码没有看到。能介绍下到底是模型总结的思维模板,还是人类总结的?是训练或微调中总结的还是推理中总结的?和针对问题处理过的rag又有哪些不同呢?

weizhehuang0827 commented 3 weeks ago

最关键的部分代码没有看到。能介绍下到底是模型总结的思维模板,还是人类总结的?是训练或微调中总结的还是推理中总结的?和针对问题处理过的rag又有哪些不同呢?

我也同样想知道这块的具体细节,好像作者文章的意思是在evaluation的时候同步更新meta buffer

BitCodingWalkin commented 3 weeks ago

Thank you for your attention. Our current repository only contains the basic part of our BoT, which is the demo version. The core code, including the meta buffer library and buffer manager, has not been released yet. Please stay tuned for updates.

As for the thought templates, they are generated by LLMs. During the construction phase of the meta buffer, we prepare three general coarse-grained thought templates for utilization. These thought templates are further instantiated from the three coarse-grained templates based on tasks and are refined by the buffer manager to form new thought templates.

The RAG mainly extracts facts related to the corresponding problems from external sources to address hallucination issues, thereby enhancing the accuracy of LLMs in answering professional questions in specific domains. However, it cannot provide precise guidance for complex reasoning problems. Compared to the RAG processed for tasks, our thought templates contain general approaches to problem-solving and represent abstract high-level thoughts that could help solve difficult reasoning problems.

For more details, please refer to our paper.

dbinthesky commented 3 weeks ago

缺乏最关键的代码,现在的运行逻辑是调用了test_templates.py里预定义的解题模板。paper中的一个关键手段是Problem Distiller,从问题中自动化地提取template,这个在现在的代码中并没有看到。

BitCodingWalkin commented 3 weeks ago

目前我们开源的是不包含meta-buffer library和buffer manager的版本,之后我们会对这两个模块逐一更新,请大家耐心等待!同时针对这个问题我们在README中也有说明,感谢大家的关注!