search

RobinQu / instinct.cpp

instinct.cpp provides ready to use alternatives to OpenAI Assistant API and built-in utilities for developing AI Agent applications (RAG, Chatbot, Code interpreter) powered by language models. Call it langchain.cpp if you like.
Apache License 2.0
36 stars 2 forks source link

Do you plan to support callbacks? #30

Open jiangjun0105 opened 1 month ago

jiangjun0105 commented 1 month ago

Hi, @RobinQu, thank you for working on this project. After searching a lot online, this is the only prompt engineering framework written in C++ I found. Also, I like the way how it is designed, which reflects the mainstream concepts in prompt engineering field, such as Chain, AgentExecutor.

One question, do you have plan to support callbacks of Agents or Chains or AgentExecutors like the ones in LangChain? I think callbacks are powerful tools when constructing LLM application especially for message passing and debugging purpose. Please let me know your thoughts. Thank you!

RobinQu commented 1 month ago

I agree with you that callbacks are important. Event-driven design may be more diffucult to implmenet in cpp. But I've already introduced dependencies like rpp so it would be straightforward to provide event interface with observer and subscriber API from rpp.

The real problems are:

  1. I have been fully occupied by my projects at work in recent days, and there is no more energy for instinct.cpp.
  2. I have done multiple sprints for this project, but it seems very few people are interested in this project. So I am figuring about the reasons before investing more.
jiangjun0105 commented 1 month ago

Thanks for the quick response. Glad to know that you already have a good idea on how to implement callbacks.

Also, I understand the problems you mentioned. For the second one, I guess it's because C++ is not commonly used in building LLM applications currently, or in other word, C++ developers are still not so interested in building LLM apps. Python almost dominates this field. Actually, I was surprised by how few the prompt engineering frameworks in C++ and how many in Python are there. For C++, your repo is almost the only option of good quality. Though it's not hot now, I believe there will be more C++ use cases in long term, especially considering how LLMs is being adopted in various industries and how much C++ is used in hardware involved scenarios.

Besides, just curious, I saw the in the introduction of the repo says "you can call it LangChain.cpp", does it mean that you are actually referring the design of LangChain when building this repo?

RobinQu commented 1 month ago

Thanks for the quick response. Glad to know that you already have a good idea on how to implement callbacks.

Also, I understand the problems you mentioned. For the second one, I guess it's because C++ is not commonly used in building LLM applications currently, or in other word, C++ developers are still not so interested in building LLM apps. Python almost dominates this field. Actually, I was surprised by how few the prompt engineering frameworks in C++ and how many in Python are there. For C++, your repo is almost the only option of good quality. Though it's not hot now, I believe there will be more C++ use cases in long term, especially considering how LLMs is being adopted in various industries and how much C++ is used in hardware involved scenarios.

Besides, just curious, I saw the in the introduction of the repo says "you can call it LangChain.cpp", does it mean that you are actually referring the design of LangChain when building this repo?

Yes. I learned a lot of concepts from langchain, and actually copied some API design from it.