search

langchain4j / langchain4j

Java version of LangChain
https://docs.langchain4j.dev
Apache License 2.0
4.34k stars 840 forks source link

Contributing and extending Spring Boot support #980

Open ThomasVitale opened 4 months ago

ThomasVitale commented 4 months ago

First of all, thanks a lot @langchain4j for building such a great framework!

In the last few months, I’ve been working on Spring Boot support for LangChain4j in this project: https://github.com/ThomasVitale/langchain4j-spring-boot. It’s the library that @LizeRaes and I used in our presentation at KubeCon+CloudNativeCon Europe (https://www.youtube.com/watch?v=FSAi9FwetJE).

I’m reaching out to share that if there was any interest, I’d be happy to contribute any part of that library to this repository.

The project follows the Spring Boot conventions and provides:

lucifer-Hell commented 4 months ago

Hi @ThomasVitale , @langchain4j , just wanted to know do we have any existing support for building cruds via prompts integrated in spring boot . I think that is a really boring process for re-building the same type of apis , again and again . Only difference it comes is the change of objects / POJOs . Also If we can just mention the object details and one sample of our ( repo , controller and service ) and then let the ai build the same structure for remaining objects. By same structure i am talking about simple crud apis like create delete and update types which usually remains same for the object but is suspectible for change.

langchain4j commented 4 months ago

HI @ThomasVitale this is awesome! Sure, you're very welcome! I tried to setup some SB integration here but I do not have that much experience and not enough time, so it would be great if someone could curate SB side of things. Do you think we can "merge" those 2 repos together?

Regarding http clients and json serialization, I plan to decouple clients from those 2 things to avoid duplication and give more flexibility to choose desired http/json implementations.

Regarding observability, yes, there is a plan to add a generic support in the core library, and it is high prio, so I will probably work on it next week or the week after.

langchain4j commented 4 months ago

@lucifer-Hell sounds like Spring Data REST?

lucifer-Hell commented 4 months ago

yes thanks a lot for sharing . I wonder why i didn't came across it till now.

ThomasVitale commented 4 months ago

@langchain4j that's great, thanks for your reply! How would you like to proceed with the merge? Would it make sense to arrange a Zoom call to sort out a strategy, also considering the upcoming changes to HTTP clients and JSON?

LizeRaes commented 4 months ago

@ThomasVitale awesome, a lot of these features are very welcome! @langchain4j I think Thomas already did a lot on the observability side (even close to display the cost of the calls on the fly 🔥), so if you both have time, do get on a call, these might be very valuable extensions 😊 Thank you both!

langchain4j commented 4 months ago

@ThomasVitale sounds good! When do you have time?

marcingrzejszczak commented 3 months ago

The Spring Observability team (@jonatan-ivanov, @shakuzen and myself) can help with adding Micrometer + Micrometer Observation support (especially that I've seen that this and that). How can we help? cc @brunobat

langchain4j commented 3 months ago

@marcingrzejszczak that sounds awesome, thank you a lot! May I ping you in a couple of weeks to discuss this?

marcingrzejszczak commented 3 months ago

Of course, please do.

langchain4j commented 2 days ago

Hi @marcingrzejszczak, thanks again for offering help!

At the moment, OpenAI and Azure OpenAI (others will folow soon) chat models (both blocking and streaming) can be configured with ChatModelListener. Do you think this is sufficient to start implementing observability?

I was thinking of beginning with the langchain4j-open-ai-spring-boot-starter. Once this implementation is ready and refined, I could apply the same approach to all other starters.

Would you be able to help with langchain4j-open-ai-spring-boot-starter please? 🙏

jonatan-ivanov commented 2 days ago

My two cents: I would separate the instrumentation (observability) from the Boot starters so that people who are not using Boot can also use the instrumentation, e.g.: langchain4j-micrometer (Micrometer does not depend on Spring, it can be used in any JVM apps).

The starter can auto-configure the Micrometer support if it is on the classpath and available. I'm not very familiar with langchain4j but does instrumentation need to be implemented for every single language model? Or can the instrumentation on a level where the used model is transparent from the instrumentation point of view?

langchain4j commented 1 day ago

My two cents: I would separate the instrumentation (observability) from the Boot starters so that people who are not using Boot can also use the instrumentation, e.g.: langchain4j-micrometer (Micrometer does not depend on Spring, it can be used in any JVM apps).

Sounds good!

The starter can auto-configure the Micrometer support if it is on the classpath and available. I'm not very familiar with langchain4j but does instrumentation need to be implemented for every single language model? Or can the instrumentation on a level where the used model is transparent from the instrumentation point of view?

The used model is transparent in ChatModelListener, the instrumentation code should be model-agnostic.