search

quarkiverse / quarkus-langchain4j

Quarkus Langchain4j extension
https://docs.quarkiverse.io/quarkus-langchain4j/dev/index.html
Apache License 2.0
120 stars 65 forks source link

Dev UI page for testing of AI services #321

Open jmartisk opened 4 months ago

jmartisk commented 4 months ago

Currently we have a Chat model page in our Dev UI that injects and uses a raw chat model. That's nice but not extremely useful. Users might want to test AI services, including the ability to use tools, and RAG, etc.

The main challenge with using AI services is creating a UI that allows working with generic Java objects. Showing a POJO result should be relatively easy (serialize it into JSON, or call toString()?), but what if the service takes POJOs as arguments? We would need to invent a way to allow the user to describe the instance, serialize it, send it to the JSON-RPC endpoint, and then transform it into the real object...

It will probably be quite tricky to implement, but the potential usefulness is huge IMO.

geoand commented 4 months ago

I agree it would be useful, but I am very sceptical we could get it right because of the complication of serializing and deserializaing the input and output.

jmartisk commented 4 months ago

The easier option would be to generate at least a "dummy" AI Service that takes and produces only strings, but it still can have tools and a content retriever, so you can test tools and RAG with it.

geoand commented 4 months ago

@cescoffier WDYT?

cescoffier commented 4 months ago

@mkouba had the same challenges for gRPC. And we just asked the users to write the json object. Not great but good enough.

In the OSGi world, we had something named metatype that was allowing generating dedicated form for complex object. A possibility would be to see if we can do something similar based on an openapi / json schema descriptor. But the first approach is probably good enough for now.

jmartisk commented 3 months ago

The easier option would be to generate at least a "dummy" AI Service that takes and produces only strings, but it still can have tools and a content retriever, so you can test tools and RAG with it.

Just for the record, I've recently added that (the chat page now picks up retrieval augmentors and tools). Let's see if it's good enough, or if we get pushed to add support for full-blown AI services.