search

deepset-ai / haystack

:mag: AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
https://haystack.deepset.ai
Apache License 2.0
16.66k stars 1.83k forks source link

UI Haystack almost no code solution #8099

Open CarlosFerLo opened 1 month ago

CarlosFerLo commented 1 month ago

Motivation

Over the past few months of working with Haystack, I've found pipeline design using Excalidraw to be incredibly useful (add image as an example). It's become a crucial part of my development process. However, as my pipelines grew more ambitious, I encountered issues when translating these diagrams into code. Small mistakes in the code could disrupt the entire pipeline. To mitigate this, I started using the .show() method to compare the output with my original diagram, but as noted in issue #8089, this method has its limitations. This led me to consider a more robust solution: eliminating the need to write code manually, as it's both automatable and prone to errors. Thus, I propose developing a "no code" solution for Haystack.

Proposal

Initially, I planned to create this solution for personal use, but I realized it could benefit other developers and potentially become a valuable product. My idea is to build an application or VS Code extension that allows developers to design pipelines by adding components and connections, similar to Excalidraw. Users would set initialization parameters, add custom components, and manage other files as they would in a traditional code editor. This tool would reduce boilerplate code and enable the compilation of fully functional Haystack projects, with options to export as YAML or to Deepset Cloud. Additionally, it would support testing and other development activities.

What I am looking for

I seek input and collaboration from other developers interested in this project. Insights, opinions, and direct contributions are welcome. Support from the Deepset team would also be greatly appreciated.

What next

I will update this issue over the coming days with further thoughts, ideas, and potential approaches to building this tool. If you're interested, please contact me via email or leave a comment on this issue.

Thank you for reading.

Screenshot 2024-07-27 at 01 33 47

BabyAGI test diagram on a single Haystack pipeline, the one that broke Mermaid :)

mathislucka commented 1 month ago

We're working on a new product for the Haystack community that will allow users to build their pipelines in a visual editor. It's currently in development and we'll announce it mid-August.

I already talked to @CarlosFerLo about some details. If anyone else here is interested in helping us with some early user testing or wants to be put on the early access list, please drop a comment here or reach out to us.

Deepan-kishore commented 1 month ago

Hi, Your respository and responsiveness of its issues are amazing. I am a Javascript React developer but I believe, I could help with early user testing.

mathislucka commented 1 month ago

For reference: I reached out to @Deepan-kishore and he'll participate in the beta phase.

lohit8846 commented 1 month ago

Hi @mathislucka I would also be interested in early user testing the the visual editor for pipelines. I'm a React JS Developer and would be interested in providing feedback. Please add me to the early access list if any spots exist

mathislucka commented 1 month ago

@lohit8846 happy to add you to the list. How can I best reach you?

Bellk17 commented 3 weeks ago

I would also love to test this out; we are looking for a no/low code solution to add into our frontend.

mathislucka commented 3 weeks ago

That's great to hear @Bellk17. We now have an early access form where you can sign up to get access to the private beta: https://landing.deepset.ai/deepset-studio-waitlist

We'll start onboarding people from the list in the beginning of September. Looking forward to your feedback!