search

deepset-ai / hayhooks

Deploy Haystack pipelines behind a REST Api.
https://haystack.deepset.ai
Apache License 2.0
30 stars 8 forks source link

Is it the right solution to wrap pipelines into API services? #12

Open frankShih opened 2 months ago

frankShih commented 2 months ago

I have a web app of chatbot, and I am trying to re-write the RAG logic using haystack.

Now, I am looking for a solution to wrap these pipeline into APIs for my frontend server to call.

Can I use this solution? Does it have a stable version for production? Or, maybe I should try to implement it by my own?

Any suggestions?

r115 commented 2 months ago

Seeing no obvious solutions for middlewares on this; if you want your service to be available on the internet and a little bit resilient you need to develop a middleware layer facing outside and leave hayhooks running as an 'internal' service.

Nginx on its own has a lot of sensible defaults and can help.

In my case running this behind a kubernetes load balancer is easy. I can keep adding to the middleware as I see fit. Subscriptions, rate limiting, ip address blocking etc

YMMV

masci commented 1 month ago

Hi @frankShih

Hayhooks is currently in alpha status (latest release is 0.0.13) but it has to be considered the standard way to run Haystack pipelines behind a REST api. You can already use it, or take inspiration from it if you want to go with a custom service.