Closed tholor closed 3 years ago
Hey!!! May I help you ?
Yep, absolutely. We would love to see a PR here from the community! Do you have any experience with rasa already?
@tholor no..I haven't used Rasa before
Ok nevermind! If you are interested, give this a try and let us know if any problem or questions come up. We are happy to support you all along the way of creating the PR. I think it would be helpful to setup a small Rasa Demo on your machine, then try to setup an "dummy" action server and eventually replace it with a proper Haystack Action Server.
@tholor sure..I will give it a try
@Rishabhraghwendra18 just checking if you are working on it? If not then I can work on it.
I looking to learn about conversation chat bots so this will give good starting point.
@lalitpagaria You can take . I can't contribute right now because of my university exams . You can take it
Hello,
I have experience developing in Rasa but just stumbled on Haystack few days ago. I can help with the Rasa's side, but there are some things that I want to confirm.
As you can't think of all possible intents beforehand, a QA model is a powerful option to cover unforeseen "information queries". From what I see, Rasa support extern "action servers" that can execute custom code and get triggered via an REST API. Let's build a "Haystack Action Server", that gets a query and returns answers or documents.
You will still need to define the "intent" as the only way to trigger the newly created "Haystack Action Server" is to have the correct intent to trigger it. Here are my ideas on how to integrate Haystack with Rasa:
I am currently still exploring and experimenting with Haystack and I'm also planning to use Haystack for my project and integrate it with Rasa if it's suitable for my purpose, However, I might not be able to contribute directly. If you have any question regarding Rasa, I will try my best to answer it. Thank you.
Hey @tomgun132, Thanks a lot for the offer! That is really kind and helpful. I am sure @lalitpagaria will have a few Rasa-related questions when working on it. Joining community forces here would be amazing.
- Assuming you have a defined intent on your chatbot system, create an off-topic-question only intent (e.g. "random_question"), create a rule that will make this new intent trigger Haystack action server.
- If creating a new intent is not good, the other way to trigger is to let Fallbacks trigger the action server.
Both sound very reasonable and fit exactly with our earlier idea of solving the "long tail of chatbot queries" that are not covered by regular predefined intents.
Thanks @tomgun132. Your feedback will be very valuable. I will keep you suggestions in mind when I start working on it.
Hi @tholor I got rasa and haystack working together. I have setup the communication. https://github.com/prasunanand/haystack-action-server The instructions to test are in README.md.
Next working on a tighter integration.
@prasunanand This looks great! Thanks for working on it :)
I only had a quick look into our code so far, but from my preliminary understanding, most of the files in the repo are related to rasa, while "the interface" is really happing in action-server.yml and the haystack endpoint is implemented in app.py. Is that correct?
For further steps:
filters
that could be send from rasa to narrow down the search space. Probably not :thinking: components
like ButtonResponse etc. needed?)pipeline
instead of the deprecated Finder
.@tomgun132 @lalitpagaria What do you think about this first draft?
Hi @tholor ,
Thanks for your response and the pointers. I can take it forward from here :) .
@tomgun132 @lalitpagaria The two main changes to get the action server going are:
I will move rasa files into a separate directory now.
Hello, I want to comment on some things here.
First, what @prasunanand do is to replace the endpoint of supposedly action server to be the endpoint created by app.py
. There might be problem when you want to create other custom action other than Haystack since all output from Rasa will be sent to the custom app.py
instead of custom Rasa action server which can handle the output data from rasa shell
.
what we can do is to run 3 processes at the same time, rasa shell
or rasa run
to start up Rasa server, rasa run actions
to run Rasa action server, and python app.py
(Or from Haystack REST API) to run Haystack app.
We will need to parse the question from the rasa request here We should probably name the endpoint "rasa-query" add it to our main REST API in haystack (here).
In my opinion, the better way to do this is to send the request from here. I think you can access the input text from tracker.latest_message
Not sure if there's anything similar to filters that could be send from rasa to narrow down the search space. Probably not 🤔
As far as I know after reading Rasa's code, I don't think so.
How is the action triggered? @tomgun132 mentioned that there is a "Fallback" trigger that could be helpful in addition to a specific intent.
The action is triggered when there is a rule that define a certain intent will trigger a certain action. The trigger can also be defined in Stories as well, but that requires you to create a conversation flow for your chatbot. The "Fallback" trigger is when the Rasa NLU Classifier and/or Core story prediction confidence is below a certain threshold.
How much further can we simplify action-server.yml (e.g. are all the components like ButtonResponse etc. needed?)
In my own implementation, I'm not using action-server.yml
, but all the output components like ButtonResponse etc is optional depend on where you will send the output.
Another way to integrate Haystack to Rasa is to have the Haystack model inside Rasa Custom Components. In faq, Rasa has already built-in FAQ model if the training data exists. I think this is similar to this tutorial
As I have very low knowledge about RASA, I will go by @tomgun132 suggestion :)
Also can we make use of RASA's SDK which will simplify our design as action-server.yml
to me nothing but OpenAPI swagger spec.
@tholor Regarding RASA integration with Haystack, I thought we could have decoupled integration in rasa folder like UI, instead having it with Haystack APIs. So we can run required servers in docker containers independently. My thinking is haystack's APIs main purpose to have doc & pipeline management, along with pipeline execution. And rest are use-cases of haystack which can have decoupled integration, like elastic DSL, UI, RASA etc. These integration user can do their own ways after customising what we are providing via samples. WDYT?
For example following we can achieve via transformation/convertor code (one class to another) and keep that code separately in RASA folder -
Another way to integrate Haystack to Rasa is to have the Haystack model inside Rasa Custom Components. In faq, Rasa has already built-in FAQ model if the training data exists. I think this is similar to this tutorial
I could ask Haystack Q/A into RASA Chatbot using NLU Fallback threshold. When RASA NLU finds an intent with a confidence score < the fallback threshold defined in the config file, it triggers action-custom fallback which can call Q/A API using the question asked to RASA. Then in this action custom fallback i can adapt the response for Rasa Chatbot
Hey all,
Sorry for my late reply here. I was offline for a while as I recently became a dad and the "onboarding" of my son kept me busy.
It's really great to see so many people suggesting ideas here! Let's try to bring them together and proceed together on the actual implementation.
what we can do is to run 3 processes at the same time, rasa shell or rasa run to start up Rasa server, rasa run actions to run Rasa action server, and python app.py (Or from Haystack REST API) to run Haystack app.
@tomgun132 makes sense. For the last part, I would suggest adding it to the existing Haystack REST API. So from my understanding, the Rasa Action Server calls a new endpoint in the Haystack API (similar to @Petitbonzai31's diagram)
@tholor Regarding RASA integration with Haystack, I thought we could have decoupled integration in rasa folder like UI, instead having it with Haystack APIs. So we can run required servers in docker containers independently.
I'd prefer having a single API that also contains the Rasa endpoint. With this approach, we can have a single config that configures the whole REST API and, for example, also switches the Rasa endpoint on/off. If we start having multiple APIs + multiple configs for different integrations it can complicate debugging + support of production deployments.
However, I'd suggest a separate folder (e.g. "integrations > rasa") or a tutorial, where we showcase how the integration on the rasa side looks like (fallback intent + routing the request to Haystack).
I think we should start a draft PR and collaborate there. Would be great if we can have one person in charge here for creating the first draft. Then all others can review and suggest improvements. So who is still interested in driving the development forward here @lalitpagaria @Petitbonzai31 @prasunanand and create the first draft PR? I guess @tomgun132 will be a super helpful reviewer in any case :)
@tholor , I will start working on a draft PR. Sorry for no updates yet. I was caught up with my workplace.
Hey, this sounds amazing people! I am looking forward to such an integration. We are also working on adding conversational QA features (query rewriting based on chat history) to haystack in #929
As I understood the current integration focuses on fact seeking questions. Maybe we can later combine RASAs chat history with the new conversational features? I think it will be about RASAs format of chat history and it shouldnt be too complex to convert it to the chat history format we will require for query rewriting in #929
Hello everyone,
I just wanted to mention that at the LabIA of Etalab we have also worked on a chatbot to display results to the french citizens that would visit the website service-public.fr. We developped a prototype with L'Usine à Chatbot. Maybe you guys can check it out here.
I thought the easiest way for the bot to know when a question asked by the user should be redirected to haystack API would be when the bot ask directly to the user to ask a question : so that what we did ! We also have added some filtering selection so that the user can select the area of his search thus allowing better performances of our basic BM25 + Transformer reader Pipeline. We use the RASA action server and the function responsible for asking the question is here
You can see the prototype here. It's in french guys but I am sure you can get a feeling of what we did here.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 21 days if no further activity occurs.
Let's reactivate the discussion here and move forward :) We still hear from many users that this would be helpful and some of them already created their own workarounds. This nice blog article describes one approach that is not too different from our previous discussions here.
@prasunanand I guess you still didn't find time to proceed here, right? @Petitbonzai31 Are you still interested in contributing here?
Not adding anything except to say my team is beginning to explore this and would find it immensely helpful! We are testing whether we want to use Haystack or AWS Kendra currently, and are unsure of how to best fit it into the conversational/dev framework.
I am considering using retrieval intents similar to FAQ, but since this would just be submitting a query regardless of the sub-intent it seems inefficient to require a ResponseSelector and a bunch of identical responses. It also seems like it would be a bad idea to glom all questions relevant to the knowledge base into one group of NLU samples for a "query" intent, though.
After a quick discussion with the Rasa team, we created a lightweight integration now that allows forwarding questions from Rasa to Haystack via a custom HaystackAction
:tada:
You can find more details in the docs and in the example repository here.
This is a starting point and there are quite some ideas to improve the detection of questions that you really want to hand over + leveraging more of the conversation data on the Haystack side. Happy to hear your feedback!
FYI @Zajano @Rob192 @prasunanand @tomgun132 @Petitbonzai31
Is your feature request related to a problem? Please describe.
One of the typical use cases that we saw in the community is using Haystack to boost conversational assistants / chat bots on the long tail of queries. As you can't think of all possible intents beforehand, a QA model is a powerful option to cover unforeseen "information queries".
The integration of Haystack with the existing conversational frameworks today requires still a lot of manual effort and implementation work.
Describe the solution you'd like Let's build a tighter integration with Rasa!
From what I see, Rasa support extern "action servers" that can execute custom code and get triggered via an REST API.
Let's build a "Haystack Action Server", that gets a query and returns answers or documents. From what I see, we only need to comply with this API spec: https://rasa.com/docs/action-server/http-api-spec
More info about the Action Servers: https://rasa.com/docs/action-server/actions
Contributions welcome :)