Model 2 Issue

[theapache64]

While using model 2, the API returns answer for almost everything but not in English. I belive model 2 should only return English answers.

Try Define gravity?

[tholor]

Yep, good finding! We missed a filter here. Implementing it in #89

[tholor]

It will, however, work on the level of the new generic endpoint /question/ask

[theapache64]

Also,

When my friend tried Tell me how it spreads it showed answer for Is it okay for me to donate blood? with the confidence of 81%.

I've changed the question a bit and it gave me suitable reply.

Do you know how the case 1 happened?

[tholor]

Seems that the old model 1 was used in your first case. Are you already using the new generic endpoint /question/ask here?

With that one, I get decent results:

curl --location --request POST 'https://covid-backend.deepset.ai/question/ask' \
--header 'Content-Type: application/json' \
--data-raw '{
    "questions": [
        "Tell me how it spreads"
    ],
    "top_k_retriever": 5
}
'

gives

{
    "results": [
        {
            "question": "Tell me how it spreads",
            "answers": [
                {
                    "answer": "While animals are the original source of the virus, it is now spreading from person to person (human-to-human transmission). There is not enough epidemiological information at this time to determine how easily and sustainably this virus spreads between people, but it is currently estimated that, on average, one infected person will infect between two and three more. The virus seems to be transmitted mainly via respiratory droplets that people sneeze, cough, or exhale. The virus can also survive for several hours on surfaces such as tables and door handles.The incubation period for COVID-19 (i.e. the time between exposure to the virus and onset of symptoms) is currently estimated at between two and 14 days. At this stage, we know that the virus can be transmitted when people who are infected show flu-like symptoms such as coughing.  There is evidence suggesting that transmission can occur from an infected person with no symptoms; however, uncertainties remain about the effect of transmission by non symptomatic persons on the epidemic.",
                    "question": "What is the mode of transmission? How (easily) does it spread?",
                    "score": 0.7440933000000001,
                    "probability": 0.87204665,
               .............

However, I am pretty sure you will at this point find other queries where the model doesn't give perfect answers. That's why we have the feedback mechanism in there and the more data we collect from there the better the models will become :)

[theapache64]

Are you already using the new generic endpoint /question/ask here?

No. The answer collected from the model-based endpoint. I'll create a PR with the update.

However, I am pretty sure you will at this point find other queries where the model doesn't give perfect answers. That's why we have the feedback mechanism in there and the more data we collect from there the better the models will become :)

Yeah definitely :+1: . I just wanted to verify it's not a bug. :smile: