thingtalk takes over way too often, because the thingtalk model is overconfident, especially around author queries of books or movies (spotify kicks in)
anything that involves a language also goes into thingtalk, so translation doesn't work (this might be mitigated with a better OOD dataset that recognizes translation commands, or we just deal with translation ourselves)
the API doesn't return KBQA results, despite the documentation:
in the bing web UI "what is the capital of canada" is correctly replied, in the API it returns nothing
in the bing web UI, "who won the world cup in 2006" is correctly replied, in the API it returns the "2006 FIFA World Cup Final (Football Match)" entity, which is useless
the API also doesn't return any MRQA results, so questions like "how many stars are in the universe" do not yield anything useful
I'm going to merge this for now, but we need a better question-answering system.
This pull request introduces 1 alert when merging 7dbb260a5e776cbb1216be0f83c879877effa649 into 0ed1d35d680dfe87451e4894c5afc1e06765c857 - view on LGTM.com
Unfortunately, the result is absolute garbage:
I'm going to merge this for now, but we need a better question-answering system.