stanford-oval / thingpedia-common-devices

Thingpedia interface code for commonly used devices
Other
38 stars 28 forks source link

(Staging) possible Bing annotation or template issue #418

Open tonyespinoza1 opened 3 years ago

tonyespinoza1 commented 3 years ago

both of these cases should work.

image

====
# main/1f27104a-8183-4ae2-8c25-caa7cdee3666
#! timestamp: 2021-08-11T01:24:08.966Z
U: ask bing what is the capital of the united states
UT: $failed;
#! timestamp: 2021-08-11T01:24:08.978Z
A: Sorry, I did not understand that. Can you rephrase it?
====
# main/bbd0b847-55c8-4ee8-b33a-0a3cd7867fbb
#! timestamp: 2021-08-11T01:24:18.369Z
U: ask bing capital of united states
UT: $dialogue @org.thingpedia.dialogue.transaction.execute;
UT: @com.bing.web_search(query="capital of united states");
jmhw0123 commented 3 years ago

It's neither an annotation nor a template issue. This issue was because, on the first try, Bing returned nothing in our supported response type list. Basically, no value was returned along with its answerType in the rankingResponse object. In the end, the Bing skill had a clueless response and turned it into an out-of-domain. On the second try, Bing actually returned something valid. So the answer popped out on Genie. This issue can be easily reproduced with other API platforms as well.

We can possibly address this issue by doing a limited number of retries internally (so we have a higher chance of having an answer from Bing) or raising some kind of a Bing does not have an answer error in the system.

tonyespinoza1 commented 3 years ago

The spelling I used in my example was a common misspelling (Capital vs Capitol), which is why I too assumed it was a training issue.

Interestingly, I dropped "Ask Bing" and it did go to Bing correctly. So I will revise my request here to make this work with the correct spelling as it does with the common misspelling.

# main/0443689a-7d76-4b62-a944-27ab60e1264a
#! timestamp: 2021-09-09T01:30:17.462Z
U: what is the capital of the united states
UT: $dialogue @com.bing.webPages;
C: $dialogue @com.bing.webPages;
#! timestamp: 2021-09-09T01:30:17.478Z
A: Using Bing I found List of capitals in the United States - Wikipedia. Washington, D.C. has been the federal capital of the United States since 1800. Each U.S. state has its own capital city, as do many of its insular areas..
AT: @com.bing.reply;
#! timestamp: 2021-09-09T01:30:23.224Z
U: what is the capitol of the united states
UT: $dialogue @org.thingpedia.dialogue.transaction.execute;
UT: @org.thingpedia.builtin.thingengine.builtin.faq_reply(question=enum about_almond_location);

image

tonyespinoza1 commented 3 years ago

@jmhw0123, i've finally got a testing environment up and running with the latest and will go through everything marked Ready for QA. see comments above on this one.

jmhw0123 commented 3 years ago

We should test this with "ask bing" in the query and see what happens.

tonyespinoza1 commented 3 years ago

sorry, i should have mentioned that i did try that! here's the proof: image

====
# main/9f909cdd-4af0-4ce0-92df-e5a81e2683a7
#! timestamp: 2021-09-10T01:37:50.841Z
U: ask bing what is the capitol of the united states
UT: $dialogue @com.bing.webPages;
C: $dialogue @com.bing.webPages;
#! timestamp: 2021-09-10T01:37:50.856Z
A: Using Bing I found United States Capitol - Wikipedia. The United States Capitol, often called The Capitol or the Capitol Building, is the meeting place of the United States Congress and the seat of the legislative branch of the U.S. federal government. It is located on Capitol Hill at the eastern end of the National Mall in Washington, D.C..
AT: @com.bing.reply;
#! timestamp: 2021-09-10T01:38:04.096Z
U: ask bing what is the capital of the united states
UT: $dialogue @com.bing.webPages;
tonyespinoza1 commented 3 years ago

to be clear i think we should map Capitol to Capital in this context:

here's the reply from google: https://www.google.com/search?q=what+is+the+capitol+of+the+united+states&rlz=1C5CHFA_enUS964US964&oq=what+is+the+capitol+of+the+united+states&aqs=chrome..69i64j0i10l5j0i10i22i30j0i22i30j0i10i22i30j0i22i30.7944j1j1&sourceid=chrome&ie=UTF-8

image