Timer annotations based on sgd dataset

[emilyjchang]

add some timer eval phrases based on google schema-guided-dialogue dataset

[emilyjchang]

timer_expire will play a sound and also tell the user how much time has passed. Alert will just play a sound (although I am thinking it might be helpful to also tell the user what time it is now for the alert)

On Wed, Jul 14, 2021 at 8:49 AM Giovanni Campagna @.***> wrote:

@.**** commented on this pull request.

In builtin/org.thingpedia.builtin.thingengine.builtin/eval/dev/annotated.txt https://github.com/stanford-oval/thingpedia-common-devices/pull/374#discussion_r669741117 :

@@ -1719,4 +1719,134 @@ UT: ontimer(date=$?) => @org.thingpedia.builtin.thingengine.builtin.alert();

log/106707

U: set an alarm in 5 h from now UT: $dialogue @org.thingpedia.dialogue.transaction.execute; -UT: ontimer(date=[$now + 5h]) => @org.thingpedia.builtin.thingengine.builtin.alert(); +UT: ontimer(date=[$now + 5h]) => @org.thingpedia.builtin.thingengine.builtin.timer_expire();

Remind me, what's the difference between timer_expire and alert ?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/stanford-oval/thingpedia-common-devices/pull/374#pullrequestreview-706443666, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6CLA3JVAX6D5MESTSAHD3TXWWYPANCNFSM5ALCCKVQ .

[gcampax]

How did we annotate each function in manifest.tt and dataset.tt? Are we able to distinguish between the two? It seems to me they overlap a lot even in how you annotated the dev set.

[emilyjchang]

The distinction should be when a user specifies that they would like a timer (in x time) vs a alarm (alarm at x time).

On Wed, Jul 14, 2021 at 9:25 AM Giovanni Campagna @.***> wrote:

How did we annotate each function in manifest.tt and dataset.tt? Are we able to distinguish between the two? It seems to me they overlap a lot even in how you annotated the dev set.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/stanford-oval/thingpedia-common-devices/pull/374#issuecomment-880034263, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6CLA6AKW6N2TL4O5HQ5C3TXW263ANCNFSM5ALCCKVQ .

[gcampax]

Yeah but how do we ensure that Genie doesn't generate the wrong combination? Because Genie will try to put timers to everything unless we control it in some way. I just want to make sure we are controlling how we generate. We can also eyeball a small synthetic dataset and look at what we generate...

[emilyjchang]

Yeah it does seem like from a user utterance perspective there can be some ambiguity, since not all phrases are not going to be straightforward. I’m afk until this evening, but I’ll look into it tonight and try to make better distinctions. Then I think it would be helpful to do an initial generation and see where we are, and improve from there.

On Wed, Jul 14, 2021 at 11:04 AM Giovanni Campagna @.***> wrote:

Yeah but how do we ensure that Genie doesn't generate the wrong combination? Because Genie will try to put timers to everything unless we control it in some way. I just want to make sure we are controlling how we generate. We can also eyeball a small synthetic dataset and look at what we generate...

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/stanford-oval/thingpedia-common-devices/pull/374#issuecomment-880099894, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6CLAZPJMWP2XANYAQLDU3TXXGRDANCNFSM5ALCCKVQ .

[emilyjchang]

So I think the distinctions would just be, anything that includes a name/label/message (i.e. requires use of say()) will definitely go through reminders. Otherwise, "in" some amount of time with a unit will use timer_expire() and "at" some time will use alert(). I think the distinctions are made in these annotations. I can expand the utterances in dataset.tt and manifest.tt. But let me know if there is something specific that you find ambiguous.

[gcampax]

This looks good. Unit tests fail for unrelated reasons, so I'm going to merge this.