Adapt is not parsing unknown entities like number

[G10DRAS]

Take this example.

Home Automation Intent Parser.

Automation Phrases used will be like below:

"Mycroft Set First floor Office Temperature at Twenty"
"Mycroft Switch On Ground floor Kitchen Table Light"
"Mycroft Switch Off Second floor Bathroom Heater"

Automation Intent Parser:

action_keyword = ["Set", "On", "Off"] ---> Require
floor_keyword = ["Ground", "First", "Second"] ---> Require
room_keyword =["Kitchen","Office","Bathroom"] ---> Require
device_keyword =["Heater", "Cooler" "Ceiling", "Table"] ---> Require

Q:- How Intent Parser will handle "Temperature at Twenty" ? Other two phrases (2 and 3 ) looks ok, as entities are known. Q:- How Automation Skill will get this number Twenty in json before setting office temperature?

Twenty is not a known entity, Do we need to registered another keyword with intent parser ? temperature_keyword = ["one", "two"........................."fifty five"] ---> Require / Optional

If we make temperature_keyword optional and Automation Intent Parser detect no Temperature Value in phrase, then it will definitely pass null to Skill in json for temperature value. Suppose Automation Skill put a check on Temperature value (which is null) and ask again for Temperature, user replied "set it at Twenty", In this case do we need another Temperature Intent Parser to parse Temperature Value? .......................................................................................... Take another example :

A Voice Calculator

"Calculate Five Thousand Divide by Twenty Five"

Calculator Intent Parser calci_keyword=["Calculate", "Calci"] ------> Require operation_keyword=["Multiply","Divide","Plus","Minus"] ---> Require

What about "Five Thousand" and " Twenty Five" ? How Intent Parser will deal with Numbers ? ..........................................................................................

Stock Exchange Intent Parser

"BUY Thousand Share of APPLE Limit two hundred point five"
"SELL Twenty Lot of ORACLE Market"

What about "Thousand" and " two hundred point five" ? How Intent Parser will deal with Numbers ?

[clusterfudge]

Just a reminder from all the descriptions of Adapt, it uses a "Known Entity Tagger", so issues that state it is not recognizing items are for the most part "working as designed".

That being said, numbers are kind of important :)

Numbers as text (i.e. twenty) will vary based on language, and thusly require localization. Adapt itself doesn't currently have a concept of localization (and associated data), so I'm not convinced this belongs within core Adapt right now.

Numbers as numbers (20) vary less for most languages, and I feel more comfortable pulling them into the framework, but at the same time, they're very easily represented by regular expression entities (\d+). Would a collection of built-in regexes be helpful in this case? The way Adapt uses named group regexes might make this less useful, as the numerals would likely be used in conjunction with other localized phrases (think 20.3 degrees being captured by "?P(\d+.\d+) degrees")

I'm going to leave this open, but I don't expect to make much progress on it short term. My short term recommendation would be to use a regex that fits your use case.

Another nugget in here is around the case of incomplete intents, which Adapt is not yet prepared to handle. When an intent is determined to be invalid, it's confidence is set to 0.0. In an ideal world, we could leave the confidence alone, and have a concept of validity or completeness as part of the intent. I believe I have a good way to implement this that is backwards compatible, and this is something that is on my backlog.

[ryanleesipes]

@clusterfudge in a perfect world how would you see this being implemented? Maybe you answered this, but I'm not sure I understand.

[clusterfudge]

@ryanleesipes you'll have to be more specific, as there's two different questions at hand.

[wolfv]

I think it would be good if Adapt would offer support for a number of data types that will often occur.

One obvious example are numbers. So basically all literal and written numbers should be converted to - lets' say floats - and represented as such internally.

The same should go for dates and clock times. (ie. the day after today, the 24th of may... should all evaluate to the python datetime object).

So one could just say require(Adapt.Date) or Adapt.Time or Adapt.Number etc.

[wolfv]

For dates there is this library which seems to be quite good and internationalized: https://github.com/bear/parsedatetime

[wolfv]

Over here: https://github.com/wolfv/adapt/tree/feature-numbers-dates

I have tried to implement a number tagger (special_tagger.py).

This kind of works, but I am currently not getting the number in the results. It seems to be pruned in the last step. Any hints would be appreciated! :)

I think a cool feature for adapt would also be to expose the original tags to the end user.

[clusterfudge]

@wolfv thanks, for looking into this. Datetimes as a first class citizen in adapt has been on my todo list for a while; I've been hung up on other mycroft stuff. I was planning on building it on top of parsedatetime, which would sadly require an n-gram expansion/search in the entity tagger that I'm not a huge fan of. Probably only do it if intent parsers require a datetime, and just make it clear that using it is potentially going to impact performance. I would gladly look at PRs in this space :smile:

As for your feature-numbers work, I'm really reluctant to pull anything into core adapt that's language specific (and yours is all english). The tokenizer is one area where I've made a concession, and it hasn't really been going well. We don't yet have a good way for someone to say "I want to use english adapt", even as a developer, and that kind of api and code organization is going to need to come into place before we start adding (for example) large amounts of vocabulary for recognizing numbers in multiple languages. I'm happy to take a look at your changes again if you have ideas on how to better organize code so it's language agnostic or namespaced, as well as the corresponding data.

[wolfv]

@clusterfudge

The parse methods in parsedatetime actually return the "remaining string" after stripping the date information. E.g. parsing "we will go out on the 8th november" will leave you with "we will go out on the"

My guess is that that could be used to either find out where the token start and endpoints are or just use the remaining stuff to analyze the intent.

Relevant part in the sourcecode: https://github.com/bear/parsedatetime/blob/master/parsedatetime/__init__.py#L1841

retS is the value

[clusterfudge]

well, that's certainly a different story. Does it handle multiple datetimes in a message, or recurrence? (think notify me every hour to take my extremely important medication).

[wolfv]

Not currently. And it has other limitations as well. Mainly not returning a confidence and returning slightly weird results in some cases.

E.g. with your example it returns the exact hour right now, as would be expected with "this hour").

But I think it's a difficult problem to solve generally! With a little more code it should be good at some point!

[clusterfudge]

Interesting; we currently compute confidence of an entity match based on edit distance (always 1.0 in mycroft land right now), and then utterance confidence based on percentage of utterance covered. Regex entities get a confidence of 0.5 (as opposed to 1.0). This date stuff is all regex based; we could probably pick a static confidence that works well based on experimentation.