Trigger sentence and custom Parameters

[MtotheKay]

Hello,

I'm trying to write my first skill.

But, I can't seem to find out how you managed to get around the question for the reservation name in the restaurant reservation example. If I try something similar, with the room name for example, I always need to answer the defined question. Or to be more specific:

I'd like to reserve a table at Luigis for tomorrow at 6 p.m. for 2 persons on the name SEPIA.

Will set all parameters in the Restaurant Example. No further questions asked.

Hello Livingroom

In the code below will ask:

What is the name of the room?

And then accept anything. So it seems I'm missing something... or am I looking at the wrong place?

Could you nudge me in the right direction?

Thanks in advance

package net.b07z.sepia.sdk.services.uid1007;

import java.util.TreeSet;
import org.json.simple.JSONObject;
import net.b07z.sepia.server.assist.answers.ServiceAnswers;
import net.b07z.sepia.server.assist.assistant.LANGUAGES;
import net.b07z.sepia.server.assist.data.Parameter;
import net.b07z.sepia.server.assist.interpreters.NluResult;
import net.b07z.sepia.server.assist.interpreters.NluTools;
import net.b07z.sepia.server.assist.interviews.InterviewData;
import net.b07z.sepia.server.assist.parameters.CustomParameter;
import net.b07z.sepia.server.assist.services.ServiceBuilder;
import net.b07z.sepia.server.assist.services.ServiceInfo;
import net.b07z.sepia.server.assist.services.ServiceInterface;
import net.b07z.sepia.server.assist.services.ServiceResult;
import net.b07z.sepia.server.assist.services.ServiceInfo.Content;
import net.b07z.sepia.server.assist.services.ServiceInfo.Type;
import net.b07z.sepia.server.core.data.Language;
import net.b07z.sepia.server.core.tools.Debugger;
import net.b07z.sepia.server.core.tools.JSON;
import net.b07z.sepia.server.core.tools.Sdk;

public class HelloWorld_1Param implements ServiceInterface{

    //Command name
    private static final String CMD_NAME = "hello_world_1Param";                //Name tag of your service (will be combined with userId to be unique)
    //Answers used:
    private static final String successAnswer = ServiceAnswers.ANS_PREFIX + "hello_success_0a";
    private static final String okAnswer = "default_not_possible_0a";   //service ran properly but can't generate desired result (e.g. a search term was not found)
    private static final String failAnswer = "error_0a";                //fallback if service or some part of it crashed
    private static final String askRoomName = ServiceAnswers.ANS_PREFIX + "hello_ask_room_name_1a";

    //Define some sentences for testing:

    @Override
    public TreeSet<String> getSampleSentences(String lang){
        TreeSet<String> samples = new TreeSet<>();
        //GERMAN
        if (lang.equals(Language.DE.toValue())){
            samples.add("Hallo Wohnzimmer");

        //OTHER
        }else{
            samples.add("Hello livingroom");
        }
        return samples;
    }

    @Override
    public ServiceAnswers getAnswersPool(String language) {
        ServiceAnswers answerPool = new ServiceAnswers(language);

        //Build English answers
        if (language.equals(LANGUAGES.EN)){
            answerPool
                .addAnswer(successAnswer, 0, "<1> says: Hello.")
                .addAnswer(askRoomName, 0, "What is the name of the room?") 
                ;
            return answerPool;
        }else if (language.equals(LANGUAGES.DE)){
            answerPool
                .addAnswer(successAnswer, 0, "<1> sagt: Hallo.")
                .addAnswer(askRoomName, 0, "Wie lautet der Name des Raums?")    
            ;
            return answerPool;
        } else {
            return null;
        }
    }

    //Basic service setup:

    @Override
    public ServiceInfo getInfo(String language) {
        //Type of service (for descriptions, choose what you think fits best)
        ServiceInfo info = new ServiceInfo(Type.plain, Content.data, false);

        //Should be available publicly or only for the developer? Set this when you are done with testing and want to release
        //info.makePublic();

        //Command
        info.setIntendedCommand(Sdk.getMyCommandName(this, CMD_NAME));

        //Direct-match trigger sentences in different languages:
        String DE = Language.DE.toValue();
        info.addCustomTriggerSentence("Hallo Wohnzimmer", DE);

        String EN = Language.EN.toValue();
        info.addCustomTriggerSentence("Hello livingroom", EN);

        //Parameters:

        Parameter p1 = new Parameter(new RoomName())
                .setRequired(true)
                .setQuestion(askRoomName);

        info.addParameter(p1);

        //Answers (these are the default answers, you can trigger a custom answer at any point in the module with api.setCustomAnswer(..)):
        info.addSuccessAnswer(successAnswer)
            .addFailAnswer(failAnswer)
            .addOkayAnswer(okAnswer);

        info.addAnswerParameters("room_name");

        return info;
    }

    @Override
    public ServiceResult getResult(NluResult nluResult) {
        //initialize result
        ServiceBuilder api = new ServiceBuilder(nluResult, 
        getInfoFreshOrCache(nluResult.input, this.getClass().getCanonicalName()));

        //get required parameters
        Parameter roomNameParameter = nluResult.getRequiredParameter(RoomName.class.getName());
        String roomName = roomNameParameter.getValueAsString();

        api.resultInfoPut("room_name", roomName);

        //all good
        api.setStatusSuccess();

        //build the API_Result
        ServiceResult result = api.buildResult();
        return result;
    }

    /**
     * Parameter handler that tries to extract a room name.
     */
    public static class RoomName extends CustomParameter {

        @Override
        public String extract(String input) {
            String extracted = "";
            System.out.println("input: " + input);          
            //English
            if (this.language.equals(LANGUAGES.EN)){
                if (input.contains("Livingroom")) {
                    extracted = "livingroom";   
                }
                if (input.contains("Study")) {
                    extracted = "studyroom";    
                }
            //Other languages
            }else if (this.language.equals(LANGUAGES.DE)){
                if (input.contains("Wohnzimmer")) {
                    extracted = "livingroom";

                }
                if (input.contains("Arbeitszimmer")) {
                    extracted = "studyroom";

                }
            }

            Debugger.println("Extension: extacted from : " + input, 2);
            return extracted;
        }

        @Override
        public String build(String input){
            //anything extracted?
            if (input.isEmpty()){
                return "";
            }else{
                //build result with entry for field "VALUE"
                JSONObject itemResultJSON = JSON.make(InterviewData.VALUE, input);
                this.buildSuccess = true;
                return itemResultJSON.toJSONString();
            }
        }       
    }

}

[fquirin]

Ok lets see :-)

I just had a very quick look at the custom parameter and the issue might be that the "input" of the "extract" method will be normalized text that is lower-case and you are checking for "Livingroom" with upper-case. It is possible to access the raw text as well, I don't know the exact variable by heart right now but it should be something like this.nluInput.textRaw.

This is just a wild guess right now. I'll take a more detailed look later and copy the code over to my SDK to play around a bit. In the meantime you could also try the existing parameter ROOM (the link will point to its extraction method):

Parameter p1 = new Parameter(PARAMETERS.ROOM)
            .setRequired(true)
            .setQuestion("smartdevice_1a");

Another interesting code example could be the existing openHAB service.

Some rules for normalized text:

• all special characters are removed
• everything is lower-case
• ä, ö, ü, ß will become ae, oe, ue, ss

For example: "Schließfächer" would be "schliessfaecher" inside the extract method.

[MtotheKay]

Sadly it was not the upper case of "Livingroom" but I found got closer to the problem, I guess...

I did try the ROOM parameter, but I think that I'm still missing some crucial step. Since it seems that the initial values in the command are not honored. e.G. "Activate livingroom movie mode" will trigger the skill, but then asks for the room and the mode... Or if not required, simply does not set them...

I did add the updated code below.

Regarding the openHAB service: Most certainly I could just extend the openHAB service to get the result that I want (Setting a scene in a room via a scene text item). But I wanted to get something "easy" running before deep diving into the existing service... ;) But after that, the question will arise, about how to alter the existing services for testing. With the current set up (openHAB as a service in the main server) I would need to rebuild the whole assistant for testing. Am I right? Could It be an option to move the services more to the sdk side of things, so that deploying gets faster and maybe reduce the threshold for new developments?

package net.b07z.sepia.sdk.services.uid1007;

import java.util.TreeSet;
import org.json.simple.JSONObject;
import net.b07z.sepia.server.assist.answers.ServiceAnswers;
import net.b07z.sepia.server.assist.assistant.LANGUAGES;
import net.b07z.sepia.server.assist.data.Parameter;
import net.b07z.sepia.server.assist.interpreters.NluResult;
import net.b07z.sepia.server.assist.interpreters.NluTools;
import net.b07z.sepia.server.assist.interviews.InterviewData;
import net.b07z.sepia.server.assist.parameters.CustomParameter;
import net.b07z.sepia.server.assist.parameters.Room;
import net.b07z.sepia.server.assist.services.ServiceBuilder;
import net.b07z.sepia.server.assist.services.ServiceInfo;
import net.b07z.sepia.server.assist.services.ServiceInterface;
import net.b07z.sepia.server.assist.services.ServiceResult;
import net.b07z.sepia.server.assist.services.ServiceInfo.Content;
import net.b07z.sepia.server.assist.services.ServiceInfo.Type;
import net.b07z.sepia.server.core.assistant.PARAMETERS;
import net.b07z.sepia.server.core.data.Language;
import net.b07z.sepia.server.core.tools.Debugger;
import net.b07z.sepia.server.core.tools.JSON;
import net.b07z.sepia.server.core.tools.Sdk;

public class HelloWorld_1Param implements ServiceInterface{

    //Command name
    private static final String CMD_NAME = "hello_world_1Param";                //Name tag of your service (will be combined with userId to be unique)
    //Answers used:
    private static final String successAnswer = ServiceAnswers.ANS_PREFIX + "hello_success_0a";
    private static final String okAnswer = "default_not_possible_0a";   //service ran properly but can't generate desired result (e.g. a search term was not found)
    private static final String failAnswer = "error_0a";                //fallback if service or some part of it crashed
    private static final String askRoomMode = ServiceAnswers.ANS_PREFIX + "hello_ask_mood_name_1a";

    //Define some sentences for testing:

    @Override
    public TreeSet<String> getSampleSentences(String lang){
        TreeSet<String> samples = new TreeSet<>();
        //GERMAN
        if (lang.equals(Language.DE.toValue())){
            samples.add("Wohnzimmer Kino Modus aktivieren");

        //OTHER
        }else{
            samples.add("Activate livingroom movie mode");
        }
        return samples;
    }

    @Override
    public ServiceAnswers getAnswersPool(String language) {
        ServiceAnswers answerPool = new ServiceAnswers(language);

        //Build English answers
        if (language.equals(LANGUAGES.EN)){
            answerPool
                .addAnswer(successAnswer, 0, "Will set the <2> to the <1> mode.")
                .addAnswer(askRoomMode, 0, "What is the name of the mode?") 
                ;
            return answerPool;
        }else if (language.equals(LANGUAGES.DE)){
            answerPool
                .addAnswer(successAnswer, 0, "<1> modus wird <2> gesetzt")
                .addAnswer(askRoomMode, 0, "Wie lautet der name des Modus?")
            ;
            return answerPool;
        } else {
            return null;
        }
    }

    //Basic service setup:

    @Override
    public ServiceInfo getInfo(String language) {
        //Type of service (for descriptions, choose what you think fits best)
        ServiceInfo info = new ServiceInfo(Type.plain, Content.data, false);

        //Should be available publicly or only for the developer? Set this when you are done with testing and want to release
        //info.makePublic();

        //Command
        info.setIntendedCommand(Sdk.getMyCommandName(this, CMD_NAME));

        //Direct-match trigger sentences in different languages:
        String DE = Language.DE.toValue();
        info.addCustomTriggerSentence("Wohnzimmer Kino Modus aktivieren", DE);
        info.addCustomTriggerSentence("Arbeitszimmer Kino Modus aktivieren", DE);

        String EN = Language.EN.toValue();
        info.addCustomTriggerSentence("Activate livingroom movie mode", EN);

        //Parameters:
        Parameter p1 = new Parameter(new ModeName())
                .setRequired(true)
                .setQuestion(askRoomMode);
        info.addParameter(p1);

        Parameter p2 = new Parameter(PARAMETERS.ROOM, "");
        info.addParameter(p2);

        System.out.println("room infos in getinfo: " + p2.getValueAsString());

        //Answers (these are the default answers, you can trigger a custom answer at any point in the module with api.setCustomAnswer(..)):
        info.addSuccessAnswer(successAnswer)
            .addFailAnswer(failAnswer)
            .addOkayAnswer(okAnswer);

        info.addAnswerParameters("mode","room");

        return info;
    }

    @Override
    public ServiceResult getResult(NluResult nluResult) {
        //initialize result
        ServiceBuilder api = new ServiceBuilder(nluResult, 
        getInfoFreshOrCache(nluResult.input, this.getClass().getCanonicalName()));

        //get required parameters
        //-mode
        Parameter modeNameParameter = nluResult.getRequiredParameter(ModeName.class.getName());
        String modeName = modeNameParameter.getValueAsString();
        System.out.println("modename after get: " + modeName);
        //-room
        Parameter roomParameter = nluResult.getOptionalParameter(PARAMETERS.ROOM, "");
        String roomName = roomParameter.getValueAsString();

        api.resultInfoPut("room", Room.getLocal(roomName, api.language));
        api.resultInfoPut("mode", modeName);

        //all good
        api.setStatusSuccess();

        //build the API_Result
        ServiceResult result = api.buildResult();
        return result;
    }

    /**
     * Parameter handler that tries to extract a room name.
     */
    public static class ModeName extends CustomParameter {

        @Override
        public String extract(String input) {
            String extracted = "";
            System.out.println("input: " + input);          
            //English
            if (this.language.equals(LANGUAGES.EN)){
                if (input.contains("movie")) {
                    extracted = "movie";
                    System.out.println("Exctracted form EN: " + extracted);
                } else  if (input.contains("normal")) {
                    extracted = "normal";
                    System.out.println("Exctracted form EN: " + extracted);
                } else {
                    return "";
                }

            //Other languages
            }else if (this.language.equals(LANGUAGES.DE)){
                if (input.contains("kino")) {
                    extracted = "movie";
                    System.out.println("Exctracted form DE: " + extracted);
                } else if (input.contains("normal")) {
                    extracted = "normal";
                    System.out.println("Exctracted form DE: " + extracted);

                } else {
                    return "";
                }
            } else {
                return "";
            }

            Debugger.println("Extension: extacted: " + extracted + " from : " + input, 2);
            return extracted;
        }

        @Override
        public String build(String input){
            //anything extracted?
            if (input.isEmpty()){
                return "";
            }else{
                //build result with entry for field "VALUE"
                JSONObject itemResultJSON = JSON.make(InterviewData.VALUE, input);
                this.buildSuccess = true;
                return itemResultJSON.toJSONString();
            }
        }       
    }

}

[MtotheKay]

I attached the result from the test run.

And it's telling me that it missed the answer for ModeName. Maybe this helps finding an angle?

2019-09-19 17:13:35 [main] INFO AssistApi - {
    "cardInfo": [],
    "more": {
        "certainty_lvl": 1.0,
        "cmd_summary": "uid1007.hello_world_1Param;;net.b07z.sepia.sdk.services.uid1007.HelloWorld_1Param$ModeName=;;",
        "context": "uid1007.hello_world_1Param;;default",
        "language": "de",
        "user": "uid1007"
    },
    "hasCard": false,
    "answer_clean": "Wie lautet der name des Modus?",
    "response_type": "question",
    "hasInfo": false,
    "resultInfo": {
        "cmd": "uid1007.hello_world_1Param"
    },
    "result": "success",
    "hasAction": true,
    "input_miss": "net.b07z.sepia.sdk.services.uid1007.HelloWorld_1Param$ModeName",
    "answer": "Wie lautet der name des Modus?",
    "dialog_stage": 1,
    "actionInfo": [
        {
            "type": "show_abort"
        }
    ],
    "htmlInfo": "",
    "HTTP_REST_SUCCESS": true
}

[fquirin]

I found the problem. Something I forgot for a minute myself ^^. If you use addCustomTriggerSentence the dialog-module will never call the parameter extraction routine, because initially this was only ment to trigger the first step of a service. In your case a sentence like "Change room mode" would be an example. I rewrote the service to use the regular expression part instead, check this out:

@Override
public ServiceInfo getInfo(String language) {
    //Type of service (for descriptions, choose what you think fits best)
    ServiceInfo info = new ServiceInfo(Type.plain, Content.data, false);

    //Should be available publicly or only for the developer? Set this when you are done with testing and want to release
    //info.makePublic();

    //Command
    info.setIntendedCommand(Sdk.getMyCommandName(this, CMD_NAME));

    //Direct-match trigger sentences in different languages:
    String DE = Language.DE.toValue();
    //info.addCustomTriggerSentence("Wohnzimmer Kino Modus aktivieren", DE);
    //info.addCustomTriggerSentence("Arbeitszimmer Kino Modus aktivieren", DE);

    String EN = Language.EN.toValue();
    //info.addCustomTriggerSentence("Activate livingroom movie mode", EN);

    info.setCustomTriggerRegX("^("
                + "wohnzimmer kino modus aktivieren|"
                + "arbeitszimmer kino modus aktivieren"
            + ")$", DE);
    info.setCustomTriggerRegX("^("
                + "activate livingroom movie mode"
            + ")$", EN);
    info.setCustomTriggerRegXscoreBoost(1);     //boost service a bit to increase priority over similar ones

    //Parameters:
    Parameter p1 = new Parameter(new ModeName())
            .setRequired(true)
            .setQuestion(askRoomMode);
    info.addParameter(p1);

    Parameter p2 = new Parameter(PARAMETERS.ROOM, "");
    info.addParameter(p2);

    System.out.println("room infos in getinfo: " + p2.getValueAsString());

    //Answers (these are the default answers, you can trigger a custom answer at any point in the module with api.setCustomAnswer(..)):
    info.addSuccessAnswer(successAnswer)
        .addFailAnswer(failAnswer)
        .addOkayAnswer(okAnswer);

    info.addAnswerParameters("mode","room");

    return info;
}

I realize now that this is rather confusing and put it in the backlog for reconsideration ^^.

[MtotheKay]

Nice, that did it.

Just a quick follow up. How can I access the device information? I would like to retrieve the device name and make it the default room for any action.

--> When I have a stationary device in the livingroom, active Movie mode would use the device name as default value:

Parameter p2 = new Parameter(PARAMETERS.ROOM, deviceName);

[fquirin]

How can I access the device information? I would like to retrieve the device name and make it the default room for any action.

I assume you mean the client device ID by "device name"? Although this information is theoretically available during getInfo it is better to implement it in getResult. You could do it like this for example:

Parameter roomParameter = nluResult.getOptionalParameter(PARAMETERS.ROOM, nluResult.input.deviceId);
String roomName = (String) roomParameter.getDataFieldOrDefault(InterviewData.VALUE);

Btw I haven't answered you other questions yet ^^:

Regarding the openHAB service: Most certainly I could just extend the openHAB service to get the result that I want (Setting a scene in a room via a scene text item). But I wanted to get something "easy" running before deep diving into the existing service... ;)

Makes sense ^^.

With the current set up (openHAB as a service in the main server) I would need to rebuild the whole assistant for testing. Am I right? Could It be an option to move the services more to the sdk side of things, so that deploying gets faster and maybe reduce the threshold for new developments?

SDK services and 'system' services are mostly identical in how they are built so in theory the existing openHAB service could be copied over to the SDK with some minor adjustments to getInfo (the missing RegExp code for example is due to historical reasons still located in one of the keyword analyzers) and then 'boosted' a little bit to overrule the old one (info.setCustomTriggerRegXscoreBoost(1);). The only difference is that SDK services run in a sand-box if it's not disabled in the Assist-server settings. This is to prevent SDK services to mess around with the server. For the openHAB service this means if it's in the SDK it cannot access the server settings an thus cannot read the smart-home-HUB URL, but this could simply be hard-coded via the SDK.

[MtotheKay]

Great!! Works like a charm :) Although the device name is too shot some room names ("Wohnzimmer" gets "Wohnzim") so I have to map a shorter version to get the right name.

Last question, than I will close this thread:

You wrote:

The only difference is that SDK services run in a sand-box if it's not disabled in the Assist-server settings. This is to prevent SDK services to mess around with the server. For the openHAB service this means if it's in the SDK it cannot access the server settings an thus cannot read the smart-home-HUB URL, but this could simply be hard-coded via the SDK.

How do I disable the sand-box in the assist server? I could not find any setting in the Control Hub that did anything in that direction...

[fquirin]

Although the device name is too shot some room names ("Wohnzimmer" gets "Wohnzim") so I have to map a shorter version to get the right name

Usually I recommend device IDs with one letter and one number, e.g. "a1" (default for Android app), "b1" (default for browser), etc.. This is because of the BLE Beacon remote support. The Beacon uses a base URL + device ID to broadcast the trigger and due to limited block size I had difficulties to put in longer IDs.

How do I disable the sand-box in the assist server? I could not find any setting in the Control Hub that did anything in that direction...

I just double-checked and realized my mind was playing tricks on me O_o. What I had in mind was the use_sandbox_security_policy setting of the SEPIA Mesh-Node. Turns out that I never implemented it for the main server :-| . Do you need it for something specific? I guess I could implement a setting for the next version that gives at least access to the 'Config' class but 'System' and 'RunTime' would likely be more tricky in the current setup.

[fquirin]

How do I disable the sand-box in the assist server?

Update: I'm testing a new command-line option (currently in the dev-branch) to deactivate the security policy (complete access to runtime etc.) and to deactivate the sandbox (access to all Java classes). The flags will be --nosecuritypolicy and --nosandbox. I've decided to use command-line options because the sandbox is loaded before the settings file and because its a pretty serious setting that should not be exposed to the server itself.

[MtotheKay]

Nice. I will try it out. For now I will close the issue, cause it worked.