cant set values for smart home devices

[royrogermcfreely]

i cant set spoken values for smart home devices.

for example i set up a sensor whitin the smart home window and teach some sentences to set the value with set sensor mediacontrol to value stop set media player spotify to value test

sepia tells me, thats not possible it thinks.

but when i set the smart home devices to "tv" i can set most of my values.

just words like "weiter", "nächstes lied", "leiser", "lauter" doesnt work since sepia trys the own mediacontrol.

i think before the last update it was working with other smart devices too.

my workarround meanwhile is to set all device that needs text values to "tv"

[Edit:] when i say "next song", sepia set the "tv mediacontrol" to value "next song" and home assistant tells spotify to play the next song. but i get the hole answer "tv mediacontrol im wohnzimmer auf wert next song, alles klaro"

can we adapt this to a single "OK" or something like this?

you mentiont here that we can adjust a file named StringsDE_remote_action_audio_stream but i couldnt find it. did you renamed it or is it named something else meanwhile?

/roy

[fquirin]

Hi /roy,

sorry for the late reply, was on vacation and took a little break ^^.

for example i set up a sensor whitin the smart home window and teach some sentences to set the value with set sensor mediacontrol to value stop set media player spotify to value test

These sentences are stored via the Teach-UI? If so they should always overrule everything else (unless in conflict with another Teach-UI command). They might work as well without Tech-UI if your devices are named "mediacontrol" and "spotify", but it is possible that there will be a conflict with other commands that are triggered by "spotify" for example. You can always use the "Assistant Testing" page in the Control-HUB to check out what is actually going on by investigating the results of "Interpret" or "Understand".

sepia tells me, thats not possible it thinks. ... but when i set the smart home devices to "tv" i can set most of my values.

"sensor" is a bit special because it only supports read and toggle actions. I think that restriction was introduced in the last update. "media player" should be fine though 🤔 .

just words like "weiter", "nächstes lied", "leiser", "lauter" doesnt work since sepia trys the own mediacontrol.

Yeah those commands will always apply to the active media player which can be the internal audio (e.g. used for radio), the embedded player (e.g. Youtube) or Android system player (e.g. via VLC intent). Custom smart-home devices that can play music will be unknown to SEPIA in this context (for now). I'd suggest to define special commands via the Teach-UI like "spotify next" or "remote next" or something :-).

can we adapt this to a single "OK" or something like this?

Commands defined via Teach-UI usually have a field Custom answer(s) or and says ... where you can define a simple "Ok". This answer should be triggered on success.

you mentiont that we can adjust a file named StringsDE_remote_action_audio_stream but i couldnt find it. did you renamed it or is it named something else meanwhile?

The file should be here ~/clexi/www/sepia/scripts/sepiaFW.local.js and controls all client localized messages and texts. The mentioned StringsDE.remote_action_audio_stream entry is the message you will hear when you send a remote action to the Client (e.g. music stream).

Hope that clarifies some things :-), Florian

[royrogermcfreely]

hey,

hope you enjoyed your break :)

Commands defined via Teach-UI usually have a field Custom answer(s) or and says ... where you can define a simple "Ok". This answer should be triggered on success.

i put a simple "OK" into "and say..." field. but then he anserwed me "OK, tv mediacontrol im wohnzimmer auf wert next song, alles klaro"

thanks for clearing up some things. maybe we can set values to more diffrent device types - now my vaccuumrobot is a "tv" 😄

[fquirin]

hope you enjoyed your break :)

I did, ty :-)

i put a simple "OK" into "and say..." field. but then he anserwed me "OK, tv mediacontrol im wohnzimmer auf wert next song, alles klaro"

This is an Execute command(s) command right? The custom answer works a little bit different there. It will be triggered in advance to the other commands that will be executed. I think we can optimize your custom command better by using control smarthome device. This will give you precise control over the parameters and the answer :-)

[royrogermcfreely]

yes i have all my commands as execute commands cause i can use <var1>

i tried it now with "control smarthome device" and i can set a specific value but i cant set spoken values right?

so i would have to make a command for example "netflix, kodi, nintendo, radio, etc." with "execute commands" i just have start <var1> and then tv radio auf wert radio<var1> and home assistant used this variable to start scripts or what else.

or do i miss something?

[fquirin]

That is unfortunately correct, you can only use variables in Execute command(s) at the moment. Maybe its time to introduce new variables for more flexible smart home commands 🤔. I've some plans to improve custom commands anyway and will keep that in mind :-).

now my vaccuumrobot is a "tv"

😅 can you show the the commands again plz, we should be able to "fix" that ^^. I'm assuming there is some priority issue with the names.

[royrogermcfreely]

sure, here is what i have:

when i say: sauge <var1> <var2> and execute command:tv staubsauger in der küche auf wert <var2>

so i can say diffrent commands like "sauge die küche" " sauge das schlafzimmer" "sauge den gang" with just one teach sentence :)

then the mqtt publish the <var2> to Home Assistant and start there a script for the positions to vacuum which i send to my robot.

when i set the robot from "tv" to "device" and use the same command with "gerät staubsauger in der küche...", i get a "this is not possible, i think"

still i can use this sentence to turn the device on and off with gerät staubsauger in der küche auf wert ein / aus

[fquirin]

Do you have the same issue when you enter the sentence directly (the one you've created a short-cut for I mean)?

There is a "assistant test" page in the Control-HUB that can give you a detailed list of what SEPIA has recognized when you enter it there and press "interpret". Could you post the output of the long sentence here please?

[fquirin]

I just realized that you probably answered my first question already ^^. Just to be sure when you said:

still i can use this sentence to turn the device on and off with gerät staubsauger in der küche auf wert ein / aus

do you mean entering this sentence directly works, but it does not work if you use it via the Execute command(s) short-cut?

[royrogermcfreely]

no both ways are working

[fquirin]

So this works: gerät staubsauger in der küche auf wert ein, but this does not: gerät staubsauger in der küche auf wert schlafzimmer

Did I get it right now? 😁

Could you enter both of these sentences in the mentioned Control-HUB page and use for example "understand" etc. to see if we can learn more:

[royrogermcfreely]

yes right. here are the logs for understand, ~interview~ and answer:

with gerät staubsauger in der küche auf wert ein i get:

understand:
{
    "result": "success",
    "best_direct_match": "---",
    "environment": "default",
    "mood": -1,
    "context": "smartdevice",
    "normalized_text": "geraet staubsauger in der kueche auf wert ein",
    "certainty": 1,
    "language": "de",
    "parameters": {
        "smart_device": {
            "value_local": "das Gerät",
            "found": "gerät staubsauger",
            "device_tag": "staubsauger",
            "value": "device"
        },
        "action": {
            "value_local": "anschalten",
            "value": ""
        },
        "general_value": {
            "value": "ein"
        },
        "smart_device_value": "",
        "room": {
            "value_local": "in der Küche",
            "room_tag": "küche",
            "value": "kitchen"
        }
    },
    "command": "smartdevice",
    "processTime": {
        "nlu": 20
    }
}

answer:
{
    "result": "success",
    "hasAction": false,
    "cardInfo": [],
    "answer": "staubsauger in der Küche auf ein, wird gemacht.",
    "more": {
        "certainty_lvl": 1,
        "cmd_summary": "smartdevice;;smart_device={"
        value_local ":"
        das Gerät ","
        found ":"
        gerät staubsauger ","
        device_tag ":"
        staubsauger ","
        value ":"
        device "};;action={"
        value_local ":"
        anschalten ","
        value ":"
        "};;general_value={"
        value ":"
        ein "};;smart_device_value=;;room={"
        value_local ":" in der Küche ","
        room_tag ":"
        küche ","
        value ":"
        kitchen "};;",
        "context": "smartdevice;;default",
        "language": "de",
        "user": "uid1007"
    },
    "hasCard": false,
    "actionInfo": [],
    "answer_clean": "staubsauger in der Küche auf ein, wird gemacht.",
    "htmlInfo": "",
    "hasInfo": false,
    "resultInfo": {
        "cmd": "smartdevice",
        "state": "ein",
        "device": "staubsauger",
        "room": "in der Küche"
    },
    "processTime": {
        "nlu": 18,
        "service": 5
    }
}

and with gerät staubsauger in der küche auf wert schlafzimmer i get:

understand:
{
    "result": "success",
    "best_direct_match": "---",
    "environment": "default",
    "mood": -1,
    "context": "smartdevice",
    "normalized_text": "geraet staubsauger in der kueche auf wert schlafzimmer",
    "certainty": 1,
    "language": "de",
    "parameters": {
        "smart_device": {
            "value_local": "das Gerät",
            "found": "gerät staubsauger",
            "device_tag": "staubsauger",
            "value": "device"
        },
        "action": "",
        "general_value": {
            "value": "schlafzimmer"
        },
        "smart_device_value": "",
        "room": {
            "value_local": "in der Küche",
            "room_tag": "küche",
            "value": "kitchen"
        }
    },
    "command": "smartdevice",
    "processTime": {
        "nlu": 16
    }
}

answer:
{
    "result": "success",
    "hasAction": false,
    "cardInfo": [],
    "answer": "staubsauger in der Küche steht auf ein.",
    "more": {
        "certainty_lvl": 1,
        "cmd_summary": "smartdevice;;smart_device={"
        value_local ":"
        das Gerät ","
        found ":"
        gerät staubsauger ","
        device_tag ":"
        staubsauger ","
        value ":"
        device "};;action=;;general_value={"
        value ":"
        schlafzimmer "};;smart_device_value=;;room={"
        value_local ":" in der Küche ","
        room_tag ":"
        küche ","
        value ":"
        kitchen "};;",
        "context": "smartdevice;;default",
        "language": "de",
        "user": "uid1007"
    },
    "hasCard": false,
    "actionInfo": [],
    "answer_clean": "staubsauger in der Küche steht auf ein.",
    "htmlInfo": "",
    "hasInfo": false,
    "resultInfo": {
        "cmd": "smartdevice",
        "state": "ein",
        "device": "staubsauger",
        "room": "in der Küche"
    },
    "processTime": {
        "nlu": 19,
        "service": 4
    }
}

[fquirin]

Thanks, I think I understand the problem now 🙂 or at least part of it. The action value is not recognized in the case of geraet staubsauger in der kueche auf wert schlafzimmer and probably defaults to "on". Just to make sure can you get the "understand" result for setze Gerät Staubsauger in der Küche auf Wert Schlafzimmer (the "setze" is the difference ^^).

When do you get "this is not possible, i think" as answer?

[royrogermcfreely]

with "setze Geräte..." my robot is finally no tv anymorre :smiley:

thanks