Add smoke detector

[sracing]

The proxy in a great way closes the painful gap of Gigaset not offering a API for its system. I´m using it for my integration into SmarthomeNG (along with KNX, Zigbee and MQTT).

The sensors all work, except the 'smoke' detector from Gigaset Elements does not connect. I see from the code that it is not yet considered.

Since I am not very familiar with programming, but could provide input on how the 'smoke' detector is managed in the base, would you be interested to add that function as a enhancement?

[ycardon]

Hi, I'll be glad to add this device to the project but I don't own one: I'll need your help.

Could you have a look at the gigaset-elements-proxy URL (http://myserverip:3000) and track the events that will relate to your smoke detector ? (http://myserverip:3000/api/v2/me/events)

Once you've got it, please send me the json fragment associated (I need the ev.type and the ev.o.type)

For instance for a door sensor, with the following event

{"id":"xxx","state":"intrusion","ts":"1554313948463","type":"close","o":{"frontendTags":{},"friendly_name":"entree","id":"yyy","type":"ds02"},"source_id":"zzz","source_name":"my base","source_type":"basestation","state_pre":"intrusion"},

they would be:

• ev.type = close
• ev.o.type = ds02

[sracing]

Perfect. I used Postman since it provides a formatted "pretty" view on the json, but I guess thats not relevant.

Before I come to the json details, some notes: 1) The parameters I was able to find are events.type and events.o.type, which is slightly different from what you asked for (but I guess you had the parameters of your code in your mind?) 2) The smoke sensors do not show up at all in the /api/v2/me/events listing unless they have been recently activated 3) The smoke sensor has at least 2 modes. "Test" for manual triggering the alarm (press the button on the front cover >1sec) and "Alarm" for whenver smoke is detected (I did activate that to track the json).

This is the reading of events.type and events.o.type for each of the operation modes (hope :

As you can see, in Test mode, the events.type remains the same, both during activating the Test mode and after ending it. That probably is a issue even if the Alarm mode has a unique identification.

However, the strange thing is, when I log my MQTT broker with journalctl -fu gigaset-server@pi, I do get the following output when I do the Test mode:

[LOG] acquired event: OG_Gallerie | sd01 | smoke_detected [LOG] event dropped: unhandled event type: sd01 [LOG] acquired event: OG_Gallerie | sd01 | end_sd01_test [LOG] event dropped: unhandled event type: sd01 [LOG] acquired event: OG_Gallerie | sd01 | smoke_no_longer_detected [LOG] event dropped: unhandled event type: sd01 [LOG] acquired event: OG_Gallerie | sd01 | end_sd01_smoke_detected [LOG] event dropped: unhandled event type: sd01

Remember, the logging above is purely from a Test-Mode. So it seems your actual code already detects more than I can see under /api/v2/me/events.

Is that a good starting point? Let me know in case you want me to analyze more (and sorry for the long text)

Regards Marcus

[ycardon]

Great ! It seems that we have all we need.

The event types values that the proxy is actually receiving are the ones you've seen in the log (which btw are not the MQTT event logs, but the proxy itself).

We can either return them as they are or filter some of them (for instance only keep smoke_detected and smoke_no_longer_detected), we can also do a mapping to new values.

For a first version, I will return them as they are. Then you can tell me what you prefer based on your tests.

[ycardon]

I've committed a v1.5 (ok, I just added two lines), can you tell me how it works ?

[sracing]

Thanks. 50% works.

On the 'Alarm' mode: That seems to works pretty good. [LOG] acquired event: OG_Gallerie | sd01 | smoke_detected [LOG] event sent as mqtt_topic: gigaset/OG_Gallerie, value: smoke_detected [LOG] acquired event: OG_Gallerie | sd01 | smoke_no_longer_detected [LOG] event sent as mqtt_topic: gigaset/OG_Gallerie, value: smoke_no_longer_detected [LOG] acquired event: OG_Gallerie | sd01 | end_sd01_smoke_detected [LOG] event sent as mqtt_topic: gigaset/OG_Gallerie, value: end_sd01_smoke_detected

On the 'Test' mode: Problem is that events.type does not change after ending the test (see my table). The system obviously detects the test end through acknowledgement in the app through a different json item:

Even after the achnowledgement in the app, the events.type maintains on the value 'test' and does not change. So for proper update in MQTT it seems the value of events.type "end_sd01_test" needs to be considered. The LOG already indicates this: [LOG] acquired event: undefined | undefined | end_sd01_test [LOG] event dropped: unhandled event type: undefined

Would be great to cover also the Test mode. Let me know in case you need more.

Thanks in advance!

[ycardon]

50% is not so bad, let's work on the 50 others :)

The logic inside the proxy is pretty simple : check the sensor type, then check the event type.

From your experiments, the test mode is using a "undefined" sensor type with an event type "end_sd01_test", we can use that event type and maybe add the sensor name as a topic (so that it behaves like the alarm mode).

I'm just puzzled about the changes in the logs, that was [LOG] acquired event: OG_Gallerie | sd01 | end_sd01_test and now is [LOG] acquired event: undefined | undefined | end_sd01_test Did you change anything ?

[sracing]

Hmmm, no, changed nothing (except the 2 lines in the code), but I am pretty sure I capture it correctly.

• Could the new code lead to a changed [LOG] behaviour?
• Any chance you change the logging to get more details?

[ycardon]

grumf, I hate when it's like that...

• Are the [LOG] acquired event: undefined | undefined | end_sd01_test logs are the only ones with undefined values ?
• Yes we could log more, but basically, it will end up with the same output as http://myserverip:3000/api/v2/me/events. Could you please check both the logs and the API from the previous URL ? Are they the same ?
• I did some 'more than 2 lines' changes in the version 1.5.1, what version are you using ?

[sracing]

I just copied the 2 lines from GitHub into the app.js of my raspberry.

Let me copy your entire code (v1.5.1) and redo the

• journalctl -fu gigaset-server@pi and the
• http://myserverip:3000/api/v2/me/events of the Test-Mode.

Will report asap.

[sracing]

Here it is (now with v. 1.5.1):

Activating Test [LOG] acquired event: OG_Gallerie | sd01 | test [LOG] event sent as mqtt_topic: gigaset/OG_Gallerie, value: test

After acknowledgement in the app: [LOG] acquired event: undefined | undefined | end_sd01_test [LOG] event dropped: Error: Configuration property "allow_unknown_events" is not defined

And the http://myserverip:3000/api/v2/me/events after acknowledgement in the app:

[sracing]

I forgot: Yes, I can confirm, the [LOG] acquired event: undefined | undefined | end_sd01_test logs are the only ones with undefined values.

[ycardon]

Ok, thank you for the different traces... I'll have a look look at them and let the night to inspire me :)

Btw, in v1.5.1, I've introduced a new configuration property (called allow_unknown_events), you should declare it in your configuration file (false is the current behaviour, true will forward any events to MQTT). With that will get rid of the Configuration property "allow_unknown_events" is not defined but not on the undefined type I'm afraid.

[ycardon]

Looking at the following section you've submitted:

It appears that the end_sd1_test event does not have a o.type section, so it's logged as unknown. If you want I can give a special treatments to these types of event, manually adding a 'normal` type.

[ycardon]

published a v1.5.2, hope it's better (and do not forget to update your config file).

[h4nc]

I also have a smoke detector and will test this soon.

So did I understand it correct, we have to set allow_unknown_events to true to make the smoke detector events show up correct?

[ycardon]

Hello @h4nc !

allow_unknown_events can be set either to true of false, it won't change so much now, but it has to be set (default is false).

You would set it to true for testing purpose, if a new type of sensor is released or if you want to add extra behaviour inside home-assistant (for instance).

any 'unknown' (ie. not implemented in this function) event will be sent to MQTT in the following way:

• topic = gigaset/<event.o.friendly_name> note that could lead to gigaset/undefined
• value = <event.type>

[ycardon]

@sracing and @h4nc, if you want we can together define the best treatment logic we want to apply to the events. I've rearrange the event mapper function to make it clearer.

// tell what mqtt topic and value a gigaset event should return
// you can change this section according to your needs, throw an exception to drop the event
function gigasetEventMapper(event) {
    let topic = MQTT_TOPIC + event.o.friendly_name

    // basestation events (based on event type)
    switch (event.type) {

        case 'isl01.bs01.intrusion_mode_loaded': // changed security mode
            return [topic, event.o.modeAfter]

        case 'end_sd1_test': // smoke detectors test session acknowledged
            return [MQTT_TOPIC + event.o.basestationFriendlyName, event.type]
    }

    // sensor events (based on sensor type)
    switch (event.o.type) {

        case 'ds02': // door sensors
        case 'ws02': // windows sensors
            if (event.type == 'close') return [topic, 'false']
            else return [topic, 'true']

        case 'ps02': // motion sensor
        case 'ycam': // motion from camera
            return [topic, 'true']

        case 'sp01': // intrusion detected (or acknowledged), siren must turn on (or turn off)
            if (event.type == 'on') return [topic, 'true']
            else return [topic, 'false']

        case 'sd01': // smoke detectors
            return [topic, event.type]

        default: // other events will be dropped (unless stated in the config)
            if (config('allow_unknown_events')) return [topic, event.type]
            else throw 'unhandled event type: ' + event.o.type 
    }
}

There is a first section that deals with 'special' basestation events (the first switch based on the event type) then a second section (the second switch based on the sensor type). Each case returns a [topic, value] pair that will be pushed to MQTT. We can make any test we want and return any pair we want.

Do you want to play with me and design the logic of your dreams?

[sracing]

Sounds great!

Would be more a universal bridge for any event type, if I followed your explanation correctly.

I´m in the boat, let me know how I can support

[ycardon]

Well the game is pretty simple:

• check the events that are fired by gigaset on your installation (using http://myserverip:3000/api/v2/me/events)
• discover the ones you want to track in the proxy
• identify a condition to be implemented in the code to recognise them (for instance event.type is isl01.bs01.intrusion_mode_loaded)
• indicate how these events are to be output in MQTT (for instance topic : event.o.friendly_name, value : event.o.modeAfter, this is the work we've done with @h4nc on the alarm mode for the base station)
• optionally, try to implement it directly on the above function (we can also do it together)
• test and enjoy ;)

[sracing]

Let me do some testing.

I guess the additional description of the acquired event is intentionally - even if the "[Date, Time] [LOG]" is missing for these lines?

[sracing]

Did not yet change anything on the code. Just preliminary info after acknowledgement of the Test-Mode, the 'friendly_name' and the 'event.o.type' are classified as 'undefined'.

Let me see if I understand how to change the code to get the right setting.

[sracing]

OK, lets sort this out.

1 - As far as I understand it, there´s a small typo in the code. Line 85 should be if (event.type == 'end_sd01_test') return [topic, event.type] not if (event.type == 'end_sd1_test') return [topic, event.type]

2- When test end has been acknowledged, the issue on

[LOG] acquired event: undefined | undefined | end_sd01_test

exists since the item which indicates the test acknowledgment has no events.o.friendly_name and no events.o.type but a events.o.basestationFriendlyName and a events.o.basestationType

Looks like the code needs some add. filters?

[sracing]

OK, with the code adaption for the basestation events in your earlier post there is still a undefined output after acknowledgement (see 1st yellow arrow), but at the same time the log shows a defined output after acknowledgement of the Test-Mode (2nd yellow arrow)

[h4nc]

I‘m afraid I cannot test to much here, as my wife won’t accept the smoke alarm sirene too often ;)

However I will start to test it when it comes near „final“, to confirm that it works for other users too.

[ycardon]

@sracing,

1 - As far as I understand it, there´s a small typo in the code.

Right ! thank you for spotting this

2- When test end has been acknowledged, the issue on [LOG] acquired event: undefined | undefined | end_sd01_test

Ah yes, the log (line 104) is always showing the same information (and undefined if the property does not exist in the JSON message). This is why I've added the full message also in the log (line 105). Maybe I should remove the first one as It's misleading.

log shows a defined output after acknowledgement of the Test-Mode (2nd yellow arrow)

Cool, it's working \o/

I will commit this first version (but we can continue to play :)

[ycardon]

It's commited now.

It seems that I getting some 'low sensor battery' events, I will try to capture them.

[sracing]

The "Test / Acknowledged" status has a bit of a ugly situation that the smoke sensor status is not clearly defined unless the smarthome system combines it with the MQTT event of the base status.

Feels like this is getting more complex, so in order to take a step back and have a more holistic view I generated a overview (not completely edited, I focused on the smoke sensor):

Do you see any chance to update the MQTT event of the smoke sensor itself, in order to give it a state "default; test; alarm" ?

[h4nc]

It doesn’t matter in this discussion, but I think there is a typo in the plug section in you table. sp01 and sp02. Are there really two different event.o.type for that.

[ycardon]

Woaaa @sracing, great analysis !

I'd say that, apart from line 18 where no events are fired, we have enough information to assign a "default; test; alarm" state. One question is: do we use the sensor friendly name as the topic (in that case we have a problem for line 19) or do we use a generic name (gigaset/smoke_detector for instance) for every smoke detectors events ?

Just one thought: (apart from the game itself) the really important state to track is the alarm one.

I will propose you an implementation based on your chart, late tomorrow.

[ycardon]

@sracing, with your permission I will also compile your chart in the readme file in order to explain what are the event the proxy is tracking and how it will transform it to MQTT (topic and value).

[sracing]

The sensor friendly name would be preferred, since in case where there are more than smoke detectors (like in my case, I do have 3 smoke detectors connected) this allows a clear association on the status of the detector.

By the way, that might be a additional topic. For more than one smoke detectors, the status back from "test" to "default" needs to have the right sensor friendly name. Guess you need to work with persistant variables or something like that (Sorry, I´m a real software novice, guess you know the best way to manage that in the code)

For the table: Sure you may use it in the readme file. I could also send the excel file to you in case there is something like a private message on github.

[ycardon]

Yes I understand your point: in fact the « ack by the user event » does not have associated sensor because it should reset all the sensors.

I’ve got an idea: we didn’t make use of the basestation status (:3000/api/v1/me/basestations), maybe we can read the actual state of the sensors instead of deducting it from the event.

[sracing]

Alternatively to ýour /api/v1/me/basestations idea: Since the Alarm Mode can be detected and it is only the Test-Mode that has issues, you could also handle the Test-Mode like the Movements, resetting the Testmode to default after a certain time. The Test-Mode is not expected to be safety critical and typically will only be activated for a few seconds.

By the way, I will be on business travel next week, so my apologies for not being able to support Mo. through Fr. Maybe @h4nc can take over testing.

[ycardon]

Yes, this would work (assuming you don't have a real alarm during that certain time).

Your idea is a little more simple to implement, I will go this path for the moment.

[h4nc]

@ycardon let me know if I should test something. But as I said I will not be able to test like 5 times in a row. 😉

[ycardon]

I've just committed, don't forget to set the new off_event_delay_after_smoke_detector_test configuration variable.

@h4nc, It should be ok but a bug is always possible (I'm coding blindly as I don't have these sensors). Don't put yourself any pressure on testing, it should stay a hobby :P

[sracing]

Hmmm, some [Error] after activating the Test-Mode:

[LOG] acquired event: {"id":"4a86ba57587f677993f731107fbf5287f878dfe9","state":"ok","ts":"1554652726610","type":"test","o":{"frontendTags":{},"friendly_name":"OG_Gallerie","id":"02ea3047cc","type":"sd01"},"source_id":"DF74354CC4106BEC052DC4AE6F8C67F9","source_name":"Schulte_Zuhause","source_type":"basestation","state_pre":"ok"} [LOG] event sent as mqtt_topic: gigaset/OG_Gallerie, value: test [ERROR] check events | unexpected gigaset response:{"events":[{"id":"4a86ba57587f677993f731107fbf5287f878dfe9","state":"ok","ts":"1554652726610","type":"test","o":{"frontendTags":{},"friendly_name":"OG_Gallerie","id":"02ea3047cc","type":"sd01"},"source_id":"DF74354CC4106BEC052DC4AE6F8C67F9","source_name":"Schulte_Zuhause","source_type":"basestation","state_pre":"ok"}],"home_state":"ok"} [INFO] authorized on gigaset cloud api [LOG] acquired event {"id":"daea96abb7c80915112b226a106da5b5409710e0","state":"ok","ts":"1554652730596","type":"movement","o":{"frontendTags":{},"friendly_name":"EG_Flur","id":"025bd2f415","type":"ps02"},"source_id":"DF74354CC4106BEC052DC4AE6F8C67F9","source_name":"Schulte_Zuhause","source_type":"basestation","state_pre":"ok"} [LOG] event sent as mqtt_topic: gigaset/EG_Flur, value: true [LOG] delayed event sent as mqtt_topic: gigaset/EG_Essbereich, value: false [LOG] delayed event sent as mqtt_topic: gigaset/EG_Flur, value: false

[sracing]

And this is the Log after acknowledgement in the app, with some'undefined' event:

[LOG] acquired event: {"id":"7c0bf29421a9bed614fa6fc0e41d020f1287f148","state":"ok","ts":"1554653794906","type":"end_sd01_test","o":{"basestationId":"DF74354CC4106BEC052DC4AE6F8C67F9","reason":"acknowledged_by_user","basestationFriendlyName":"Schulte_Zuhause","basestationType":"bs01"},"source_id":"app-mids-devices-management@dkrh4.reef","source_type":"iml01","state_pre":"ok"} [LOG] event sent as mqtt_topic: gigaset/undefined, value: end_sd01_test [LOG] acquired event: {"id":"4d4dee9e710136b6fd31802e7418fc42d124fb82","state":"ok","ts":"1554653797488","type":"off","o":{"consumed":"0","reason":"cmd","frontendTags":{},"friendly_name":"Alarm","delay":"0","id":"02d52460f8","type":"sp01"},"source_id":"DF74354CC4106BEC052DC4AE6F8C67F9","source_name":"Schulte_Zuhause","source_type":"basestation","state_pre":"ok"}

[ycardon]

arglll.... 1/ I've added more logs for the error (you did add the new config variable right ;) 2/ I've explicitly ignored end_sd01_test as they are treaded by the delayed event

[sracing]

Yes, default.yaml was updated.

This is the log with the extended error reporting (yellow line represents acknowledgement):

[ycardon]

Ok, nailed it ! Silly mistake... It's committed Thank you for your patience :)

[sracing]

Looks good! Well done! Great extension of the Bridge.

Back on Friday.

[ycardon]

great ! I'm closing the issue.

[h4nc]

@ycardon Ok so I testet this new feature. I did not yet fire an alarm but I pushed the test button and I got the mqtt event.

Yet there is one thing I don't like a lot. A smoke detector will (hopefully) be default most of the time. The thing is when I restart home assistant the state of the smoke detector mqtt sensor in ha will go to "unknown". And because I won't push the test button on the detector after every restart the state of this sensot will be "unknown" most of the time. Doesn't look that good.

So it would be nice to add the state to the "force refresh" service. Maybe also for the battery states.

Thanks again for all of this.

[ycardon]

Sounds fair, what status do you see on ‘/basestation’ ?

[h4nc]

Status = online

Also seems like there is another issue. The smoke alarm (testet it in the meanwhile) triggered my intruder automation in ha. But I have to dig in a little bit before I get more concrete. Could be an issue of my ha config. Else I will have to bother you again... sry

Edit:

Ok so I looked into my config. My Intruder automation gets activated because the sirene is active for the smoke alarm too.

So now I cannot use the siren mqtt trigger anymore, because I want to trigger different automations (intruder or Fire)

Is there a way to make two different mqtt events for those alarms. If not I could look for state with HomeAssistant (restful). But as we discussed before this needs a lot of polling, because it needs to look for a new state every few seconds to make it work reliable.

[h4nc]

@ycardon What can we do about the Smoke detector being unknown all the time?

[ycardon]

Oh yes you are right, I missed that point :(

Can you post an extract of the /basestations URL showing smoke detectors ? (It seems that they don't have a position_status attribute like other sensors, this may be why they do not appear as unknown).

[h4nc]

Sure, but currently I'm not at home. Maybe @sracing can help in the meantime.

[h4nc]

this is from /basestations

            {
                "battery": {
                    "state": "ok"
                },
                "firmware_status": "up_to_date",
                "friendly_name": "Smokedetector",
                "fw_version": "00***00",
                "id": "0*****",
                "latest_version": "00***00",
                "status": "online",
                "ts_button": 1*****,
                "type": "sd01"
            },