External sensors

[robross0606]

Most of the instructions and design focus on exposing ST devices to external systems. However, I would like to introduce external sensors (motion, open/close, etc.) into ST. It is unclear to me how I could create a device in SmartThings that equates to a device that originated in HomeAssistant. I've seen some discussion of using virtual switches for simple on/off switches, but how can this be done for sensors?

[sgupta999]

If you have device / sensors in Hass I believe u don’t even have to create a config - this is the legacy implementation that I continue to support.

So just make sure the device capability u are trying to import is there in the SmartApp capability map and make sure u have a virtual device to actually interface with it

Since I don’t have a HASSio environment I have not been able to test but perhaps you can.

So look at the config file and the big section that is all commented out. I believe HA published MQTT topic in that format preface/device/attribute/command and the server automatically configure that topic. ( however if you specific the same device in the config file - that will override this logic and use what u specify in config file)

[robross0606]

So, as an example, I have a motion sensor which is native to HA (not ST). I want that in ST. I already created a device in ST using "motion.capability" as the Type. Then I opened the MBS SmartApp on my mobile device and choose that newly created "virtual" device under the "Motion" section of the MBS SmartApp (which aligns to "motion.capability" in your CAPABILITY_MAP).

What I don't understand is how to get the devices aligned and mapped to MQTT topics. I must be missing it because I don't see where you set up the topic structure. I have the node.js service up and running but don't see any topics being published for native HA devices. I had assumed I needed to set up my device.cfg file on the node.js service.

BTW, I don't have a HASSio environment either. I'm running HA on a Windows machine right now.

[robross0606]

What's really bizarre is that I took a look at the logs on the node.js service and saw this:

2019-11-28 09:05:41 AM info: ===================================ACTUAL SUBSCRIPTIONS REQUESTED FROM SMARTAPP ============================================
2019-11-28 09:05:41 AM info: Currently subscribed to smartthings/KBR-Door/contact, smartthings/Electricity Meter/demand, smartthings/Electricity Meter/mqttmsg, smartthings/Contacts/notify, smartthings/System/notify, smartthings/LR-Recessed-M3/switch, smartthings/Patio-M2/switch, smartthings/LR-FanLight-M4/switch, smartthings/LR-Fan-M5/switch, smartthings/PoolMotor-M6/switch, smartthings/OutdoorLight-M7/switch, smartthings/SNF2/switch, smartthings/SNF3/switch, smartthings/Tasmota Test/switch, smartthings/KBR-Light-M8/switch, smartthings/MBR-FanLight-M9/switch, smartthings/a-wemos-ch1/switch, smartthings/a-wemos-ch2/switch, smartthings/GBR-Fan-M10/switch, smartthings/GBR-FanLight-M10/switch, smartthings/LR-Lamp1/switch, smartthings/LR-Lamp2/switch, smartthings/KBR-Heater/switch, smartthings/LR-Recessed-M3/update, smartthings/Patio-M2/update, smartthings/LR-FanLight-M4/update, smartthings/LR-Fan-M5/update, smartthings/PoolMotor-M6/update, smartthings/OutdoorLight-M7/update, smartthings/SNF2/update, smartthings/SNF3/update, smartthings/Tasmota Test/update, smartthings/KBR-Light-M8/update, smartthings/MBR-FanLight-M9/update, smartthings/a-wemos-ch1/update, smartthings/a-wemos-ch2/update, smartthings/GBR-Fan-M10/update, smartthings/GBR-FanLight-M10/update
2019-11-28 09:05:41 AM info: ============================================================================================================================

I took another look at the MBS SmartApp code and there are ZERO references to these devices. Just to be safe, not only did I disable the separate devices.cfg file but I physically removed it from the config folder. From where is it getting this information?

[robross0606]

Okay, figured that part out. It was using old data in the "state" folder. I had to delete those files to remove it. So, with that resolved, the file now shows that it is subscribing to zero things. I don't see any publish activity at all either so how do I get the node.js service to publish native HA devices?

[sgupta999]

Okay u can do it two ways

First Can u pls provide the topic format for subs and pubs that these native hass devices r publishing? Is it the same format as the commented out section under MQTT config in the config.yml? If yes u just need to uncomment those parts out, specify the right preface and should be all good.

2nd part I recommend this as more flexibility. In config.yml set deviceconfig flag to true.

Create a devices.yml file - follow the format as in the devices.example file - that is where u would specify the topics and commands

[sgupta999]

How did u install this ? Did u use docker image or NPM? I must have messed up somewhere in an upload - didn’t mean to upload my config files with log and state information - please make sure all directories under config are empty and only populated with your config.yml and devices.yml. I need to fix whatever source you downloaded these from - please let me know

[robross0606]

I installed via directly NPM. I think it was my fault. Your files come down with a devices.cfg file that contains your setup. I must've run once with that accidentally before removing it.

[robross0606]

One other question I have is that you say you need to enter a MAC address but don't give an example to derive format. MAC addresses can be represented many different ways, so I don't know how to format the hex values.

[robross0606]

Can u pls provide the topic format for subs and pubs that these native hass devices r publishing?

That's what I'm saying. NOTHING is being published to MQTT at all from HASS. All that I see in the log file for your service is this:

2019-11-28 09:14:19 AM info: Loading previous state
2019-11-28 09:14:19 AM info: request object - [object Object]
2019-11-28 09:14:19 AM info: ===================================ACTUAL SUBSCRIPTIONS REQUESTED FROM SMARTAPP ============================================
2019-11-28 09:14:19 AM info: Currently subscribed to 
2019-11-28 09:14:19 AM info: ============================================================================================================================
2019-11-28 09:14:19 AM info: Saving current state
2019-11-28 09:14:19 AM info: Connecting to MQTT at mqtt://xxx.xxx.xxx.xxx
2019-11-28 09:14:19 AM info: Configuring autosave
2019-11-28 09:14:19 AM info: Configuring API
2019-11-28 09:14:19 AM info: Listening at http://localhost:8099

I have logging turned all the way up to debug but I see nothing. The service never publishes a thing to my MQTT server automatically.

[robross0606]

Honestly, now that I look at this, I don't see how this bridge is related to HASS at all. The flow is between SmartThings and MQTT. There's nothing in the configuration of your service that even knows about HASS. You don't configure a connection address or port for HASS. Unless it assumes it is running on the same server as HASS, I'm not sure how your service would even know about native HASS devices.

[robross0606]

At this point, I've fallen back on just trying to make sure the typical flow is even working. I've added a bunch of native ST devices to the bridge configuration to see if I can get them to flow into your service and be published to MQTT. So far, nothing. Now I'm thinking this may be some type of networking issue. What is the proper expected format for the MAC address?

[sgupta999]

okay one more try: if you look at config file

Suffix for the topics that receive state from SmartThings $PREFACE/$DEVICE_NAME/$PROPERTY/$STATE_READ_SUFFIX
    # Your physical device or application should subscribe to this topic to get updated status from SmartThings
    # state_read_suffix: state

so assuming in Hass you subscribe to topic in format $DEVICE_NAME/$PROPERTY/state to get a state from device , server will do this automatically for you - it will get your event from hub and publish to this topic.

similarly for command topics

# Suffix for the command topics $PREFACE/$DEVICE_NAME/$PROPERTY/$COMMAND_SUFFIX
    # command_suffix: cmd

assuming hass send $DEVICE_NAME/$PROPERTY/cmd to mqtt broker, server will take that and trigger appropriate event is ST hub

again i do not have a hass implementation so if hass is not publishing to $DEVICE_NAME/$PROPERTY/cmd or subscribing to $DEVICE_NAME/$PROPERTY/state then go with the other option

[robross0606]

I'm not trying to deviate from the standard "smartthings" prefix or anything. I was right that my mac address structure wasn't correct. I had it with ":" between but changed to "-" and it started working at least for subscriptions. Nothing at all is publishing though. This application architecture isn't really designed to work from native HA devices to virtual ST devices. HA wouldn't know about the devices until something was published by ST. But nothing will ever be published by ST until a state changes. No state would change on a virtual device so the linkage never gets made. What I am trying to do right now instead is us an Automation in HA with the native mqtt.publish service to publish states from HA onto the same topic as expected by ST. But that doesn't appear to be working either.

[sgupta999]

MAC address should be aa:bb:cc:dd:ee:ff format looks like your ST is not sending subscription to the server so no communication between ST and server yet.

also looks like you are not using devices.yml - so delete state files, fix bridge param in IDE, go to smartapp config scree and save to resubscribe to everything again

[robross0606]

I do have subscriptions from ST to HA working now. It seems to be functioning for the typical documented use case. However, the reverse direction where state goes from HA to ST and cmd goes from ST to HA doesn't appear to be supported. It isn't a true bi-directional bridge.

[sgupta999]

it does work. Lets say you have a virtual switch. When you press on button in virtual switch - the hub will generate an event [device][attribute][command/payload].

smartapp will take that and pass it on to the server. server will publish [device/attribute/state][payload] to mqtt broker - so assuming hass is subscribed to device/attribute/state to receive events from ST and then turn device on.

similarly ST is subscribed to device/attribute/cmd and device/attribute/set_state assuming topics that hass publishes to

so true two way

[sgupta999]

if that is not your hass setup then just use the devices.yml and there you can set up any config you want

• multiple ST devices listening to one hass topic
• one ST device listening to multiple hass topics, or one hass topic
• multiple st devices publishing to one hass topic
• one st device publishing to one hass topic or multiple hass topics

so if base case doesn't work use this

[robross0606]

You just described what I said would not happen. You said I would press the button on the virtual switch to initiate the communications ST->HA. That's not what's happening here. There's no "button" to press, nor would I want to have to press it every time anything rebooted in order to set up the bridge communications. The devices are native to HA and all state changes come from HA, not ST. Because of this, the bridge never knows about the device to begin with because nothing is published to MQTT until a state changes. To resolve this, the MBA Bridge SmartApp should be "priming" MQTT with the initial state of all configured devices when the hub starts up. That way the communications would be set up. But, the way it works now, if you never "prime" the devices to HA by triggering state change FIRST in ST, nothing ever communicates.

[sgupta999]

one by one:

first you need the mac address to either have ':' or nothing, not sure the '-' would work (its an ST internal implementation no clue on that). mac address is needed to get communication from server to bridge(IDE) - so if that is not working look at that. so if you are getting events from the server to the device your current MAC is probably working. sending from bridge(IDE) to server - ip and port are sufficient

[sgupta999]

if you create a virtual switch - and use the default DTH you will have on / off switches in the app.

every time the server starts up it the mqtt broker publishes last retained message (state) to server which is sent to smartapp and IDE so without doing anything you should get the correct state in the virtual device

its upto you if you want to use virtual device to also control the physical device - but when you do click on the virtual device switch the command is published to the appropriate topic - i am assuming once hass receives a command topic from the mqtt broker then it initiates an action on the physical device.

if the physical device has a direct mqtt interface you can even choose to bypass hass if you want to

[sgupta999]

if you could please send me the topics you want ST to subscribe to and publish to I might get a better sense of what you are trying to accomplish.

[robross0606]

if you create a virtual switch - and use the default DTH you will have on / off switches in the app.

These are not virtual switches. They are motion and door sensors.

every time the server starts up it the mqtt broker publishes last retained message (state) to server which is sent to smartapp and IDE so without doing anything you should get the correct state in the virtual device

This is still ignoring the FIRST time. The way the system works now, there must be something that happens on the ST side to initiate the data flow. It isn't possible for HA to initiate the data flow. Also, your suggestion assumes the MQTT server is never restarted. If, for example, we were to have a power outage, I would have to go back into ST and "prime" all these virtual devices to get the data flowing with HA again.

[robross0606]

if you could please send me the topics you want ST to subscribe to and publish to I might get a better sense of what you are trying to accomplish.

I'm not sure why the specific topics matter. I don't really care what the topics end up being. I care about the data actually flowing. What I have is several devices that are native to HA:

• Six (6) simple contact sensors. Four (4) "door" and two (2) "window".
• Two simple (2) motion sensors.

I need state data to flow from HA to ST. That means I need to have similar devices set up in ST that are not real. They're not "simulated" either. They're "virtual" but I don't see any option for "virtual" motion or contact sensors -- just buttons and switches.

I don't need any command data to flow the other direction (ST -> HA) right now.

[sgupta999]

Please ignore last message

You are right for motion and contact sensor no data from ST to hass.

You forget that I save state and every 15 minutes I resubsribe everything again if not subscribed. So you wouldn’t have to reconnect these devices again.

I will test with MQTT broker being down for protracted time - not sure I have tested it rigorously but that’s the expected behavior

So for your use case you only need data to flow from hass to bridge to st

[sgupta999]

R u seeing any activity in logs to what topics ST is subscribing

[sgupta999]

ST does have simulated contact sensor, not sure but think they might have a simulated motion sensor too otherwise use contact sensor as motion sensor.

[sgupta999]

Topics matter because if you are not specifying a devices.yml then the server is making assumptions about what format the topic is in based on the config.yml setting and subscribing to those topics - so if hass is not publishing to those topics then u will not see them - but I think u said u were already getting subscription messages from hass to ST

[sgupta999]

Simulated contact sensor https://github.com/SmartThingsCommunity/SmartThingsPublic/blob/61b864535321a6f61cf5a77216f1e779bde68bd5/devicetypes/smartthings/testing/simulated-contact-sensor.src/simulated-contact-sensor.groovy

Simulated motion Sensor https://github.com/SmartThingsCommunity/SmartThingsPublic/blob/61b864535321a6f61cf5a77216f1e779bde68bd5/devicetypes/smartthings/testing/simulated-motion-sensor.src/simulated-motion-sensor.groovy

[rhamblen]

@robross0606 chuck MQTT-spy on a machine on the network. It should help double check what is being passed. You can use it to both listen and send MQTT packets.

and sandeep beat me to the sensors... there in the library, not the default loads.

[robross0606]

As far as I know, "Simulated" sensors are not the same as "Virtual" sensors. "Simulated" sensors are triggered by physical tile presses from the app. Hence the "tile" being defined in the code.

[robross0606]

@robross0606 chuck MQTT-spy on a machine on the network. It should help double check what is being passed. You can use it to both listen and send MQTT packets.

@rhamblen, I already have MQTT Spy and MQTT browser. That's how I know what's being passed and when.

[robross0606]

Topics matter because if you are not specifying a devices.yml then the server is making assumptions about what format the topic is in based on the config.yml setting and subscribing to those topics - so if hass is not publishing to those topics then u will not see them - but I think u said u were already getting subscription messages from hass to ST

No, I said I was getting subscription messages from ST to MQTT. That's the typical glide path. I can see published messages from ST to MQTT. There's nothing in the service that even talks directly to Hass. The only way I've gotten anything into or out of Hass via MQTT is by using the built-in abilities of Hass. The only thing your service does is bridge Smartthings to MQTT.

What I've done so far is:

• Make some devices with names in Smartthings, and add them to your bridge. I am then able to see those topics in MQTT Browser.
• Use Automations in Hass to trigger publishing of state data from Hass. I am then able to see those topics in MQTT Browser.

The next logical step was to set up "virtual" devices in ST, expose them via the bridge, and set up Hass to publish state to those same topic names. I did this, but your bridge does not appear to subscribe to state changes, bring them back into ST and update the virtual device's state. It is this last step that is blocking me right now.

[robross0606]

Here's a couple topic names Hass is currently publishing as examples:

• /smartthings/Kitchen Motion/motion
• /smartthings/Garage Inner Door/contact

The devices in ST are named:

• Kitchen Motion
• Garage Inner Door

I've already tried both of them using various "Type" to see if anything would work:

• Simulated Motion Sensor
• Motion Capability
• Motion Sensor
• Simulated Contact Sensor
• Contact Capability
• Door/Window Sensor (SG)

I found a "Virtual Contact Sensor Switch" so I'm trying that now on some of the contact sensors.

[robross0606]

I'm wondering if part of the problem could be semantics. For example, when Hass sets a motion state for this device, it is either "on" or "off". However, when ST virtual motion detector sets the same "motion" state, the value is either "active" or "inactive". Similarly, I see conflicts with contact sensors. Hass is still using "on" or "off" but ST is using "open" and "closed".

[robross0606]

Nope, that didn't fix it either. I changed my Hass automations to translate the values to what ST uses. So Hass now publishes "active" to the motion sensor at "/smartthings/Kitchen Motion/motion" when the motion sensor is triggered. However, the MQTT bridge completely ignores this value and even changes it back to "inactive" in MQTT shortly after Hass changed it. It would appear the bridge ignores these values altogether and doesn't push them back to ST. I'm guessing it is designed right now to only push "commands" up to ST and not "states".

[sgupta999]

absolutely that would be a problem. But before we get there Are your able to see any message in log like MQTT --> ST - Topic: [smartthings/Kitchen Motion/motion][off] [Kitchen Motion][motion][off]

That would mean that at least the bridge is getting the message from Hass and is subscribed to the right topic.

Then I would check in the IDE live logging to see if the bridge and the Smartapp are receiving the event [Kitchen Motion][motion][off] that would tell us that server to bridge is working fine and you bridge configuration is correct (mac address, port , ip)

Once those parts are working then there are multiple plays we can help translate the message lets first make sure the messages are getting both ways.

from a tile perspective both simulated and virtual are essentially same. motion sensors and contact sensor will have no actions associated with them file switches will. You should be okay using either .

Once you can confirm that two way communication is happening we will go to the next step

also not sure you need the first / in the topic before smartthings - i believe that could also cause an issue

[robross0606]

I think the problem may be that I didn't realize from the way your config.yaml file was commented that I needed to use the read_state and set_state suffixes on my topics. Trying that now.

[sgupta999]

again you only need those suffixes if thats what is the topic that Hass is publishing to.

you would need set_state and cmd here since you are not sending state from ST - but define it anyway if you might use switches in the future

[sgupta999]

using STATE_WRITE_SUFFIX (set_state) basically tells ST to issue a passive update state (do not fire an ST event) using COMMAND_SUFFIX (cmd) tells ST to update state and fire an ST event in case you are using webcore or other automation routines in ST to control other devices.

The suffix could be anything you want - here set_state and cmd

So in that case your Hass instance would publish to smartthings/Kitchen Motion/motion/set_state for passive state change and smartthings/Kitchen Motion/motion/cmd for state change with event

[robross0606]

Ahh... Okay. What's unclear from your commented config.yaml is if there are default values for these prefixes. Is "set_state" and "cmd" a default or do I need to uncomment those lines to specify?

[sgupta999]

you will need to uncoment those - nothing commented is readable

[robross0606]

I didn't know if the file was indicating that there was a default in play in the code itself. Not that the comment was "readable". Also, in your sample config.yaml you have "cmnd" specified for both the state_read_suffix and command_suffix which is quite confusing.

[sgupta999]

This the is the legacy approach. The alternative approach would be to use devices.yml set deviceConfig: true in config.yml

create a devices.yml file

---

Kitchen Motion:
  subscribe:
    motion:     
        smartthings/Kitchen Motion/motion:
          command:
            'off': 'inactive'
            'on': 'active'
  retain: 'false' 

Garage Inner Door:
  subscribe:
    contact:     
        smartthings/Garage Inner Door/contact:
          command:
            'off': 'closed'
            'on': 'open'
  retain: 'false' 

See here i translated your original on off commands from Hass to what ST expects but you can play with it any way you want.

No change in the IDE and you will be all set . this will by pass the legacy implementation and a lot more flexibility in what you want to do

[sgupta999]

if the suffix are the same every state change is treated as an event trigger

[robross0606]

okay, then I'm not sure why this wasn't working to begin with. If I had all the suffixes commented out then everything should have been an event trigger. But nothing published from Hass to MQTT was ever getting pushed back up to ST.

[sgupta999]

did you check the IDE live logging to see if you are getting any messages from the server? also on the server side did you get the logs in the format i specified above.

This is a legacy implementation from a previous author so i have not changed it at all in case anyone is still using it. Apparently you would still need to uncomment them and add a suffix because it is expecting a suffix before it will trigger an event (even if the suffixes are the same).

[sgupta999]

without the command suffix it will send a passive state change

[sgupta999]

so if you could confirm 2 things

1. server side logs anything like --
   MQTT --> ST - Topic: [smartthings/Kitchen Motion/motion][off] [Kitchen Motion][motion][off]

2. In the ide Live logging in bridge logs Parsed '{"command":true,"name":"Kitchen Motion","type":"motion","value":"off"}' in the smartapp logs Received device event from bridge: ["command":true,"name":"Kitchen Motion","type":"motion","value":"off"]

if you see 1 and not 2 then the bridge is not configured correctly

[robross0606]

I set up the suffixes, cleared out logs and restarted. I now see this in the server-side logs:

2019-11-29 11:12:27 AM debug: From MQTT: smartthings/Kitchen Motion/motion/trigger = active
2019-11-29 11:12:27 AM info: MQTT --> ST - Topic: [smartthings/Kitchen Motion/motion/trigger][active]   [Kitchen Motion][motion][active]

However, I'm not seeing anything in the live logs on ST IDE. So that's 1 and not 2. What could be configured incorrectly?

[robross0606]

However, I also set up some REAL ST devices which are ending up back in MQTT so I know the bridge is working in that direction:

2019-11-29 11:08:27 AM debug: From ST:  - temperature - 11
2019-11-29 11:08:27 AM debug: Device from SmartThings: smartthings/Kitchen Freezer/temperature/read = 11
2019-11-29 11:08:27 AM info: ST --> MQTT: [Kitchen Freezer][temperature][11]    [smartthings/Kitchen Freezer/temperature/read][11]

That's an actual ST device publishing temperature and humidity data, which I exposed via the bridge and is now publishing to MQTT. So how is the bridge incorrectly configured if that is working?