HomeSmartMesh / smart_home_3d_webapp

interactive, 3d, video game like, overview and control of your home from a web app
MIT License
85 stars 11 forks source link

Testing Mqtt with ESP8266 - MQTT Payload format #8

Closed enrutador closed 4 years ago

enrutador commented 4 years ago

Hi I am testing the functionality of the MQTT, I only have nodemcu to be a client that sends the temperature and humidity to the mosquitto server to simulate the "lzig / kitchen heat" attribute of blender and see how the color changes.

The data from the mcu reaches to mosquito well:

1591195187: Sending PUBLISH to 10.0.0.253#3d_webapp (d0, q0, r0, m0, 'lzig/bedroom heat', ... (4 bytes)) 1591195190: Received PUBLISH from nodeMcuDHT11 (d0, q0, r1, m0, 'lzig/kitchen heat', ... (5 bytes)) 1591195190: Sending PUBLISH to 10.0.0.253#3d_webapp (d0, q0, r0, m0, 'lzig/kitchen heat', ... (5 bytes)) 1591195190: Received PUBLISH from nodeMcuDHT11 (d0, q0, r1, m0, 'lzig/bedroom heat', ... (5 bytes))

3d_webapp subscription also subscribe well

1591195241: Received SUBSCRIBE from 10.0.0.253#3d_webapp 1591195241: tele/tasmota_1C2880/SENSOR heat (QoS 0) 1591195241: 10.0.0.253#3d_webapp 0 tele/tasmota_1C2880/SENSOR heat 1591195241: Sending SUBACK to 10.0.0.253#3d_webapp 1591195241: Received SUBSCRIBE from 10.0.0.253#3d_webapp 1591195241: lzig/bedroom heat (QoS 0) 1591195241: Sending PUBLISH to 10.0.0.253#3d_webapp (d0, q0, r1, m0, 'lzig/bedroom heat', ... (5 bytes)) 1591195241: 10.0.0.253#3d_webapp 0 lzig/bedroom heat 1591195241: Sending SUBACK to 10.0.0.253#3d_webapp 1591195241: Received SUBSCRIBE from 10.0.0.253#3d_webapp 1591195241: lzig/kitchen heat (QoS 0) 1591195241: Sending PUBLISH to 10.0.0.253#3d_webapp (d0, q0, r1, m0, 'lzig/kitchen heat', ... (5 bytes)) 1591195241: 10.0.0.253#3d_webapp 0 lzig/kitchen heat 1591195241: Sending SUBACK to 10.0.0.253#3d_webapp 1591195241: Received SUBSCRIBE from 10.0.0.253#3d_webapp 1591195241: lzig/living heat (QoS 0) 1591195241: 10.0.0.253#3d_webapp 0 lzig/living heat 1591195241: Sending SUBACK to 10.0.0.253#3d_webapp 1591195241: Received SUBSCRIBE from 10.0.0.253#3d_webapp 1591195241: tele/tasmota_1C2880/STATE heat (QoS 0)

The data reaches the web, but the color sets it to 000000

mqtt_app> lzig/bedroom heat => 16.18 home_app.js:188 home_app> mqtt heater : Bedroom_Heater ratio at NaN three_app.js:555 update_color_ratio undefined objeto Bedroom_Heater three_app.js:556 Bedroom_Heater color update three_app.js:568 dentro cambio color :setting Bedroom_Heater to color 000000

Code test of Arduino:

`

include

include

include

define DHTPIN D4 // what pin we're connected to

define wifi_ssid "SSID"

define wifi_password "password"

define mqtt_server "10.0.0.253" // MQTT Cloud address

define humidity_topic "lzig/bedroom heat"

define temperature_topic "lzig/kitchen heat"

define default1 "lzig/kitchen temp:"

define DHTTYPE DHT11 // DHT 11

WiFiClient espClient; PubSubClient client(espClient); DHT dht(DHTPIN, DHTTYPE); void setup() { Serial.begin(115200); setup_wifi(); client.setServer(mqtt_server, 1883); dht.begin(); } void setup_wifi() { delay(10); WiFi.begin(wifi_ssid, wifi_password); while (WiFi.status() != WL_CONNECTED) { delay(500); Serial.print("."); } } void reconnect() { // Loop until we're reconnected while (!client.connected()) { Serial.print("Attempting MQTT connection..."); if (client.connect("nodeMcuDHT11")) { Serial.println("connected"); } else { Serial.print("failed, rc="); Serial.print(client.state()); Serial.println(" try again in 5 seconds"); delay(5000); } } }

bool checkBound(float newValue, float prevValue, float maxDiff) { return newValue < prevValue - maxDiff || newValue > prevValue + maxDiff; } long lastMsg = 0; float temp = 0.0; float hum = 0.0; float diff = 1.0; void loop() { if (!client.connected()) { reconnect(); } client.loop(); long now = millis(); if (now - lastMsg > 3000) { // Wait a few seconds between measurements lastMsg = now; float newTemp = random(1, 4000) / 100.0 ; //dht.readTemperature(); float newHum = random(1, 4000) / 100.0 ; //dht.readHumidity();

    if (checkBound(newTemp, temp, diff)) {
        temp = newTemp;
        Serial.print("New temperature:");
        Serial.println(String(temp).c_str());
        client.publish(temperature_topic, String(temp).c_str(), true);
    }
    if (checkBound(newHum, hum, diff)) {
        hum = newHum;
        Serial.print("New humidity:");
        Serial.println(String(hum).c_str());
        client.publish(humidity_topic, String(hum).c_str(), true);
    }

}

} `

Am I sending the data format wrong?

A lot of thanks

wassfila commented 4 years ago

Hi @enrutador , first I try here to format your log so that I can understand it. To debug this, I do not thik that you should start with the complete system so I would leave the arduino out of the game for the moment and debug with Mqtt only, with command line it's possible to inject any sort of message in any topic, and check if the 3d app react as expected, only then I would check the arduino and monitor the topic, to tell who's falty, the 3d app or arduino.

If you can make such a test and focus the issue on one or the other, that could narrow it down and help solve it faster, in the meanwhile, I'll have a look at the look and try to guess what goes wrong.

I've noticed some comments coming from different line numbers, did you bring singnificant changes to the code ? I'll go with the assumption that it's very similar to the latest master of this repo.

1591195187: Sending PUBLISH to 10.0.0.253#3d_webapp (d0, q0, r0, m0, 'lzig/bedroom heat', ... (4 bytes)) 
1591195190: Received PUBLISH from nodeMcuDHT11 (d0, q0, r1, m0, 'lzig/kitchen heat', ... (5 bytes)) 
1591195190: Sending PUBLISH to 10.0.0.253#3d_webapp (d0, q0, r0, m0, 'lzig/kitchen heat', ... (5 bytes)) 
1591195190: Received PUBLISH from nodeMcuDHT11 (d0, q0, r1, m0, 'lzig/bedroom heat', ... (5 bytes))
1591195241: Received SUBSCRIBE from 10.0.0.253#3d_webapp 
1591195241: tele/tasmota_1C2880/SENSOR heat (QoS 0) 
1591195241: 10.0.0.253#3d_webapp 0 tele/tasmota_1C2880/SENSOR heat 
1591195241: Sending SUBACK to 10.0.0.253#3d_webapp 
1591195241: Received SUBSCRIBE from 10.0.0.253#3d_webapp 
1591195241: lzig/bedroom heat (QoS 0) 
1591195241: Sending PUBLISH to 10.0.0.253#3d_webapp (d0, q0, r1, m0, 'lzig/bedroom heat', ... (5 bytes)) 
1591195241: 10.0.0.253#3d_webapp 0 lzig/bedroom heat 
1591195241: Sending SUBACK to 10.0.0.253#3d_webapp 
1591195241: Received SUBSCRIBE from 10.0.0.253#3d_webapp 
1591195241: lzig/kitchen heat (QoS 0) 
1591195241: Sending PUBLISH to 10.0.0.253#3d_webapp (d0, q0, r1, m0, 'lzig/kitchen heat', ... (5 bytes)) 
1591195241: 10.0.0.253#3d_webapp 0 lzig/kitchen heat 
1591195241: Sending SUBACK to 10.0.0.253#3d_webapp 
1591195241: Received SUBSCRIBE from 10.0.0.253#3d_webapp 
1591195241: lzig/living heat (QoS 0) 
1591195241: 10.0.0.253#3d_webapp 0 lzig/living heat 
1591195241: Sending SUBACK to 10.0.0.253#3d_webapp 
1591195241: Received SUBSCRIBE from 10.0.0.253#3d_webapp
1591195241: tele/tasmota_1C2880/STATE heat (QoS 0)
mqtt_app> lzig/bedroom heat => 16.18 home_app.js:188
home_app> mqtt heater : Bedroom_Heater ratio at NaN three_app.js:555 
update_color_ratio undefined objeto Bedroom_Heater three_app.js:556 
Bedroom_Heater color update three_app.js:568 
dentro cambio color :setting Bedroom_Heater to color 000000
wassfila commented 4 years ago

in order to help also, here's the log I get from a running 3d app, I just tested it right now and the color do correctly update. Below is the log from Google chrome

three_app.js:379 set_stats_view() to true
hue_app.js:35 hue_app> init() hue registred
hue_app.js:174 using username from browser's local storage
three_app.js:371 three_app> model loading 5 %
three_app.js:371 three_app> model loading 40 %
hue_app.js:126 hue_app> get_lights(), hue available, all good, 3d model will be hidden
/favicon.ico:1 Failed to load resource: the server responded with a status of 404 (Not Found)
three_app.js:371 three_app> model loading 65 %
three_app.js:371 three_app> model loading 100 %
three_app.js:79 three_app> create_camera() '' with fov: 45
three_mouse.js:45 three_mouse> init()
three_control.js:46 three_control> init()

ning, all good, 3d model will be hidden
mqtt_app.js:54 mqtt_app> - subscribed to nrf/Balcony tag
mqtt_app.js:54 mqtt_app> - subscribed to nrf/Bathroom tag
mqtt_app.js:54 mqtt_app> - subscribed to nrf/Bedroom tag
mqtt_app.js:54 mqtt_app> - subscribed to nrf/Hallway tag
mqtt_app.js:54 mqtt_app> - subscribed to nrf/Kitchen tag
mqtt_app.js:54 mqtt_app> - subscribed to nrf/Livingroom window tag
mqtt_app.js:54 mqtt_app> - subscribed to nrf/Livingroom tag
mqtt_app.js:54 mqtt_app> - subscribed to nrf/Office tag
mqtt_app.js:54 mqtt_app> - subscribed to lzig/kitchen heat
mqtt_app.js:54 mqtt_app> - subscribed to lzig/living heat
mqtt_app.js:54 mqtt_app> - subscribed to lzig/bathroom heat
mqtt_app.js:54 mqtt_app> - subscribed to lzig/bedroom heat
mqtt_app.js:54 mqtt_app> - subscribed to lzig/office heat
home_app.js:177 home_app> mqtt temperature : Bathroom_floor ratio at 0.36
three_app.js:557 Bathroom_floor color update
home_app.js:177 home_app> mqtt temperature : Bedroom_floor ratio at 0.31
three_app.js:557 Bedroom_floor color update
home_app.js:177 home_app> mqtt temperature : Livingroom_window_floor ratio at 0.38
three_app.js:557 Livingroom_window_floor color update
home_app.js:170 home_app> mqtt heater : Kitchen_Heater ratio at 0.00
three_app.js:557 Kitchen_Heater color update
home_app.js:170 home_app> mqtt heater : Livingroom_Heater ratio at 0.00
three_app.js:557 Livingroom_Heater color update
home_app.js:170 home_app> mqtt heater : Bathroom_Heater ratio at 0.00
three_app.js:557 Bathroom_Heater color update
home_app.js:170 home_app> mqtt heater : Bedroom_Heater ratio at 0.00
three_app.js:557 Bedroom_Heater color update

three_app.js:557 Kitchen_floor color update
home_app.js:177 home_app> mqtt temperature : Kitchen_floor ratio at 0.37
three_app.js:557 Kitchen_floor color update
home_app.js:177 home_app> mqtt temperature : Kitchen_floor ratio at 0.37
three_app.js:557 Kitchen_floor color update
home_app.js:177 home_app> mqtt temperature : Livingroom_floor ratio at 0.36
three_app.js:557 Livingroom_floor color update
home_app.js:177 home_app> mqtt temperature : Livingroom_floor ratio at 0.36
three_app.js:557 Livingroom_floor color update
home_app.js:177 home_app> mqtt temperature : Livingroom_floor ratio at 0.36
three_app.js:557 Livingroom_floor color update
home_app.js:177 home_app> mqtt temperature : Livingroom_floor ratio at 0.36
three_app.js:557 Livingroom_floor color update
home_app.js:177 home_app> mqtt temperature : Office_floor ratio at 0.38
three_app.js:557 Office_floor color update
home_app.js:177 home_app> mqtt temperature : Office_floor ratio at 0.38
three_app.js:557 Office_floor color update
home_app.js:177 home_app> mqtt temperature : Bathroom_floor ratio at 0.35
three_app.js:557 Bathroom_floor color update
home_app.js:177 home_app> mqtt temperature : Balcony_floor ratio at 0.52
three_app.js:557 Balcony_floor color update
home_app.js:177 home_app> mqtt temperature : Balcony_floor ratio at 0.52
three_app.js:557 Balcony_floor color update
home_app.js:177 home_app> mqtt temperature : Balcony_floor ratio at 0.52
three_app.js:557 Balcony_floor color update
home_app.js:177 home_app> mqtt temperature : Bedroom_floor ratio at 0.31
three_app.js:557 Bedroom_floor color update
home_app.js:177 home_app> mqtt temperature : Bedroom_floor ratio at 0.31
three_app.js:557 Bedroom_floor color update
home_app.js:177 home_app> mqtt temperature : Bedroom_floor ratio at 0.31
three_app.js:557 Bedroom_floor color update
home_app.js:177 home_app> mqtt temperature : Bedroom_floor ratio at 0.31
three_app.js:557 Bedroom_floor color update
home_app.js:177 home_app> mqtt temperature : Livingroom_window_floor ratio at 0.38
three_app.js:557 Livingroom_window_floor color update
home_app.js:177 home_app> mqtt temperature : Livingroom_window_floor ratio at 0.38
three_app.js:557 Livingroom_window_floor color update
home_app.js:177 home_app> mqtt temperature : Livingroom_window_floor ratio at 0.38
three_app.js:557 Livingroom_window_floor color update
home_app.js:170 home_app> mqtt heater : Kitchen_Heater ratio at 0.00
three_app.js:557 Kitchen_Heater color update
home_app.js:177 home_app> mqtt temperature : Kitchen_floor ratio at 0.37
three_app.js:557 Kitchen_floor color update
home_app.js:177 home_app> mqtt temperature : Kitchen_floor ratio at 0.37
three_app.js:557 Kitchen_floor color update
home_app.js:177 home_app> mqtt temperature : Livingroom_floor ratio at 0.36
three_app.js:557 Livingroom_floor color update
home_app.js:177 home_app> mqtt temperature : Livingroom_floor ratio at 0.36
three_app.js:557 Livingroom_floor color update
home_app.js:177 home_app> mqtt temperature : Office_floor ratio at 0.39
three_app.js:557 Office_floor color update
home_app.js:177 home_app> mqtt temperature : Office_floor ratio at 0.39
three_app.js:557 Office_floor color update
home_app.js:177 home_app> mqtt temperature : Balcony_floor ratio at 0.52
three_app.js:557 Balcony_floor color update
home_app.js:177 home_app> mqtt temperature : Balcony_floor ratio at 0.52
three_app.js:557 Balcony_floor color update
home_app.js:177 home_app> mqtt temperature : Bedroom_floor ratio at 0.31
three_app.js:557 Bedroom_floor color update
home_app.js:177 home_app> mqtt temperature : Livingroom_window_floor ratio at 0.38
three_app.js:557 Livingroom_window_floor color update
home_app.js:177 home_app> mqtt temperature : Livingroom_window_floor ratio at 0.38
three_app.js:557 Livingroom_window_floor color update
home_app.js:177 home_app> mqtt temperature : Hallway_floor ratio at 0.32
three_app.js:557 Hallway_floor color update
home_app.js:177 home_app> mqtt temperature : Hallway_floor ratio at 0.32
three_app.js:557 Hallway_floor color update
home_app.js:177 home_app> mqtt temperature : Kitchen_floor ratio at 0.37
three_app.js:557 Kitchen_floor color update
home_app.js:177 home_app> mqtt temperature : Kitchen_floor ratio at 0.37
three_app.js:557 Kitchen_floor color update
wassfila commented 4 years ago

from a quick analysis, it looks like you're not sending the payload format expected by the 3d app. Your log shows teh publish topic with a payload of 4 bytes. The 3d apps expects a payload in JSON format likw the following below examples.

I have two types of devices that trun an MQTT message into color. You do not need to have the same device, but you would need to assign a type as shown here : https://github.com/HomeSmartMesh/smart_home_3d_webapp#mqtt-config-in-blender


If now you do not want to use json as it's complex for Arduino (I do have examples when I used json from ESP32, I used the ArduJSON library), and if you'd like to keep it simple, you need to modify this function to handle your custom payload even with bytes if you want 
https://github.com/HomeSmartMesh/smart_home_3d_webapp/blob/f34d24d913edbdc6914e4086f63c82c973db5376/src/home_app.js#L159
```javascript
function onMqttMessage(e){
    const obj_name = mqtt_mesh_name[e.detail.topic];
    const scene = three.getScene();
    if(typeof(scene) === "undefined"){
        console.warn(`home_app> mqtt message but scene not ready yet`);
        return
    }
    const obj = scene.getObjectByName(obj_name);
    if(obj.userData.type == "heating"){
        const heating_demand = e.detail.payload.pi_heating_demand;
        const ratio = heating_demand / 255;
        console.log(`home_app> mqtt heater : ${obj_name} ratio at ${ratio.toFixed(2)}`);
        send_custom_event('three_param',{name:obj_name,color_ratio:Math.sqrt(ratio)});
    }
    if(obj.userData.type == "temperature"){
        const temp = e.detail.payload.temperature;
        if(typeof(temp) != "undefined"){
            const ratio = (temp - (15.0)) / 28.0;
            console.log(`home_app> mqtt temperature : ${obj_name} ratio at ${ratio.toFixed(2)}`);
            send_custom_event('three_param',{name:obj_name,color_ratio:ratio});
        }
    }
}

Let me know if this helps you to get it working. Basically it's the e.detail.payload that you need to use in the sameformat that was sent by your Arduino example.

wassfila commented 4 years ago

By the way, I looked in your Arduino code and you're using .c_str() have look at : http://www.cplusplus.com/reference/string/string/c_str/ That is not converting the float to a string, only turning it into a pointer for Characters. I do not recommend that as you might not have binary compatibility between c++ and javascript, I recommend to at least turn the float into a text like String(f) string constructor or other.

enrutador commented 4 years ago

A lot of thanks Wassfila for your analisis, logs and comments¡¡¡¡. Now I understand much more how this fantastic development works.

I also have ESp32, if you have examples and can you upload them they will be very helpful,

Thanks for your time

wassfila commented 4 years ago

I'm glad you found it helpful.

When it comes to ESP32, all my projects are already shared, here are the links :

Here's a direct link to the github repo and function https://github.com/HomeSmartMesh/esp32_iot/blob/fd83f2f62ba76fabb3f61439a3899c72a1124a02/rgb_led/main/app_main.cpp#L406

Generating JSON payload should be even easier. And also, I used the Espressif SDK and not Arduino. If you want to use Arduino, you can find help on how to generate JSON payload in the library documentation, it's well documented https://arduinojson.org/

But most of the time, you don't need to generate JSON from the ESP32 and a text string is enough, that's how I did in the bed heater ESP32 here : https://github.com/HomeSmartMesh/esp32_iot/blob/master/bed_heater/main/app_main.c

You can find all my projects on Hackaday https://hackaday.io/projects/hacker/205962 each project has a link to the github repo. I have projects with ESP32, nRF52, STM32 and raspberry pi. Feel free to ask on each project page if you have questions, I can answer you there.

I will close this issue then, and you can re-open it if it still face problems.

enrutador commented 4 years ago

Great Wassfila ¡¡¡¡ ohhh very big suite of tools for investigate. I will read your works carefully. Yes please close the issue . Thank you very much ¡¡¡