detect no connection (initial startup, potential loss of)

[kayno]

Currently when the heatpump starts up (e.g. after a power outage/disconnection from power supply) the ESP8266 has to wait for some time before connect() will work. We should add some code to detect if the heatpump is connected or not, so that connect can be called. And by "add come code", I don't mean delay(99999) :)

I like the way the Arduino pubsubclient for MQTT does this:

while (!mqtt_client.connected()) {
    // Attempt to connect
    if (mqtt_client.connect(client_id, mqtt_username, mqtt_password)) {
      mqtt_client.subscribe(heatpump_set_topic);
    } else {
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }

The looping until connected is done in the user's sketch, not the library, which i think is the way to go. Allows the user to control how they deal with connecting, and what they want to do if it can't connect. We could allow for something like this:

while(!hp.connected()) {
  delay(2000);
  hp.connect();
}

Or if we could get connect() to detect if connect has worked, return a true/false value for the user to use, like they do in the mqtt example, except for heatpump it would look like:

while (!hp.connected()) {
  // Attempt to connect
  if (!hp.connect()) {
    // Wait 5 seconds before retrying
    delay(5000);
  }
}

So we just need to figure out a) how can we determine at anytime if we are connected to the hp, which will define what connected() returns, and b) how can we tell if connect() has worked or not.

I note from debugging that when I connect() to my heatpump I get back some packets we don't handle:

heatpump/debug {"packetSent":"fc 5a 01 30 02 ca 01 a8 "} // sending CONNECT[8] packet once
heatpump/debug {"packetSent":"fc 5a 01 30 02 ca 01 a8 "} // sending CONNECT[8] packet twice
heatpump/debug {"packetSent":"fc 42 01 30 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7b "} // sending a request for settings (sync() called from mqtt example)
heatpump/debug {"packetRecv":"fc 7a 01 30 01 00 54 00 38 ec fe 3f d8 04 10 40 c0 87 fe 3f 00 00 "} // what is this response - 7a??
heatpump/temperature {"roomTemperature":0} // mqtt example script reporting temperature, which hasn't been obtained from heatpump yet. should handle this (not a library issue, but an example issue)
heatpump/debug {"packetRecv":"fc 7a 01 30 01 00 54 00 5c ea fe 3f 10 3d 20 40 60 ea fe 3f 00 00 "} // another 7a packet, different data to last one... what is 7a? was it sent twice because we sent CONNECT[8] twice?
heatpump/debug {"packetRecv":"fc 62 01 30 10 02 00 00 00 08 08 00 00 00 00 0c ae 00 00 00 00 91 "} // finally a response to our request for settings, we now know the settings
heatpump {"power":"OFF","mode":"AUTO","temperature":23,"fan":"AUTO","vane":"AUTO","wideVane":"SWING"} //mqtt example reports settings (callback fired)
heatpump/debug {"packetSent":"fc 42 01 30 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7a "} //request room temp
heatpump/debug {"packetRecv":"fc 62 01 30 10 03 00 00 0b 00 00 aa 00 00 00 00 00 00 00 00 00 a5 "} //room temp response - we now know room temp
heatpump/temperature {"roomTemperature":21} //mqtt example reports room temp (callback fired)
heatpump/debug {"packetSent":"fc 42 01 30 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7b "} // periodic request settings - now into the normal sync() rhythm 
heatpump/debug {"packetRecv":"fc 62 01 30 10 02 00 00 00 08 08 00 00 00 00 0c ae 00 00 00 00 91 "}// periodic receive settings
heatpump/debug {"packetSent":"fc 42 01 30 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7a "}// periodic request room temp
heatpump/debug {"packetRecv":"fc 62 01 30 10 03 00 00 0b 00 00 aa 00 00 00 00 00 00 00 00 00 a5 "}// periodic receive  room temp

Perhaps the 0x7a is a connect() response?

And as for periodically determining if we are connected(), I am not sure how to do this. Perhaps if we don't get a response from a sync() call, we just assume we are disconnected? perhaps if it's 5 missed responses to a sync() request? I need some input from everyone on this one :)

[SwiCago]

@kayno, agree this was on the to do list. I never connected it to HP, while powered off..always while on. Interesting on the 7a response. Could be or it could also be a magic number signifying what HP type it is

[kayno]

I think the 0x7a might signify a response to connect, and the data in the packet may contain what kind of heatpump it is, and who knows what else!?

I haven't tried connecting ESP8266 whilst powered off either, but @uronito is doing this and says he needs a delay of 80 seconds before connect() can be called. I only ever plug my ESP8266 module in to the heatpump after flashing the latest version of the library to it when it's connected to the power supply, so I am yet to try it. But I know it would affect me if we had a power outage and I wasn't around to reboot the ESP8266 once the heatpump was ready for connect(). Plus, eventually I need to put the ESP8266 inside the heatpump, rather than have it continue to dangle out on the side!!

[kayno]

interesting that the data in 7a is different each time. i await with interest to see what @uronito posts in #11 where we are debugging another issue.

[SwiCago]

@kayno #11 is confirmed as fixed by @uronito, we may want to no look at the possibility of connection timeout, with re-connect feature

[jarrod180]

@kayno In my development I took the 0x7a response to mean "already connected" or "init not accepted". My reason: I had a loop like you guys did (now taken out), the first init packet I thought did not receive a reply BUT subsequent sends of the init packet DID receive a 0x7a response.

I also thought the 0x7a was an empty (header only message) though, I'll have to look at this again cos it definitely seems like you're getting some data there so you could be on to something.

These types of protocol normally include something like that where the unit identifies itself, you might be on to something there

[jarrod180]

Also, the way I plan to implement a "reconnect", is to determine that if I haven't received expected responses to my info requests for a while, send the init packet again to attempt to re-establish the connection.

[kayno]

@jarrod180 interesting that you say that the first time you send it, you don't get a response. I always get two responses, although this is only after reconnecting the ESP, not after powering down the heatpump at the mains

heatpump/debug {"packetSent":"fc 5a 01 30 02 ca 01 a8 "} // sending CONNECT[8] packet once
heatpump/debug {"packetSent":"fc 5a 01 30 02 ca 01 a8 "} // sending CONNECT[8] packet twice
heatpump/debug {"packetRecv":"fc 7a 01 30 01 00 54 00 38 ec fe 3f d8 04 10 40 c0 87 fe 3f 00 00 "}
heatpump/debug {"packetRecv":"fc 7a 01 30 01 00 54 00 5c ea fe 3f 10 3d 20 40 60 ea fe 3f 00 00 "} 

The data in each of the 7a packets differs too. Will be interesting to compare everyone's 7a packets and see!

Out of interest, I take it that someone has tried the code without sending the connect/init packet, i.e. just sending it requests for settings/room temp? Do we know for certain that the init packet has to be sent before it will respond to requests?

[jarrod180]

I wonder what's going on there in that data, I also wonder whether the 0xca and 0x01 in the connect packet mean something and can be altered. It could be useful for everyone to post their 0x7a responses and unit model/type to see if there's a commonality. Although, the control board might know nothing about the serial number or even the model type so who knows.

edit: IIRC, when I was testing yeah you don't get anything back without sending the connect packet, it's easy enough to test though

[kayno]

it seems the control boards are replaceable items. if you read the service manuals whenever a fault can't be fixed by switching it off and on again, etc, the solution for the repairman is always "replace the faulty board". So it would seem unlikely that it would report serial numbers of the unit. Model number of series is more likely though.

[schotime]

They are definitely replaceable. I may or may not have had one replaced due to a faulty 5v -> 3.3v converter, which did bad things to it's brain.

This is why I have switched to using the nano. No converter needed.

[kayno]

@schotime oh no! that was always my biggest fear with this - frying the control board. Was it a faulty voltage converter or logic level converter?

I know you probably don't want to dwell on it, but would be good to know what happened so we can all learn, and perhaps think about ways to safeguard against this. Maybe a diode somewhere in the cable to prevent damage by reverse polarity, or something...

[schotime]

faulty logic level converter, measured an open circuit from the 3.3v side to the 5v side which slowly but surely started to fry the internals of the raspberry pi, causing it to draw more from the power supply unit connected to the 12v line from the unit which burnt out the 12v circuit in the heatpump control board until the pi was dead and the control board was dead.

[kayno]

damn, that's tough. i can see why you switched to nano - no need to touch that 12V supply. thanks for the info. is there anything you have done differently with the nano hookup based on your experience with the pi? should we be considering additional components to prevent things like this?

[schotime]

Well for the Nano, it just plugs directly into the unit. There are no extra parts required. I'll take a photo and post. Super easy.

[schotime]

The power supply hooks up directly then the radio connects.

[kayno]

oh - no resistors tying tx and rx to 5v?

i'm more thinking of how to prevent accidental cross wiring. for example i am always pulling my ESP out and flashing it. I'm super careful about connecting this right wires to the right pins after flashing (and making sure i disconnect GND last when removing it and connect GND first when I put it back), but I'm wondering if say putting a diode on the 5v and/or gnd line for reverse polarity protection would be advisable? I don't know enough about electronics to know for sure..

looking at your photo, what is the circuit board on the right?

[schotime]

Its the NRF24L01+ radio hooked up like https://www.mysensors.org/build/connect_radio

For the moment I'm just pulling the whole plug out. But I have another one that i just have a male -> female black adapters taped together at an intermediate point.

[kayno]

ah ok. cool. forgot the nano didn't have wifi, i'm so used to using the ESP8266 for everything arduino!

I've got a similar cable from heatpump -> ESP as you have, with individual female headers that i plug into 4 different pins on the ESP (adafruit huzzah esp8266). But i plugged the cable into the heatpump, and put the heatpump's cover back on, just leaving the female headers hanging out through a gap in the cover. I have also soldered in the resistors in-line on the cable, and heatshrinked them up.

[schotime]

Not having WIFI has been good. No passwords or anything stored on them, and these radios work in a mesh network. Great for my 5 split systems.

Just have one gateway which communicates with them all.

[schotime]

Found these yesterday too. https://www.amazon.com/gp/product/B010O1G1ES

[kayno]

I'm waiting on the Wemos D1 mini pro to come back in stock. Plan on getting 10 odd, and IoT'ing anything that I can open with a screwdriver :)

[kayno]

today i had a power outage, and when it came back on, mosquitto (my mqtt broker) didn't restart. i started mosquitto, and subscribed to the heatpump topic, and the heatpump was already sending out debug request settings/temp packets.

this makes me a) wonder if the connect/init packet is actually required, because it would have been sent within about 3-4 secs of the power coming back on and the heatpump would have only had the same amount of time to start up and b) think that if it is needed, then unlike @uronito I do not need much/if any delay between the heatpump powering on and the connect packets being sent.

[kayno]

I can confirm that at least 1 CONNECT packet is required! I commented out the writePacket() call in connect() to test (so all connect() did was open the serial port) and I also enhanced the custom packet calls so that you can send < 20 bytes (so I can test sending the CONNECT packet manually.

When I connected the ESP (without it auto-sending the CONNECT packets, I got no response to the requests for settings/room temp.,Then I manually sent the CONNECT packet, and it all came to life:

heatpump/debug {"packetSent":"fc 42 01 30 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7b "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7a "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7b "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7a "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7b "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7a "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7b "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7a "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7b "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7a "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7b "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7a "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7b "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7a "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7b "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7a "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7b "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7a "} // no response
heatpump/debug {"packetSent":"fc 42 01 30 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7b "} // no response
heatpump/set {"custom": "5a 01 30 02 ca 01 a8"} // manual CONNECT packet published to topic
heatpump/debug {"customPacket":"5a 01 30 02 ca 01 a8 "} // debug in mqtt example, showing what it received
heatpump/debug {"packetSent":"fc 5a 01 30 02 ca 01 a8 "} // custom packet sent to heatpump
heatpump/debug {"packetSent":"fc 42 01 30 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7a "} // (another request is also sent, before a read happens)
heatpump/debug {"packetRecv":"fc 7a 01 30 01 00 54 3f 7d 00 00 00 bc 16 ff 3f d0 00 00 00 00 00 "} // 0x7a response to the CONNECT
heatpump/debug {"packetRecv":"fc 62 01 30 10 03 00 00 0d 00 00 af 00 00 00 00 00 00 00 00 00 9e "} // room temp comes back too, business as usual!
heatpump/temperature {"roomTemperature":23.00} // room temp published
heatpump/debug {"packetSent":"fc 42 01 30 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7b "} // request for settings
heatpump/debug {"packetRecv":"fc 62 01 30 10 02 00 00 00 03 03 00 00 00 00 0c b8 00 00 00 00 91 "} // settings received
heatpump {"power":"OFF","mode":"COOL","temperature":28.00,"fan":"AUTO","vane":"AUTO","wideVane":"SWING"} // settings published
heatpump/debug {"packetSent":"fc 42 01 30 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7a "} // happy days!
heatpump/debug {"packetRecv":"fc 62 01 30 10 03 00 00 0d 00 00 af 00 00 00 00 00 00 00 00 00 9e "} // rinse repeat

I also tried randomly sending it the CONNECT string, and each time I did I got the 7a response:

heatpump/set {"custom": "5a 01 30 02 ca 01 a8"}
heatpump/debug {"customPacket":"5a 01 30 02 ca 01 a8 "}
heatpump/debug {"packetSent":"fc 5a 01 30 02 ca 01 a8 "}
heatpump/debug {"packetRecv":"fc 7a 01 30 01 00 54 3f 7d 00 00 00 bc 16 ff 3f d0 00 00 00 00 00 "}
...
heatpump/set {"custom": "5a 01 30 02 ca 01 a8"}
heatpump/debug {"customPacket":"5a 01 30 02 ca 01 a8 "}
heatpump/debug {"packetSent":"fc 5a 01 30 02 ca 01 a8 "}
heatpump/debug {"packetRecv":"fc 7a 01 30 01 00 54 3f 7d 00 00 00 bc 16 ff 3f d0 00 00 00 00 00 "}

Same data in the response as well, each time.

I will submit a PR for my changes to the custom packet stuff. I think I have it pretty fool proof now.

[SwiCago]

@kayno, my first demo webapp sent has send connect manually too. And only a single connect is needed, however I did see KUMO cloud send connect twice, so that is why I coded to send it twice, in case the first failed...but then again, if we check for no response in x seconds, resend connect that would take care of that. And yes the 7a seems to be the confirm connection and the other bytes some extra info about the unit maybe or it's current setting at startup?

[SwiCago]

@kayno BTW, I found that it is enough to pull the TX and RX of an ESP-01 to 3.3V with 10k resistors. I soldered SMD 10k resistors directly to the ESP-01 pins. May not be pretty ;) but works great. Made a simple adapter plug, so I can quickly connect and disconnect from the heatpump. I chose the ESP-01 beacuase it fits in the small void next to the RF shield of my wall units. I'll post a pic of my setup tonight. Do basically the TX and RX can either be pulled up to 5V or 3.3V, what ever is easier for the end user

[lekobob]

I don't know the best place to post this so I will add it here for now. I created a custom component for Home Assistant that uses a standard climate component to control my heatpump . You can find it here https://github.com/lekobob/mitsu_mqtt

[SwiCago]

@lekobob , if you would like to, do a branch of the library and add this as an example for home assistant integration, then request a pull. I'd also like to see one for openhab as well. Gives people something to start with.

[kayno]

@lekobob @SwiCago perhaps an "integrations" folder, and within that "openhab.org" and "home-assistant.io" folders?

I'm not sure if there is a convention for this?

[SwiCago]

@kayno @schotime can either of you test my "test" branch. Use the kayno's MQTT example sketch. I added auto self re-connect if a loss of data is detected and also a bool return for connect, so sketches can wait till connected before continuing. Works on my setup, but want to be sure it works for others, before merge. I did following tests -Plug in while live. -Plug in while powered off -Power cycle HeatPumps with a few minute off time -Pulled TX and RX wires and after seeing re-connect plugged them back in. Thanks

[schotime]

@SwiCago @kayno I've tested the changes in that sketch but I don't use mqtt so not the exact sketch. It is connecting so thats good but there is something strange going on.....

Was getting the wrong values out of the temp reading. Was reporting 24.5 but after i restarted it was 22.5. Setting the temp wasn't working either. Will need to investigate further.

[schotime]

Rolled back to the version before the autoReconnect and it works as expected again.

Just looking at the code...it could be to do with the loops and millis(). You are storing it in an int, where as it needs to be a long. This isn't a problem on your chip probably because they're 32bit but the nano is 8bit. I'll see if I can have a quick look.

[SwiCago]

@schotime if you figure it out, do a fresh branch and add the changes, we can then test to see if it still works for ESP correctly and then do a pull/merge

[kayno]

@schotime nice pickup on the millis() into int. looking at https://www.arduino.cc/en/reference/millis:

Please note that the return value for millis() is an unsigned long, logic errors may occur if a programmer tries to do arithmetic with smaller data types such as int's. Even signed long may encounter errors as its maximum value is half that of its unsigned counterpart.

I'll do a pull request now - I can see three places where it needs to be changed.