Switch to esp-idf and use uarts instead of hardware serial

[dwmw2]

Still building with Arduino locally as there are other things the esp-idf compiler is more picky about (mostly unhandled cases in switch() statements). And mostly untested as I haven't actually plugged anything into my heat pumps yet. But it does seem to be sending and receiving serial port bytes.

[gekkekoe]

good first attempt, thnx! there are 2 flows for the serial 1) handle loop (sends the get status cmd and resend cmd that have not been acknowledged) 2) response listener, that reads results and update the esphome sensors. currently this works via a thread

the component is now a polling device and an uart device. the polling loop is used for query and uart for reading. that's okay, but probably can be combined, though the polling interval might need to be adjusted

double check that the Read uarts are thread safe (not needed if everything is in one loop), else need to use mutex. in the esphome model they don't really like you to make your own threads, you need to handle everything via the loop/update. that's a bit annoying since messages are only handled per loop interval.

also when you have the hardware, enable debug yaml and monitor the loop time when applying changes, should be around 14ms.

[dwmw2]

Thanks.

So with what I have so far, I've moved handle_response() to the periodic loop, while handle_loop() is still called less frequently from the update() method. I'm guessing esphome invokes those all on the same thread at different times?

It's tempting to combine them as you say, perhaps by _(renaming handle_loop() to actually say something about tx and)_ moving it to be called from loop() too. We'd probably need to make sure it only sends the next packet when there are no responses outstanding (that haven't timed out).

The lack of interrupt support is annoying but I suspect 14ms is probably reasonable latency. At 2400 baud isn't a full character (plus start plus stop bit) going to take about 4.2ms anyway?

[dwmw2]

(I do have the ESP32 hardware; it's just not connected to the heat pumps yet. It's connected to my PC's serial port where I'm watching the �Zz�\ packets turn up while I type crap at it and see the logs...

[08:56:17][I][ecodan.component:253]: Attempt to tx CONNECT_CMD!
[08:56:17][D][uart_debug:114]: >>> FC 5A 02 7A 03 CA 01 00 5C

[08:56:34][E][ecodan.component:059]: Dropping serial data, header magic mismatch
[08:56:34][D][uart_debug:114]: <<< 73 64 66 73 64

)

[dwmw2]

Honestly, it could probably do with looking a lot more like https://github.com/tobias-93/esphome-ecodan-heatpump/blob/main/components/ecodan/ecodan.cpp although there are things I'm not fond of in that one too. Will play with it some more, but probably ought to get some real work done today :)

[gekkekoe]

Honestly, it could probably do with looking a lot more like https://github.com/tobias-93/esphome-ecodan-heatpump/blob/main/components/ecodan/ecodan.cpp although there are things I'm not fond of in that one too. Will play with it some more, but probably ought to get some real work done today :)

I really don't like multi line macros, so I want to stay away from that if possible. Also agree on the real work lol

Thanks.

So with what I have so far, I've moved handle_response() to the periodic loop, while handle_loop() is still called less frequently from the update() method. I'm guessing esphome invokes those all on the same thread at different times?

It's tempting to combine them as you say, perhaps by _(renaming handle_loop() to actually say something about tx and)_ moving it to be called from loop() too. We'd probably need to make sure it only sends the next packet when there are no responses outstanding (that haven't timed out).

It can be done. also keep in mind how you want to control a second uart, in the eshome model you need to probably instantiate another uart device for the slave, haven't looked into that.

The lack of interrupt support is annoying but I suspect 14ms is probably reasonable latency. At 2400 baud isn't a full character (plus start plus stop bit) going to take about 4.2ms anyway?

use that as a check to see if you introduced something that takes 'a lot' of time.

I really also don't understand why esphome does it like this. The esp32 have dual core, and I remember reading that one was for wifi en esphome stuff, and the other was for the user. I did some tests and it seemed that own code runs on the other core, but that was at the beginning. but if we can make it work nicely, then 8226 is also supported (since I did want spent time on removing the threads, and that's blocking support for 8226 (though these don't have enough hardware uarts, making proxy difficult)

[dwmw2]

I can play with instantiating the second UART later; I don't think it's difficult. We won't be a superclass of that one; it'll be optional if there is a proxy_uart setting.

Agree about the multi-line macros, although as a Linux kernel hacker I've been guilty of more than a few of those myself in the past :)

Once your serial_rx() has got the header, it'll block until the remainder of the packet comes in. On first glance I think that means if the UART drops a byte, it'll wait a full ten seconds before giving up on that packet? In the actual mainloop. That's a bit more than 14 ms :)

I think I'll refactor it to keep a local Message and just add a byte at a time to it until it's got enough.

[gekkekoe]

I was cheating a bit, serial_rx lives on another thread, don't think the esphome loop counter counts that... but during testing, it's always almost instant, but I just wait up till 10s just to be sure.

[dwmw2]

I think if the serial port were to drop a byte due to wiring problems, it'd hit the 10 second thing. And I'm attempting to move that from your thread into the main loop() :)

[gekkekoe]

I think if the serial port were to drop a byte due to wiring problems, it'd hit the 10 second thing. And I'm attempting to move that from your thread into the main loop() :)

yes agree, but in the main loop we should not try up to 10s. during testing I always got it right away, but if it fails somehow we could be wasting 10s.

[dwmw2]

Made it take a single byte at a time as and when they're available, until it has a whole packet. Maybe it should have a timeout for partial packets, but that's easy enough to add.

I refactored the uart support a little not to subclass UARTComponent, because that makes the proxy easier. Then added basic proxy support (just for receiving messages; no idea what to do with them).

[dwmw2]

uart:
 - id: uart_main
   rx_pin:
    number: GPIO34
    inverted: true
   tx_pin:
    number: GPIO48
    inverted: true
   baud_rate: 2400
   parity: EVEN
   stop_bits: 1
   debug:
    direction: BOTH
    dummy_receiver: false
    after:
      delimiter: "\n"
    sequence:
      - lambda: UARTDebug::log_hex(direction, bytes, ' ');

 - id: uart_proxy
   rx_pin:
    number: GPIO44
    inverted: true
   tx_pin:
    number: GPIO43
    inverted: true
   baud_rate: 2400
   parity: EVEN
   stop_bits: 1
   debug:
    direction: BOTH
    dummy_receiver: false
    after:
      delimiter: "\n"
    sequence:
      - lambda: UARTDebug::log_hex(direction, bytes, ' ');

ecodan:
  id: ecodan_instance
  uart_id: uart_main
  proxy_uart_id: uart_proxy

[gekkekoe]

thanks a lot, really liking what I'm seeing so far. Hopefully you will get the hardware soon to test.

for proxy we need to see if the melcloud and procon can handle another entity sending cmd while it's connected. I spoke with the heishamon guys and they only support read-only when a slave is connected. The slave crashes and behaves abnormally when their unit was also sending. I hope that mitsubishi is a bit more robust.

Also too bad I sold my procon, since I was done with reverse engineering :o so I can only proxy the component with itself, and will de pend partly on you for actual device testing if thats okay.

one note with the nonblocking serial rx. esphome complains if you are taking more than 200ms per loop. The esphome way I think is to continue on the next iteration

[dwmw2]

I was assuming we'd forward anything we receive from the MelCloud unit (except the connect command, to which we reply ourselves if we're connected to the HP). And then we filter the responses and send back only responses to the commands that it sent. We can snoop on them ourselves though.

[gekkekoe]

I was assuming we'd forward anything we receive from the MelCloud unit (except the connect command, to which we reply ourselves if we're connected to the HP). And then we filter the responses and send back only responses to the commands that it sent. We can snoop on them ourselves though.

I was hoping that we could also send set cmd, because the melcloud and procon periodically sends out status query that will pick up the response of our cmd. So I have good hope that this works, but needs testing

[dwmw2]

My CN105 socket cable arrived today so I could theoretically hook something up. The only device I have with two 5v-capable UARTs is the Tindie Quad FTDI thing so maybe I could use that, but I'll have to finish porting do that it can run in the host platform first.

[dwmw2]

I was hoping that we could also send set cmd

I think we can send whatever we like to the HP. We might just have to limit which responses we forward to the MelCloud wifi unit, to avoid confusing it with responses to commands it didn't send?

[dwmw2]

one note with the nonblocking serial rx. esphome complains if you are taking more than 200ms per loop. The esphome way I think is to continue on the next iteration

That's exactly what I'm doing. Consume as many bytes I can in each loop iteration but never wait. Just leave the packet there ready to have more bytes added to it.

[gekkekoe]

I was hoping that we could also send set cmd

I think we can send whatever we like to the HP. We might just have to limit which responses we forward to the MelCloud wifi unit, to avoid confusing it with responses to commands it didn't send?

it will work, but the slave is the issue, need to verify how it behaves, but the way it does the get all statuses, I'm having good hopes. Some cmd are dependent on the internal state of the slave, so those are interesting to see.

My CN105 socket cable arrived today so I could theoretically hook something up. The only device I have with two 5v-capable UARTs is the Tindie Quad FTDI thing so maybe I could use that, but I'll have to finish porting do that it can run in the host platform first.

you did also order the heishamon right (https://www.tindie.com/products/thehognl/heishamon-communication-pcb/) ? that one has 5v connectors with proper levelshifting. It comes with a esp32-s3, so pretty much perfect. It's used for the panasonic heatpumps.

[dwmw2]

Yep, ordered one of those. Sadly it doesn't run off the 12v like the Faikin does, but I have a converter I can use for that if needs be. He's sending it with the connector cable already set up with the right pinout for the CN105; he said you'd also asked :)

Pushed a bunch of changes and now it's almost building. Just need to chase down the last incomplete switch statements...

/home/dwmw/esphome/base.yaml: In lambda function:
/home/dwmw/esphome/base.yaml:154:12: error: enumeration value 'OFF' not handled in switch [-Werror=switch]
/home/dwmw/esphome/base.yaml:154:12: error: enumeration value 'HEAT_COMPENSATION_CURVE' not handled in switch [-Werror=switch]

[dwmw2]

I have it working with esp-idf now. (For same version of 'working' as before; I can send it packets on either the main or the proxy UART and I see log messages confirming that they're received)

[gekkekoe]

Yep, ordered one of those. Sadly it doesn't run off the 12v like the Faikin does, but I have a converter I can use for that if needs be. He's sending it with the connector cable already set up with the right pinout for the CN105; he said you'd also asked :)

Pushed a bunch of changes and now it's almost building. Just need to chase down the last incomplete switch statements...

/home/dwmw/esphome/base.yaml: In lambda function:
/home/dwmw/esphome/base.yaml:154:12: error: enumeration value 'OFF' not handled in switch [-Werror=switch]
/home/dwmw/esphome/base.yaml:154:12: error: enumeration value 'HEAT_COMPENSATION_CURVE' not handled in switch [-Werror=switch]

LOL -:)

Why do you want to power it via 12v ? The board uses the 5v from the heatpump and redirects the 12v to the slave. I know for one that the procon needs 12v (slave) to boot.

[dwmw2]

Apparently the 5v isn't always there, and might not be sufficient to power both boards.

[gekkekoe]

Apparently the 5v isn't always there, and might not be sufficient to power both boards.

I also relied on it with the atomS3 worked fine (been using atom for half year, and before that a custom esp32 board for about a year). So the procon did not use the 5v, it only wanted 12v. I expect the melcloud adapter to be the same.

the cn105 has 5v and a 12v, so you can use the first for the esp and the 12v for the slave.

default pinmapping of heishamon board:

define HEATPUMPRX 18

define HEATPUMPTX 17

define PROXYRX 9

define PROXYTX 8

[dwmw2]

Host mode working now too with this config.

I have it and the Faikin sending CONNECT_CMD to each other.

[17:26:02][I][ecodan.component:252]: Attempt to tx CONNECT_CMD!
[17:26:02][D][uart_debug:114]: >>> FC 5A 02 7A 03 CA 01 00 5C
[17:26:05][I][ecodan.component:232]: Unknown serial message type received: 0x5a
[17:26:06][D][uart_debug:114]: <<< FC 5A 02 7A 03 CA 01 00 5C

If I connect one to the proxy UART of the other, I see:

[17:26:44][I][ecodan.component:010]: Proxy RX: RES { .Hdr { FC, 5A, 02, 7A, 03 } .Payload { CA, 01, 00, 5C, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[17:26:44][D][uart_debug:114]: <<< FC 5A 02 7A 03 CA 01 00 5C

[dwmw2]

default pinmapping of heishamon board:

define HEATPUMPRX 18

define HEATPUMPTX 17

define PROXYRX 9

define PROXYTX 8

Where did you get those from? The documentation seems quite sparse, and the pics on Tindie don't seem to show the model with the proxy port.

I think I'm about ready to remove the "DO NOT MERGE" from this PR... except for the fact that it hasn't actually been tested in anger.

[gekkekoe]

default pinmapping of heishamon board:

define HEATPUMPRX 18

define HEATPUMPTX 17

define PROXYRX 9

define PROXYTX 8

Where did you get those from? The documentation seems quite sparse, and the pics on Tindie don't seem to show the model with the proxy port.

I think I'm about ready to remove the "DO NOT MERGE" from this PR... except for the fact that it hasn't actually been tested in anger.

https://github.com/IgorYbema/HeishaMon in https://github.com/IgorYbema/HeishaMon/blob/main/HeishaMon/HeishaMon.ino I will try to flash it on the atom s3 in the heatpump en see if I can find some spare time

[gekkekoe]

did you push everything ? I am getting some config errors with default ecodan-esphome.yaml

Failed config

uart: [source ecodan-esphome.yaml:97]

  Pin 2 is used in multiple places.
  - [source ecodan-esphome.yaml:97]
    id: uart_ecodan
    rx_pin: 
      number: 2
      inverted: True
      mode: 
        input: True
        output: False
        open_drain: False
        pullup: False
        pulldown: False
      ignore_pin_validation_error: False
ecodan: [source ecodan-esphome.yaml:116]

  Pin 2 is used in multiple places.
  id: ecodan_instance
  uart_id: uart_ecodan
  rx_pin: 2
  tx_pin: 1
  update_interval: 1000ms
Pin 1 is used in multiple places

Pin 1 is used in multiple places

[dwmw2]

Take the pins off the Ecodan instance.

[dwmw2]

I haven't removed the tx_pin and rx_pin options yet, although they do nothing (except make it complain about a conflict). I was going to see if I could make it automatically instantiate a UART based on an old-style config.

[gekkekoe]

I haven't removed the tx_pin and rx_pin options yet, although they do nothing (except make it complain about a conflict). I was going to see if I could make it automatically instantiate a UART based on an old-style config.

ah ok, going to remove them locally then

[gekkekoe]

also

esp32:
  board: esp32-s3-devkitc-1
  framework:
    type: arduino
    version: latest

in esp32s3.yaml still uses arduino right?

[dwmw2]

Yes, I didn't change esp32s3.yaml. But it should work with esp-idf now (might need newer version for some ESP32S3 boards, as I did for the AtomS3 Lite).

[gekkekoe]

[21:05:23][D][uart_debug:114]: >>> FC 5A 02 7A 03 CA 01 00 5C
[21:05:28][D][uart_debug:114]: >>> FC 5A 02 7A 03 CA 01 00 5C
[21:05:33][D][uart_debug:114]: >>> FC 5A 02 7A 03 CA 01 00 5C

receiving seems to fail, let me see if I can debug it

[gekkekoe]

if I change config to

uart:
  id: uart_ecodan
  rx_pin:
    number: GPIO2
    mode:
      input: true
      pullup: true
  tx_pin:
    number: GPIO1
    mode:
      output: true
  baud_rate: 2400
  parity: EVEN
  stop_bits: 1

the connect works, but I seem to get checksum fails on response, but at least we are talking to the ecodan!

21:43:41][D][uart_debug:114]: <<< FC 61 02 7A 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 13
[21:43:42][D][uart_debug:114]: >>> FC 41 02 7A 10 34 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FE FC 42 02 7A 10 A2 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 90
[21:43:42][I][ecodan.component:072]: Serial port message checksum invalid
0 00 00 00 00 00 00 00 00 00 00 00 00 00 13

Header receive seems okay, payload seems to fail (only 0s)

[gekkekoe]

I dumped the msg while being read:

[22:22:24][W][ecodan.component:053]: Write offset: 0
[22:22:24][W][ecodan.component:054]: RES { .Hdr { 00, 00, 00, 00, 00 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 1
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 00, 00, 00, 00 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 2
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 00, 00, 00 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 3
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 00, 00 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 4
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 00 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 5
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 6
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 7
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 8
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 9
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 10
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 11
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 12
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 13
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 14
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 15
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 16
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 17
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 18
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 19
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 20
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }
[22:22:24][W][ecodan.component:053]: Write offset: 21
[22:22:24][W][ecodan.component:054]: RES { .Hdr { FC, 61, 02, 7A, 10 } .Payload { 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00, 00 } .Chk { 00 } }

its really odd that the payload is always 0, if I have some more time tomorrow I will try to trace it. Do you want my locally adjusted files ? (can I push it to your branch?)

[dwmw2]

Weird. I'd only fed it one packet with

echo -en \\xfc\\x5A\\x02\\x7A\\x03\\xCA\\x01\\x00\\x5c  > /dev/ttyUSB0 

It always liked that. But it doesn't like yours when I

echo -en \\xFC\\x61\\x02\\x7A\\x10\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x13 > /dev/ttyUSB0

I think writeOffset_ is one too large when it's calculated?

[gekkekoe]

let me dump that var on every append. Ehm it was dumped already. I basically did this:

            ESP_LOGW(TAG, "Write offset: %d", msg.get_write_offset());
            ESP_LOGW(TAG, msg.debug_dump_packet().c_str());

            // Add the byte to the packet.
            msg.append_byte(data);

so before the append, it seems okay

[dwmw2]

Yeah, just writeOffset_--; as the first line of verify_checksum() fixes it I think? It's because append_byte() increments writeOffset_ even past the checksum byte.

[gekkekoe]

riteOffset_--;

that worked

[22:36:46][D][text_sensor:064]: 'Reset Reason': Sending state 'Software Reset CPU'
[22:36:46][I][ecodan.component:252]: Attempt to tx CONNECT_CMD!
[22:36:47][D][uart_debug:114]: >>> FC 5A 02 7A 03 CA 01 00 5C
[22:36:47][I][ecodan.component:208]: connection reply received from heat pump
[22:36:47][D][uart_debug:114]: <<< FC 7A 02 7A 01 00 09
[22:36:48][D][uart_debug:114]: >>> FC 41 02 7A 10 34 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FB FC 42 02 7A 10 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 30
[22:36:48][D][uart_debug:114]: <<< FC 61 02 7A 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 13
[22:36:48][D][sensor:094]: 'ESP Uptime': Sending state 9.76900 s with 0 decimals of accuracy
[22:36:49][D][uart_debug:114]: >>> FC 41 02 7A 10 34 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 F7 FC 42 02 7A 10 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 2F
[22:36:49][D][uart_debug:114]: <<< FC 61 02 7A 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 13
[22:36:50][D][uart_debug:114]: >>> FC 41 02 7A 10 34 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 EF FC 42 02 7A 10 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 2E
[22:36:50][D][uart_debug:114]: <<< FC 61 02 7A 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 13
[22:36:51][D][sensor:094]: 'Heap Free': Sending state 211108.00000 B with 0 decimals of accuracy
[22:36:51][D][sensor:094]: 'Loop Time': Sending state 26.00000 ms with 0 decimals of accuracy
[22:36:51][D][uart_debug:114]: >>> FC 41 02 7A 10 34 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 DF FC 42 02 7A 10 05 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 2D
[22:36:51][D][uart_debug:114]: <<< FC 61 02 7A 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 13
[22:36:52][D][uart_debug:114]: >>> FC 41 02 7A 10 34 40 00 00 00 00 00 00 00 00 00 00 00 00 00 00 BF FC 42 02 7A 10 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 2B
[22:36:52][D][uart_debug:114]: <<< FC 61 02 7A 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 13
[22:36:53][D][uart_debug:114]: >>> FC 41 02 7A 10 32 20 00 00 00 00 00 00 12 5C 00 00 00 00 00 00 73 FC 42 02 7A 10 09 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 29
[22:36:53][D][uart_debug:114]: <<< FC 61 02 7A 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 13
[22:36:53][D][sensor:094]: 'ESP Uptime': Sending state 14.76900 s with 0 decimals of accuracy
[22:36:54][D][uart_debug:114]: >>> FC 41 02 7A 10 32 08 00 00 00 00 04 00 00 00 00 00 00 00 00 00 F5 FC 42 02 7A 10 0B 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 27
[22:36:54][D][uart_debug:114]: <<< FC 61 02 7A 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 13
[22:36:55][D][uart_debug:114]: >>> FC 42 02 7A 10 0C 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 26
[22:36:55][D][sensor:094]: 'Feed Temp': Sending state 21.50000 °C with 1 decimals of accuracy
[22:36:55][D][sensor:094]: 'Return Temp': Sending state 22.50000 °C with 1 decimals of accuracy
[22:36:55][D][sensor:094]: 'DHW Current Temp': Sending state 48.00000 °C with 1 decimals of accuracy
[22:36:55][D][sensor:094]: 'DHW Secondary Temp': Sending state 25.00000 °C with 1 decimals of accuracy
[22:36:55][D][climate:396]: 'DHW Climate' - Sending state:
[22:36:55][D][climate:399]:   Mode: HEAT
[22:36:55][D][climate:401]:   Action: IDLE
[22:36:55][D][climate:419]:   Current Temperature: 48.00°C
[22:36:55][D][climate:425]:   Target Temperature: 47.00°C
[22:36:55][D][uart_debug:114]: <<< FC 62 02 7A 10 0C 08 66 75 08 CA 77 12 C0 B0 09 C4 0B 00 00 00 80
[22:36:56][D][sensor:094]: 'Heap Free': Sending state 211108.00000 B with 0 decimals of accuracy
[22:36:56][D][sensor:094]: 'Loop Time': Sending state 26.00000 ms with 0 decimals of accuracy
[22:36:56][D][uart_debug:114]: >>> FC 42 02 7A 10 0D 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 25
[22:36:56][D][sensor:094]: 'Boiler Flow Temp': Sending state 25.00000 °C with 1 decimals of accuracy
[22:36:56][D][sensor:094]: 'Boiler Return Temp': Sending state 25.00000 °C with 1 decimals of accuracy
[22:36:56][D][uart_debug:114]: <<< FC 62 02 7A 10 0D 09 C4 0B 09 C4 0B 09 C4 0B 09 C4 0B 00 00 00 A5
[22:36:57][D][uart_debug:114]: >>> FC 42 02 7A 10 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22
[22:36:57][D][binary_sensor:036]: 'In1 Thermostat': Sending state ON
[22:36:57][D][binary_sensor:036]: 'In6 Thermostat 2': Sending state OFF
[22:36:57][D][binary_sensor:036]: 'In5 Outdoor Thermostat': Sending state OFF
[22:36:57][D][uart_debug:114]: <<< FC 62 02 7A 10 10 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01
[22:36:58][D][uart_debug:114]: >>> FC 42 02 7A 10 13 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1F
[22:36:58][D][sensor:094]: 'Operating Runtime': Sending state 12148.00000 h with 0 decimals of accuracy
[22:36:58][D][uart_debug:114]: <<< FC 62 02 7A 10 13 00 00 30 00 79 00 00 00 00 00 00 00 00 00 00 56
[22:36:58][D][sensor:094]: 'ESP Uptime': Sending state 19.76900 s with 0 decimals of accuracy
[22:36:59][D][uart_debug:114]: >>> FC 42 02 7A 10 15 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1D
[22:36:59][D][binary_sensor:036]: 'Water Pump Status': Sending state OFF
[22:36:59][D][binary_sensor:036]: '3-way Valve': Sending state OFF
[22:36:59][I][ecodan.component:040]: Could not publish state of sensor 'status_water_pump_2' with value: '0'
[22:36:59][I][ecodan.component:040]: Could not publish state of sensor 'status_three_way_valve_2' with value: '0'
[22:36:59][D][uart_debug:114]: <<< FC 62 02 7A 10 15 00 64 FF 00 00 00 00 00 00 00 04 00 00 00 00 96
[22:37:00][D][uart_debug:114]: >>> FC 42 02 7A 10 14 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1E
[22:37:00][D][sensor:094]: 'Flow Rate': Sending state 0.00000 L/min with 0 decimals of accuracy
[22:37:00][D][binary_sensor:036]: 'Booster Heater': Sending state OFF
[22:37:00][D][binary_sensor:036]: 'Immersion Heater': Sending state OFF
[22:37:00][D][uart_debug:114]: <<< FC 62 02 7A 10 14 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FE

its populating values in esphome now. But we should adjust read actions to read writeOffset_ -1 I guess ? Because that was the last valid data

[dwmw2]

Well, now verify_checksum() is already taking that extra one off, so I think it should be correct. Not entirely pretty but I'll give it some more thought in the morning. The solution may end up being a nicer code comment along with the writeOffset--; to make it look like it's by design :)

[gekkekoe]

I did some testing with setting values temps and that seems to work. going to check if esp-idf works.

[gekkekoe]

can confirm that these settings works: esp32: board: esp32-s3-devkitc-1 variant: esp32s3 framework: type: esp-idf version: 5.0.2 platform_version: 6.3.2

[dwmw2]

Nice, thanks for testing. I'll clean up the code a little in the morning. Want to push your config files and other changes, and I'll work them in?

[gekkekoe]

esp32:
  board: esp32-s3-devkitc-1
  variant: esp32s3
  framework:
    type: esp-idf
    version: recommended
    # Custom sdkconfig options
    sdkconfig_options:
      COMPILER_OPTIMIZATION_SIZE: y
    # Advanced tweaking options
    advanced:
      ignore_efuse_mac_crc: false  

the recommended settings also works from the esphome site.

I tried to push my local changes but it failed :/

[gekkekoe]

patch.txt

[gekkekoe]

I will swap back to the previous version, but will flash it tomorrow and monitor it during the day. don't have the guts yet to let it running overnight :D

Thanks a lot for all the work, lets make the proxy happen.

[gekkekoe]

already seeing the following: 1) esp32 temp is a around 37c vs 41c (was that value the last 24h) 2) wifi db went from -41 to -33 (awesome!) 3) free heap went from 200k to 250k (awesome!)

[gekkekoe]

I did leave it running overnight and looks good. I think we should resolve the verify checksum properly (now you cannot invoke verify checksum twice, it will fail after the first time. we currently only do it once, but I don't want to rely on this) and then I'm good with merging this as first part, we can do proxy as second pr. That would require some testing from your side since I can only test with esphome as slave (sadly sold the procon).

btw: we also need to adjust the default esp32.yaml to use esp-idf before merging. I think we should move the uart config with pins to the esp32 yamls. This way we can add heishamon board with different pin layout (esp32s3-heishamon.yaml or something).

[dwmw2]

Yeah, I definitely want verify_checksum() to be idempotent. Might use the valid flag. Coffee first...

[gekkekoe]

can you please apply this patch ? I can't push to your pr. tidy-up.txt