DTU flashing

[lordlightman]

Hello @lufton. I would like to thank you for your amazing work integrating the inverter into ESPHome.

I am most interested in flashing the ESPHome firmware on the DTU. I was surprised but also glad to find out that PowMr's WiFi dongle uses an ESP32 chip, since I was planning on buying one on the upcoming AliExpress sale and trying it out. It's great that it can be re-flashed and used to integrate my inverter into HA via ESPHome. But unfortunately, I do not have soldering skills. On the images that you provided I saw some pins on the PCB, would connecting the Dupont wires to them be enough to flash the new firmware? You also mentioned USB to TTL module, I'm guessing that's where I need to connect the wires from the PowMr PCB? Would any USB to TTL module work or I need to buy some specific one? Do you have any recommendations regarding that?

[lufton]

Hello @lordlightman, glad you appreciated my work. There is nothing difficult to flash the DTU. The only pins you need to solder is GND, BOOT, RX, TX and VCC to your PCB close to ESP32 module (on the left on this picture). For that you can use some header pins or just strip of wires. If you are deft enough you can try just inserting wires in the holes and maybe knotting them but you need to make sure everything is sturdy and not touching each other. Then you can hook everything to basically any TTL converter (better the one that operates 3.3V logic, e.g. this one). Pay attention that you should connect RX and TX cris-cross. Also to enter flashing mode you need to "pull down" BOOT pin, that means it need to be connected to GND during module start. So you can wire GND with BOOT before plugging USB connector of TTL convertor into PC. Then plug it in. Wait couple seconds and BOOT can be disconnected from GND, you ESP32 should be in flashing mode. Then just use tool and command that I provided on the main page to flash it.

[lufton]

@lordlightman actually I'm sorry, you don't necessarily need 3.3V module because there should be linear voltage stabilizer that actually reduces voltage from 5 down to 3.3. But the module I suggested also has 5V rail. So it will work too.

[lordlightman]

Thanks for such a detailed reply @lufton. Although, I'm still a bit confused about the soldering part. Did the PowMr board depicted on the images come with pins out of the box or you made some mods of your own to it?

[lufton]

@lordlightman yes, I soldered them on my own. There is really nothing difficult, just grad a decent soldering iron for couple bucks and some Flux-Cored Solder Wire. Then apply a little solder on the iron and on the PCB (don't forget to insert header pins or wire first).

[lordlightman]

Appreciate your help, @lufton. I guess I'll order the DTU, some pin headers, USB to TTL module and a soldering set and try my hand at soldering once it arrives.

The DTU re-flashing solution seems to be the most elegant and does not require external power compared to approaches I've seen so far, like connecting the RS232 to TTL module to inverter via a cable with RJ45 connector, then the module to ESP32 board and providing power to the latter.

[lufton]

@lordlightman you are welcome.

[lordlightman]

Hello @lufton. Got a question for you. Did you have any experience re-flashing non-PowMr DTUs? Like this one for example. Do you think they differ in terms of hardware or do they just have different firmware flashed from the factory?

[lufton]

@lordlightman AFAIK this is the same module. You can find it by the code name WBS1-V001. Should work with this project and inverter.

[lordlightman]

Okay, thanks, @lufton. AFAIU, your custom local firmware does not require any DTU serial number registration, right?

[lufton]

No, it is local integration. No Chinese cloud is involved.

[lordlightman]

Hello @lufton. I finally got everything I need to carry on with the project, and I also had a bit of soldering practice. But before I move on, I wanted to clarify some things first.

The only pins you need to solder is GND, BOOT, RX, TX and VCC to your PCB close to ESP32 module

So GND is GND1, BOOT is marked as BOT1 I guess, RX and TX are RXD1 and TXD1 on right of the board, and VCC is +5V on the left of the board. Correct me if I am wrong.

Then you can hook everything to basically any TTL converter (better the one that operates 3.3V logic, e.g. this one). Pay attention that you should connect RX and TX cris-cross. Also to enter flashing mode you need to "pull down" BOOT pin, that means it need to be connected to GND during module start. So you can wire GND with BOOT before plugging USB connector of TTL convertor into PC. Then plug it in. Wait couple seconds and BOOT can be disconnected from GND, you ESP32 should be in flashing mode.

I purchased the TTL module you mentioned, so if I understood everything correctly I need to connect the wires in the following way:

DTU	TTL module
+5V	5V
RXD1	TXD
TXD1	RXD
BOT1	GND

After I plug the TTL module into my PC, wait and then unplug the GND, do I need to connect GND1 from DTU to GND on the TTL module? Is there any indication on the DTU that I'm in flashing mode?

[lufton]

You are correct. And yes, you need to connect both GNDs (DTU and TTL) together. And for the moment you plug in USB also connect BOT1 pin to GNDs (you can even simply solder a short wire to BOT1 and just touch it to GND before inserting USB and release afterwords). ESP is checking the state of GPIO0 (BOT1 in this case) during boot cycle and if it is LOW (voltage relative to GND is close to 0) then it boots into flashing mode. After booting in flash mode ESP stays in this mode until rebooted. So there is no longer need to keep GNS and BOT1 connected after it’s loaded into flash mode.

[lufton]

Regarding indication: while in a flashing mode, regular program loop is not executed, so as indication there shouldn’t be any LEDs (COM, POW or NET) blinking.

[lordlightman]

Thanks for the info, @lufton.

[lordlightman]

Hello @lufton. I managed to solder the PINs to the board. It was more challenging than I thought since they are so small. So, I connected the wires to the TTL module, BOT1 to GND first and then GND1 to GND. Then I tried running the esptool to make a dump of the original firmware but I get the following error:

esptool.exe -p COM5 -b 460800 read_flash 0x0 0x400000 MPS-VX-DTU.bin
esptool.py v4.7.0
Serial port COM5
Connecting......................................

A fatal error occurred: Failed to connect to Espressif device: No serial data received.
For troubleshooting steps visit: https://docs.espressif.com/projects/esptool/en/latest/troubleshooting.html

or sometimes this one:

esptool.exe -p COM5 -b 460800 read_flash 0x0 0x400000 MPS-VX-DTU.bin
esptool.py v4.7.0
Serial port COM5
Connecting......................................

A fatal error occurred: Failed to connect to Espressif device: Invalid head of packet (0xD8): Possible serial noise or corruption.
For troubleshooting steps visit: https://docs.espressif.com/projects/esptool/en/latest/troubleshooting.html

Do you know why this error occurs?

Also, I wanted to note that when I first connect the TTL module to USB with BOT1 connected to GND the red LED on the TTL is lit up and no LEDs are lit up on the DTU. But when I connect GND1 to GND on the TTL, the blue LED on the TTL flashes occasionally and also the POW1 on the DTU flashes as well (every second or so). I tried connecting +5V to 3.3V on the TTL, but the result is the same.

[lufton]

Did you install drivers for your TTL module? Did you enter correct COM port number?

[lordlightman]

Yeah, I did. Here is what my Device Manager shows when I connect the TTL module.

[lufton]

Can you also send a photo or better a video with your connections and process you’re doing?

[lordlightman]

I might have, but I think something happened to the DTU's PCB cause the POW1 LED is no longer flashing. Maybe I accidentally inserted the 5V wire into the GND on the TTL module, or the DTU was defective from the get-go. Do you know if there is a way to check whether the DTU is still alive and kicking?

[lufton]

If DTU was blinking before and is not now (even after disconnecting and restarting) then yes, probably something is burned. Could you take a photo or video of your connection?

[lordlightman]

Sorry @lufton, my phone's camera is not good at filming small objects. I connected everything as we discussed and made sure that the wires are secure, even checked the continuity with the multimeter, and measured the voltage at the +5V and GND1 pins, so something must be wrong with the DTU's board. I guess I'll order another one, this time directly from PowMr Official Store and give it another go.

[lufton]

Well, if it was blinking before and not blinking anymore then it most likely something is burned. I would say try to check linear voltage regulator U2. If it outputs power. You can check voltage at capacitor C3. Put your multimiter at DC voltage measurement, connect +5 and GND from TTL (no need to connect BOOT, TX or RX for now) and touch probes to capacitor legs like shown on the picture. Your multimeter should read ~3.3v. Also you can try connecting to inverter. Maybe it's your TTL converter failed.

[lordlightman]

Hello @lufton. Yeah, I tried connecting it to the inverter via supplied cable, but I get the same result, the green POW1 LED is no longer blinking. BTW, I measured the voltage between +5V and GND1 when powered from the inverter and it was 11.4V for some reason. Same voltage on U2 voltage regulator. I also measured the voltage between C3 capacitor legs and it read 54mV. So, I guess it's fried.

[lufton]

Inverter indeed outputs ~12v to DTU. Then voltage regulator transforms it to 3.3v to supply to ESP. There is chance you have burned it. It should be pretty straightforward to fix. You just need to purchase and re-solder new IC.

[lordlightman]

Sorry, new IC? What's that? How difficult is it to re-solder it? Is there a chance that some other components have also been damaged?

[lufton]

IC stands for "integrated circuit". In your case it looks like you've burned LP2985IM5X-3.3 swapping GND and 5V on input rail. It has 5 legs and should be pretty easy to resolder by someone who has a bit of expirience with soldering iron or gun. You can try source IC locally, there also should be some replacements available. You can try yourself, just check some video tutorials on how to solder SMD IC. I don't think you can make it worse ) But first of all you need to confirm that regulator is the issue.

1. Connect DTU to Inverter
2. Check voltage between GND and 1 leg of IC (I marked it as 12V). It should be ~12V.
3. Check voltage between GND and 5 leg of IC (I marked it as 3.3V). It should be ~3.3V if regulator is alive. Otherwise reulator is indeed fried.
4. If regulator is dead unplug DTU and connect GND of TTL to GND of DTU and 3.3V of TTL to 5th leg of IC or to positive pad of capacitor C3.
5. Your DTU should go live. Leds should blink. That means everything you need to replace is U2. There are many other voltage regulators that has simular characteristics but different package (shape). If you wouldn't be able to source SOT-23-5 package you can buy any other regulator (even with 3 legs as this PCB only uses IN/OUT/GND pins) that can accept upto 12v (better little higher to be safe) and output 3.3v.

[lordlightman]

Hello @lufton, thanks for the detailed explanation, I may try re-soldering this sometime in the future to practice. As for inverter integration into HA, I feel like I'm better off just ordering a new DTU.

[lordlightman]

Hello @lufton. Hope you are doing well.

Recently I received a new DTU and tried reading the firmware off of it and I'm getting the same error as last time:

Failed to connect to Espressif device: No serial data received.

Once again I connected all the wires as discussed previously, ran the continuity check for TXD and RXD, as well as checked that 5V are delivered to the DTU's board. I've inserted a small piece of wire into BOT1 hole, I'm holding it together with the GND1 while connecting the TTL module to USB. No LEDs are blinking on the board afterwards, so I'm guessing ESP32 is in flash mode. But still I'm getting the error and cannot read the firmware.

Here is what I've tried so far:

• made sure that my USB cable supports data transfer, and indeed it does
• tried a bunch of things described on ESP32 troubleshooting page like lowering the baud rate and specifying the chip explicitly but no avail
• used a different USB to TTL module, got the same result

So, I guess it's a problem with the DTU or entering the flash mode.

• Do you remember what version of drivers and esptool you used during flashing?
• Are you certain that RST1 on the DTU is not supposed to be used?
• Do you think that there may be some anti-tampering mechanism used on the ESP32 to prevent reading the memory and re-flashing it?

[lordlightman]

Also, does it matter if I connect the RX1 and TX1 (close to +5V) to the TTL or the RXD1 and TXD1 (next to the ESP32 chip)?

[lufton]

First of all you should not use TX and RX pins that are close to RJ45 plug. They are RS232 related and have higher voltage (I believe around 12V) so will fry your TTL module, which can handle only 5V logic.

[lufton]

Secondly. With the TTL I have recommended I actually forgot, that (at least my unit) has issue with entering connected device into boot mode using boot pin. So during flashing I manually short (connect) BOOT pin and GNG. You can safely leave it connected during whole flashing process (no need to release). It may help as unit could restart several times during flashing so it could leave boot mode before process completely finish.

[lufton]

Thirdly, there should not be any protection against reading or writing flash. I’m not even sure it is possible with this chip.

[lufton]

Finally, disconnect everything. Do step by step according my guide. Put it on a table. Double check you did everything correctly. Take a photo of your connection with all wires visible. Better even make short video with your attempt. I’m willing to spend my time and help you. But it’s difficult to debug without visualization. Also you are welcome to reach me directly in Telegram.

[lordlightman]

Hello @lufton. So I finally succeeded in re-flashing my DTU. Still a little unsure what made it all work but here is what I did:

• used a different USB to TTL module and its provided cables
• connected BOT1 to GND1 and kept them connected while dumping/flashing

IDK, maybe that other TTL module also got damaged when I misplaced 5V and GND and fried the old DTU or maybe other Dupont cables had poor connection (even though the continuity test was successful). I gotta add that after every operation I had to re-insert the TTL module into USB because it once again responded with Failed to connect to Espressif device: No serial data received. after finishing the operation. Maybe touching the RST1 and GND1 would have yielded the same result, but I didn't want to ruin the setup so I didn't check.

Here is the log if you are interested:

./esptool.exe -p COM4 -b 460800 write_flash 0x0 .\powmr-dtu.factory.bin
esptool.py v4.7.0
Serial port COM4
Connecting....
Detecting chip type... ESP32-C3
Chip is ESP32-C3 (QFN32) (revision v0.4)
Features: WiFi, BLE, Embedded Flash 4MB (XMC)
Crystal is 40MHz
MAC: xx:xx:xx:xx:xx:xx
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00112fff...
Compressed 1123888 bytes to 649562...
Wrote 1123888 bytes (649562 compressed) at 0x00000000 in 19.2 seconds (effective 469.5 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

./esptool.exe -p COM4 -b 460800 flash_id
esptool.py v4.7.0
Serial port COM4
Connecting......................................

A fatal error occurred: Failed to connect to Espressif device: No serial data received.
For troubleshooting steps visit: https://docs.espressif.com/projects/esptool/en/latest/troubleshooting.html

[lordlightman]

However, I am facing a different problem now. After connecting the DTU to the inverter I receive no data from it. LEDs on the DTU are all lit up except the one closest to RJ45 port - it blinks every 5 seconds.

I flashed it with firmware using config obtained from your example config for 24V inverter. The flashing process was successful, here is the log.

There are no readings from the inverter, I also tried setting Backlight to On but the inverter's screen did not light up and this change was not made in the inverter settings.

Do you have any thought as to what may be wrong?

[lufton]

First of all it is a big step. As now you don't need USB-TTL to flash new version. Everything could be done via OTA update. So congrats! Secondly according to logs it looks like module is trying to communicate with inverter but it fails. COM led blinks upon communication with inverter. Why communication is not successful I don't know. Are you sure your ethernet cable is okay? Try using different one. The shorter is better. Also did you try to use DTU with original flash? Did communication worked for you? If you didn't try, you can flash original firmware back and try (instruction and my dump if needed is in README). Other potential issue could be with burned SP3232EEN but I'm really doubt you could manage to burn it. Also you could try to enable debugging, for that use https://github.com/lufton/esphome-inv-8851/blob/main/dtu-wbs1-v001-24v-example-local.yaml or https://github.com/lufton/esphome-inv-8851/blob/main/dtu-wbs1-v001-48v-example-local.yaml respectively and change packages/core-local.yaml file logger and uart sections:

logger:
  level: VERBOSE

uart:
  - id: inv_8851_uart
    baud_rate: 9600
    tx_pin: ${uart_tx_pin}
    rx_pin: ${uart_rx_pin}
    debug:
      direction: BOTH
      dummy_receiver: false

And check for uart output.

[lordlightman]

Thanks @lufton. Yes, as a matter of fact I have already performed some OTA updates wirelessly.

Secondly according to logs it looks like module is trying to communicate with inverter but it fails. COM led blinks upon communication with inverter.

Okay, good to know. I guess the LED blinks according to update_interval of 5 seconds set in core.yaml config.

Why communication is not successful I don't know. Are you sure your ethernet cable is okay? Try using different one. The shorter is better.

No, not really sure but I tried both grey cables that were provided with DTUs. Chances that both of them are faulty are pretty slim. I didn't try any other CAT6 network cables that are lying around my home since I'm not sure that their pinouts are the same - didn't want to risk damaging the inverter's board.

Also did you try to use DTU with original flash? Did communication worked for you? If you didn't try, you can flash original firmware back and try (instruction and my dump if needed is in README).

I may try this if nothing else helps cause I really don't want to create any accounts on Chinese servers and provide them my Wi-Fi credentials.

Other potential issue could be with burned SP3232EEN but I'm really doubt you could manage to burn it.

Sorry, what's SP3232EEN? Is it something on the DTU or the inverter PCB? Is there an easy way to check that it's not burned?

Also you could try to enable debugging, for that use https://github.com/lufton/esphome-inv-8851/blob/main/dtu-wbs1-v001-24v-example-local.yaml or https://github.com/lufton/esphome-inv-8851/blob/main/dtu-wbs1-v001-48v-example-local.yaml respectively and change packages/core-local.yaml file logger and uart sections:

So I made these changes and installed the OTA update, tried setting the Backlight to On again. Here is the log output.

Do you have any ideas how to check if this is an issue with DTU or the inverter communication board?

[lufton]

Hm, this is interesting. I'm curious why there are json decoding envolved. Let me flash my DTU with config I suggested to you. I'll come back later.

[lordlightman]

@lufton, I think it has something to do with the web_server component that I was running. Once I disabled it, messages like Attempting to allocate 512 bytes for JSON serialization were gone.

Anyway, I've also collected the logs after disabling web_server and enabling VERY_VERBOSE log level. Perhaps they can be of use to you.

[lordlightman]

Hello @lufton, how are you?

Did you attempt to flash your DTU?

[lufton]

Hello, @lordlightman, yes I just uploaded configuration to mine and realize I forgot to include several parameters for uart component. Without it it doesn't log messages. So please update your uart section to:

uart:
  - id: inv_8851_uart
    baud_rate: 9600
    tx_pin: ${uart_tx_pin}
    rx_pin: ${uart_rx_pin}
    debug:
      direction: BOTH
      dummy_receiver: false
      after:
        delimiter: "\n"
      sequence:
        - lambda: UARTDebug::log_hex(direction, bytes, ' ');

You should be able to see communication between inverter and DTU. Something like this: Whereas >>> means DTU sending command to inverter and <<< inverter is sending response. Communication protocol packets are always started with 88 51 sequence, thats why this project is called esphome-inv-8851. So if you see anything else it could be a noise and it is filtered out by packet length and CRC validation.

[lordlightman]

I updated the uart config according to your snippet and here are the logs that I got:

logs_powmr-dtu_run.txt

Looks like the inverter is not sending any responses. Do you know why might that be?

[lufton]

I would flash original firmware and check if inverter could communicate with DTU. If you care about your credentials, then just temperarly change wifi password or create second ssid (some routers allow that).

[lordlightman]

Okay, I'll consider it. In the meantime, can you please answer the following:

• Did you attempt to use any Ethernet cables other than the one provided with the DTU? Do they have the same pinouts?
• Previously you mentioned the SP3232EEN. What's that? Is there an easy way to check that it's not burned?
• Do you think that Ethernet to TTL converter connected to ESP32 dev board could help clarify where the problem is?

[lordlightman]

Hello @lufton, can you please provide some answers to questions above if you've got the time.

[lufton]

1. I used only provided one, but I checked and pinout is standard, only make sure it is not a crossover cable (color scheme on 1 side should exactly match color scheme on other side).
2. It is the IC that responsible of decoding RS232 signals to UART. I have no idea how to check if it not burned other than with special equipment or visual examination for burned spot.
3. It has no benefits over USB TTL.

[cristianoggi]

Hello everyone. I find myself in the same situation as @lordlightman.

uart:

• id: inv_8851_uart baud_rate: 9600 tx_pin: ${uart_tx_pin} rx_pin: ${uart_rx_pin} debug: direction: BOTH dummy_receiver: false after: delimiter: "\n" sequence:
  • lambda: UARTDebug::log_hex(direction, bytes, ' ');

how do I edit it in my yaml?

[lufton]

@cristianoggi you need to install python, pip and esphome module locally on you PC, clone this repo, edit packages/core-local.yaml and use [dtu-wbs1-v001-24v-example-local.yaml](https://github.com/lufton/esphome-inv-8851/blob/main/dtu-wbs1-v001-24v-example-local.yaml) and run something like:

esphome run dtu-wbs1-v001-24v-example-local.yaml

inside repo folder.

[cristianoggi]

I made the change. I didn't understand how and where to run the file

[lufton]

First of all install latest python for your platform from here. Then made changes to packages/core-local.yaml Then open console/terminal/bash etc and run line by line:

cd /folder/where/you/cloned/repo
pip3 install esphome
esphome run dtu-wbs1-v001-24v-example-local.yaml

Select corresponding flashing method (you can try OTA if you didn't change standard name in configuration or use USB-TTL COM port otherwise).