Some questions regarding Sensorbox 2

arpiecodes commented 2 years ago

Hello,

Some time ago I've installed the latest ESP32 version with WiFi enabled on my sensorbox v2. This worked for some time (could access web interface/WiFi connected) and the SmartEVSE communicated with the Sensorbox for P1 measurements, but after a while the SmartEVSE started to show 'NO COMM' error on its display. The Sensorbox ESP32 also seemed to have disappeared from my WiFI. After some further investigation, it seems the ESP32 does not want to boot anymore (serial output shows some CRC error on blocks from SPI flash). I guess the internal storage got corrupted.

I tried to create my own custom ESPHome-based firmware to at least take over the P1 communication part/integrate it better with home assistant, but this had the same issue. The ESP32 actually flashes a new firmware image without problems, and then boots into it once after the esptool resets it. But after that, any subsequent reboots it gets stuck in the CRC error boot loop again.

Now my questions;

is this particular ESP32 bricked and do I have to replace it? Or is there some special boot enablement code I have not found yet due to the way pins are pulled down/up during boot?
I see in documentation the pic is (re)programmed every boot. Isn't this wearing the PIC's programming flash? Or does it only flash if it sees the firmware is old? Does it actually function without this program on-boot or do I have to port this part to ESPHome as well?

My general goal would be to make a minimally viable version based on ESPHome that can communicate through modbus, work the P1 sensor data from DSRM meter and also read the CT clamps data. Then expose it through MQTT/Home Assistant API easily.

Hope you can help me with above.

Thanks!

Jorrit

arpiecodes commented 2 years ago

Nevermind, I actually got the ESP32 to work again. I was using some modbus library that was probably interfering with the chip's initialisation.

I'll see what I can get working in ESPHome.

mstegen commented 2 years ago

Hi Synergic, Glad you got it to work again.

ref point 2. The PIC is only reprogrammed when it finds a .hex file on the file system of the ESP32. After programming the .hex file will be deleted, so it will only program it once. The first version of the Sensorbox which used a PIC18F26K40, had everything in the PIC chip, the ESP32 was not doing anything other then blinking the led's. So it is essential that you reprogram it otherwise it will interfere with the RS485 communication. -Michael

arpiecodes commented 2 years ago

Hi @mstegen,

Thanks for your reply!

It seems the PIC is not doing anything on my Sensorbox. No serial communication seems to happen when I debug the incoming data on pin 23 (UART RX).

When I use the code inside the original Sensorbox firmware it detects an PIC16F1704. I do not see the bundled .hex file for flashing this chip, though.

How can I verify the PIC is actually doing it's job, or instruct it to start sending its data through UART so I can consume it on ESPHome?

Thanks in advance!

mstegen commented 2 years ago

Ah, then you have the new hardware with the PIC16F1704. Here, the PIC sends it's measurement data through it's UART to the ESP32. The ESP32 does the final calculation for the CT measurements. And handles the RS485 communication.

You should be able to see the data coming in, every second or so. The PGC pin is used to set the 3 wire/4 wire or phase rotation on the PIC. Make sure that the PIC is not held in Reset.

arpiecodes commented 2 years ago

Thanks again. Just tried changing the code a bit.

I took the SetupPIC contents and traced the pinmode toggles from the main.cpp file on the original firmware. Put those into a custom component for ESPHome, which is initialised on ESP boot (also included the original header definitions for below code);

    void setup() override {
        RESET_OUTPUT;
        DAT_INPUT;
        CLK_INPUT;
        RESET_HIGH;

    pinMode(PIN_PGD, INPUT);
        pinMode(PIN_PGC, OUTPUT);

        ESP_LOGD("custom", "PIC has been enabled");
    }

But still no UART communication received from the PIC on pin 23;

uart:
  - id: uart_pic
    baud_rate: 115200
    tx_pin: 1
    rx_pin: 23
    debug:
      direction: RX
      dummy_receiver: false
      after:
        delimiter: "\n"
      sequence:
        - lambda: UARTDebug::log_string(direction, bytes);

Am I still missing something? Does the PIC need some time to boot? I do see a while loop that waits for Serial.begin() inside the main.cpp file.

arpiecodes commented 2 years ago

Sorry, I totally overlooked the fact that the PIC is terminating its serial communication with something else than a newline char. I got it to work now. Thanks a lot for your patience and help.

I will continue to try to port the full sensorbox code to ESPHome. I guess it's still a problem that the SmartEVSE is the master and not the sensorbox with regards to full control over the SmartEVSE's from control point of view. But it is definitely workable if you would regard them as separate entities, and having a way to control SmartEVSE's separately through MQTT.

My goal of the ESPHome based project is to utilise the Sensorbox for P1 readings (while also publishing the full DSMR data to my Home Assistant for energy monitoring purposes, just like the Slimmelezer project by Marcel Zuidwijk using their amazing dsmr ESPHome component https://esphome.io/components/sensor/dsmr.html?highlight=dsmr) but at the same time also use the CT clamp readings for monitoring my solar energy production separately. Then, use the gathered data to expose through modbus to the SmartEVSE master so I can remove the MQTT/Home Assistant/WiFi dependency for load balancing.

I'll keep you updated on my progress. Shouldn't be that hard to get something functional from here on forward.

I must say, the Sensorbox is a very nice piece of hardware. Even for use-cases without a SmartEVSE. The combination of ESP32, PIC for energy measurements with the CT's and the modbus interface make it very versatile.

mstegen commented 2 years ago

Glad you got it to work!

You might have seen that the PIC data changes, depending if the mains cable is connected. When it's unconnected it will only do normal (rms) current measurements. When it's connected it will also take phase angle of each measurement, and thus can determine the direction of the current.

You can also try running it stand alone, it can also be powered from the P1 ports power pin. Although i have not tried this yet. It might not work, as the P1 port can only provide 500mA @ 5V.

Yes please keep me updated on your progress.

arpiecodes commented 2 years ago

I got it to work quite nicely. I've packaged all basic Sensorbox functionality (borrowed some code from original Sensorbox code, hope that's OK) inside a custom ESPHome component. Needed to do some tinkering to get the modbus communication to work as the modbus_server library I found for esphome did not support toggling DE and RE pins nor responding to input register reads so I had to add that, but all worked out eventually. Interestingly enough the UART debug function of ESPHome prevented proper modbus communication, so that took some time for me to realise that.

The component provides the following functionality;

Sends current and voltage information to SmartEVSE master through modbus
Both P1 and CT readings are supported through simple configuration
Possibility to adjust power rotation and 3/4 wire configurations
Basic protection against loss of CT/P1 data (internal timer unsets dataready telegram if no fresh data)
Allows to use CT readings without actually using the CT readings for SmartEVSE (for example; solar production)
Allows to log/consume a whole array of extra P1 measurement data with the DSMR ESPHome component

Basically if one already uses Home Assistant and ESPHome, it can easily be flashed and used as a drop-in replacement for the original Sensorbox firmware without losing its core functionality of feeding P1/CT power data to SmartEVSE and without compromising measurements when/if WiFi, network or (for example) Home Assistant goes down.

Some further ideas come to mind to expand on this. For example, using the CT clamps to also be able to sense the EV power consumption next to P1 readings through the same Sensorbox. Or better solar balancing by also being able to monitor the exact amount of solar production. Last but not least allowing some sort of setting change from Sensorbox to SmartEVSE, like a global 'access' switch (disabling or enabling ALL hooked up SmartEVSE's), changing SmartEVSE global mode, etc. May also be nice to retrieve status from SmartEVSE back as well and publish it through the Sensorbox as a WiFi/MQTT/HA gateway, reducing the need to network the SmartEVSE master itself.

A lot of ideas, but this would also require additions/changes to the SmartEVSE firmware. So less trivial.

Let me know if you want to make the component semi-official or if you want me to prepare the code so you can organise it under the SmartEVSE Github. In the meantime I will be cleaning up the code and opening a pull request with the author of the modbus_server component so my changes can be merged upstream as well.

https://github.com/synegic/esphome-smartevse-sensorbox

Mawverick commented 1 year ago

Another question regarding Sensorbox 2.0. Does the sensorbox 2.0 built based on PIC16F1704 can work it stand alone without ESP same as it is with older PIC18F26K40.

mstegen commented 1 year ago

No the PIC16F in the Sensorbox 2 can not work stand alone, the ESP32 is used for the communication and calculations. A stand alone version is the Sensorbox plus, but it can only do CT measurements.

Mawverick commented 1 year ago

CT measurements handle by PIC16F would be sueficent feature. Determine of current direction is another benefit . Does the code for PIC16F stand alone version also avaliable for communy?

arpiecodes commented 1 year ago

CT measurements handle by PIC16F would be sueficent feature. Determine of current direction is another benefit . Does the code for PIC16F stand alone version also avaliable for communy?

Could you explain a bit more what your goal is to do? If you put code on the ESP32 in the Sensorbox, it can communicate with the PIC through serial port and retrieve those measurements.

SmartEVSE / Sensorbox-2

Some questions regarding Sensorbox 2 #7