search

mp-se / victron-receiver

Victron Instant Readout receiver for Home Assistant
MIT License
3 stars 0 forks source link

Failed flash #1

Open thomasjobs opened 4 days ago

thomasjobs commented 4 days ago

Hi,

As I have a number of Victron devices in my motorhome and this project will be most useful to integrate Victron data into my HA.

Unfortunately I'm inexperienced user of Arduino and ESP32. I have problem after flashing of the Waveshare ESP32-S3-Touch-LCD-2. I used esptool and the binaries on GitHub - after flashing the board does not transmit the SSID.

I would be grateful for any advice pointing me in the correct direction.

Thanks in advance!

Brgds Thomas

esptool.js Serial port WebSerial VendorID 0x303a ProductID 0x1001 Connecting...Debug: _connect_attempt default_reset false Debug: Timeout Debug: Sync Debug: Sync err Error: Timeout .Debug: Sync Debug: 538052359

Debug: Chip Magic 9 Detecting chip type... ESP32-S3 Chip is ESP32-S3 Features: Wi-Fi,BLE,Embedded PSRAM 8MB (AP_3v3) Crystal is 40MHz MAC: d8:3b:da:9b:6d:cc Debug: In _post_connect 4 Uploading stub... Debug: mem_begin 5076 1 6144 40378000 Debug: check_command enter RAM download mode Debug: check_command write to target RAM Debug: mem_begin 252 1 6144 3fcb2bf4 Debug: check_command enter RAM download mode Debug: check_command write to target RAM Running stub... Debug: check_command leave RAM download mode Stub running... Changing baudrate to 921600 Debug: 0 Changed Debug: Sync

mp-se commented 4 days ago

Hi, no problem. I have not done a complete guide for how to do flashing. I will take a look at it and write up some instructions and make sure all the needed files is added to github. I can copy most of that from some other project I have been working on.

thomasjobs commented 4 days ago

Hi,

Thanks! Appreciate it.

Brgds Thomas

mp-se commented 4 days ago

What devices do you have ? I can see if I can add support for them ?

thomasjobs commented 4 days ago

GlobalLink 520 Smart Shunt SmartSolar MPPT 100/20 SmartBattery Sense Smart DC/DC charger 30A Smart IP22 charger 12/30(3)

thomasjobs commented 4 days ago

Fantastic if you could add support for the devices :-)

Thanks!

thomasjobs commented 4 days ago

Do you need any details or data from the devices?

mp-se commented 4 days ago

There is another project that is parsing most of those devices, so I will see what I can gather from those first. I need someone to test them. I have not really sorted out the AC charger yet since the specification does not fully match the data being sent and I have not found any other project handling that either... But lets start with fixing flashing.

I created a simple python script that will handle the devices I have build targets for (but I have only tested the waveshare option). It assumes that the flash files are in a bin directory....

First you will need the firmware32s3w.bin + partition32s3w.bin from the github repo.

the full commandline for the waveshare target should be ....

python3 -m esptool --port (your port) --chip esp32s3 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size 16MB 0x0000 bin/bootloader32s3w.bin 0x8000 bin/partitions32s3w.bin 0xe000 bin/boot32s3w.bin 0x10000 bin/firmware32s3w.bin

Files are gzipped since I cannot update bin/py here.

flash.py.gz

you will also need these files

boot32s3w.bin.gz bootloader32s3w.bin.gz

thomasjobs commented 4 days ago

Ok, thanks. Will try to follow instructions…….

Brgds Thomas

mp-se commented 4 days ago

It would be good to get some test data so I can test some of the features at least. If you add the encryption keys for your devices then it should show up as an unknown device. In the current version the copy button does not really work so a workaround is to get the data direct from the device.

Let me know if you get the flashing to work or we can try another approach...

Once you have seen that it has detected and decoded some of your devices navigate to the following url:

http://your_device_ip/api/status

that should give you a document that looks like below: copy the data below victron_device in this thread, that is the data I need to do some testing.. This is from one of my smart battery monitors. 


{
    "id": "c99910",
    "temp_format": "C",
    "app_ver": "0.1.2",
    "app_build": "..db660c",
    "mdns": "victronc99910",
    "platform": "esp32s3",
    "rssi": -61,
    "wifi_ssid": "",
    "total_heap": 324196,
    "free_heap": 173724,
    "ip": "192.168.1.188",
    "wifi_setup": false,
    "uptime_seconds": 59,
    "uptime_minutes": 2,
    "uptime_hours": 0,
    "uptime_days": 0,
    "victron_device": [
        {
            "name": "LiBatt",
            "data": "{\"name\":\"Smart Battery Monitor\",\"model\":\"0xA3A5\",\"decrypted_data\":\"0xFF,0xFF,0xFD,0x03,0x00,0x00,0x00,0x73,0xFE,0xFF,0x7F,0xFF,0xFF,0xFF,0xFF,0x3F,0xF6,0x94,0x00,0x00,0x00\",\"battery_voltage\":10.21,\"temperature\":21.25}",
            "mac": "D1:F4:EF:4D:19:AD",
            "update_time": 3,
            "push_time": 8
        }
    ]
}```
thomasjobs commented 4 days ago

I'm using Arduino IDE for flashing - correct? Get an error....

I'm banging on my limit of knowledge.....but learning along the way......:-)

C:\Users\thoma\AppData\Local\Temp.arduinoIDE-unsaved20241012-9832-r3gjy9.e086\sketch_nov12b\sketch_nov12b.ino:1:1: error: 'python3' does not name a type 1 | python3 -m esptool --port com3 --chip esp32s3 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size 16MB 0x0000 bin/bootloader32s3w.bin 0x8000 bin/partitions32s3w.bin 0xe000 bin/boot32s3w.bin 0x10000 bin/firmware32s3w.bin | ^~~

exit status 1

Compilation error: 'python3' does not name a type

mp-se commented 4 days ago

You dont need arduino ide for flashing, just python with esptool installed.

Install python3 (download from python.org) run> pip install esptool run> python3 flash.py esp32s3w <your serial port, depends on your OS, com3 as above>

thomasjobs commented 4 days ago

Ok, I try……..good practice this exercise

I think I have to buy you a beer when passing Helsingborg

thomasjobs commented 3 days ago

Hi again,

I have no success on flashing the ESP32 module, have tried two methods :-(

Python is installed but not possible to install esptool (tried both within py and command line). I get this error:

pip install esptool File "", line 1 pip install esptool ^^^^^^^ SyntaxError: invalid syntax

Have also tried Flash Download Tool https://www.espressif.com/en/support/download/other-tools and used your comandline to setup the flash.

Module will not start.

Brgds Thomas

mp-se commented 3 days ago

Did you att pyton the path? What happends when you run python or pyton3?

thomasjobs commented 3 days ago

Found an article discussing the path issue. Removed and reinstalled python - and this time, during install, I included path. After restart of PC esptool installation worked. Have upgraded pip after installation.

Trying to figure out where to put files when using command: run> python3 flash.py esp32s3w <your serial port, depends on your OS, com3

mp-se commented 3 days ago

Just create an empty directory and place the flash.py in the root. Create a directory called bin and place all the bin files there.

You can also download the whole git repository as zip file and unpack that (or clone the git repository) then just run the flash.py in the root.

thomasjobs commented 3 days ago

SUCESS! I managed to flash the board and it's now up and running. Will play around to learn. Tomorrow or Friday I will bring the board to the motorhome and collect the data you requested from Victron devices.

Magnus, many thanks for your assistance!! I learned a lot.

Brgds Thomas

mp-se commented 3 days ago

Great, when you collect the data please note down what you see in the app as well (or make a screenshot) so i can validate the result. I found some data for similar devices that I'm trying out, i will let you know if I have some success...

thomasjobs commented 3 days ago

Ok, will do.

mp-se commented 3 days ago

You can upgrade to this version here, https://github.com/mp-se/victron-receiver/blob/master/bin/firmware32s3w.bin

It should report as v0.1.6

It should support the AC Charger, Shunt and Solar, DCDC and BatteryMon. I will look for the global link as well..

I've also added a meny option where you can see all the detected data and values. Please see if there are something that is wrong.

thomasjobs commented 3 days ago

Brilliant! Will install tonight and check Friday........

Brgds Thomas

thomasjobs commented 2 days ago

Hi,

Managed to setup the MQTT and now receiving data from GlobalLink 520 to HA. Tomorrow I will test in the motorhome and post more info.

Brgds Thomas

image

thomasjobs commented 2 days ago

Hi,

I have connected the Waveshare board to the Victron devices in the motorhome. Should I post the files and pictures here or send to an email?

A few comments from installation.

Brgds

mp-se commented 2 days ago

You can post the pictures here, just remove anything that is sensitive (you can crop the pictures to show the valid data). Smart charger should support instant readout I had to install the beta app in order to upgrade to a version that supports instant readout, see https://www.victronenergy.com/live/victronconnect:beta

thomasjobs commented 1 day ago

Ok, here the files..... api_status.txt serial console 14nov2024_jobsvictronsniffer.txt Device settings Integration HA

mp-se commented 1 day ago

I think you missunderstod me, I need screenshots from the victron app to compare the values displayed there compared to what my software reports. But if you can validate that they are correct than thats also fine.

It looks like most of the data is correct (or they seam to be resonable). The one I had issues with is the SOC value from the shunt that i'm not sure about.

I can see that you dont have any load on the devices so its hard to validate some of the values. Would be good if you can dump the data under some load to make sure everything is correct.

But it looks like its mostly correct, so I thank you for the validation. Let me know if there is something wrong and I will correct it.

mp-se commented 1 day ago

I will take a detailed look tomorrow and do an update of the software and create a new release. I will also add some more data so I can do some better analysis the next time.

thomasjobs commented 1 day ago

Aha…..ok, will connect the battery to the system to get load on shunt. And screen shoots from the app.

mp-se commented 1 day ago

There is a new version called 0.1.7 that include some more data of the ble packets that could help decoded the values. Same link as above.

thomasjobs commented 1 day ago

Updated with new version and generated new files and screen shoots. Only strange values are SmartShunt SoC and "battery_current" at 14:46. For info, battery is a new install and have approx 50% charge. I have manually set the charge state. VictronReader_data15nov2024.zip

mp-se commented 22 hours ago

i'm sorry but I need you to update to the 0.1.7 version and get the API data from that version. The data from the older versions dont contain the raw data from the bluetooth packats for me to test any changes. It should be this version; https://github.com/mp-se/victron-receiver/blob/master/bin/firmware32s3w.bin

The status response should look like this; I need the decrypted data part to do changes and validate the results.

{ "name": "Unknown", "model": "0xA339", "decrypted_data": "0x06,0x00,0x2C,0x05,0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0x00,0x00,0x00,0x86,0xCA,0x00,0x00,0x00", "vendor_id": "0x02E1", "beacon_type": "0x10", "victron_record_type": "0x08" }

mp-se commented 22 hours ago

I think the soc and consumed ah are probably the values that i have not got correct for the shunt. The specification and other examples are a little strange. Appreciate if you can write these down once you have a dump.

thomasjobs commented 11 hours ago

I did flash yesterday but I must have used the wrong file……….baaah…….will do again. I’m not sure I have time to do a flash today. Will try….

mp-se commented 11 hours ago

Perfect, check that it reports the correct version, so i haven't screwed up in the build

thomasjobs commented 11 hours ago

Ok, will do…….

thomasjobs commented 11 hours ago

I’m using my Nabu casa link for the MQTT server. Is that possible?

When I listen in HA (#) I only get 3 packets - from global link. When enabling MQTT for a Shelly local device packets are revived as expected.

Not sure if my setup is wrong or a problem with Victron Recevier.

IMG_0736

mp-se commented 10 hours ago

I assume your devices are not on the same network ? So we are talking about remote access to your mqtt server that is locally at your home?

I guess you have your mqtt server published over internet so it can be reached ? Not sure if Nabu Casa link will expose that (not using that myself). How do you do authentication ? Username / Password or Certificate ?

Can you see any error messages in the serial console that can give any hints on what the problem is ?

As long as you can reach the mqtt server and login it should work fine. There is currently no support for certificate based authentication but user/password authentication via SSL is possible (automatically activated for ports over 8000) when using MQTT.

mp-se commented 9 hours ago

here is a list of mqtt error codes and what they mean;

-1 - Connection refused -2 - Send header failed -3 - Send payload failed -4 - Not connected -5 - Connection lost -6 - No stream -7 - No HTTP server -8 - Too little RAM available -9 - Error encoding -10 - Error writing to stream -11 - Read timeout

thomasjobs commented 9 hours ago

In the attached txt file from serial console there is a lot of error messages concerning MQTT. All have code -3 Send payload fail. I assume this means my Nabu Casa server is not working?

It's a bit strange as I receive 3 packages from Globallink when listening (#) in HA. Ref picture.

When checking HA logs no error messages related to MQTT.

serial console.txt

HA_listen to MQTT packages

mp-se commented 8 hours ago

There is very little information on the topic. Have not found anything that supports access to mqtt via nabu casa. There is a ha api that might be used or it require that the mqtt server is accessible via internet

mp-se commented 8 hours ago

It should not be to hard to support the ha rest api as well as mqtt. That should work with nabucasa. Just need to figure out the payload to send, i have most of the code in other projects already

thomasjobs commented 4 hours ago

I’m still puzzled why I (in HA) receive 3 packets from global link. Could globallink interfere with the MQTT setup? Will check tomorrow by disconnecting globallink.

mp-se commented 4 hours ago

globallink is probably using the REST API and not mqtt, thats why it works

thomasjobs commented 4 hours ago

Hmmm.... I Goggled REST API.......this is way over my skill level :-)

mp-se commented 3 hours ago

Think of its as different ways to get to the target, car or train. The destination is the same.... Just technology mumbo-jumbo ;-)

mp-se commented 2 hours ago

You can try this version; https://github.com/mp-se/victron-receiver/blob/master/bin/firmware32s3w.bin I have added rest api support and it works locally. It should work with nabu casa as well but I dont have that option.

You need to disable the MQTT (remove the server name) and add the settings under the REST API page instead. The token you get from home assistant and your user page (security tab). It should be a string of 200+ chars. It should report as 0.1.8.

This will be much slower to update sensors compared to mqtt, so I will see if I can remove those that does not contain any valid data.. but thats a later topic. first i want to get the data decoding correct.

thomasjobs commented 2 hours ago

Ok, I try tomorrow