search

wcbonner / VictronBTLELogger

Log Victron Direct Bluetooth LE messages and graph battery states
MIT License
2 stars 0 forks source link

help needed - analysis of Battery Monitor raw data #1

Open schonstudent opened 1 week ago

schonstudent commented 1 week ago

Hi WCBonner, you might can help me. My BLE raw data is 0E12740401000000AFF5FF2B0170F2.

The data TTG, Battery Voltage, Alarm Reason looks all good to interpret but I struggle with "Battery Current" which are 22 Bits long. Can you analyse the data I provided above to see if I get wrong data from the BLE analysis - I doubt that.

Thanks a lot Best Sascha

wcbonner commented 1 week ago

Hi WCBonner, you might can help me. My BLE raw data is 0E12740401000000AFF5FF2B0170F2.

The data TTG, Battery Voltage, Alarm Reason looks all good to interpret but I struggle with "Battery Current" which are 22 Bits long. Can you analyse the data I provided above to see if I get wrong data from the BLE analysis - I doubt that.

Since I don't have the device, I had not prebuilt support into my app. I just looked at adding the structure for the Battery monitor into my code based on https://github.com/wcbonner/VictronBTLELogger?tab=readme-ov-file#battery-monitor-0x02 and came up with this:

struct __attribute__((packed))
{
  unsigned int ttg : 16;
  unsigned int battery_voltage : 16;
  unsigned int alarm_reason : 16;
  unsigned int three_values : 16; // Aux Voltage, Mid Voltage, Temperature. Don't know how this is used. Temperature coud be 7 bits, but the other two are 16 bits in other devices.
  unsigned int aux_input : 2;
  unsigned int battery_current : 22;
  unsigned int consumed_ah : 20;
  unsigned int soc : 10;
} BatteryMonitor; // 0x02

I absolutely hate that victron did this bit packing, but it's what we live with. Your data of 0E12740401000000AFF5FF2B0170F2 would fit so that the first sections line up easily, but after the 2 bit value, doing things by hand is a mess.

ttg 0E12 battery_voltage 7404 alarm_reason 0100 three_values 0000 the rest AFF5FF2B0170F2

Using windows calculator, I put AFF5FF2B in as hex, and can see that the first two bits in binary are 10. which may be the aux_input value. (Binary: 1010 1111 1111 0101 1111 1111 0010 1011)

The next 22 bits (Binary: 10 1111 1111 0101 1111 1111) should be the battery current. (0x2F F5FF) Decimal: 3,143,167 which doesn't make any sense so I'm sure I'm running into LSB MSB issues with my manual translation.

What really needs to be done is to convert the entire string of HEX digits to binary, slice it up into the values you care about, then convert it back to human readable numbers.

@schonstudent: I'm assuming that the data you supplied for this example was after you decoded it from the original encrypted format?

schonstudent commented 1 week ago

Thanks for the quick answer - just to be on the same page - the data is the decrypted data starting with Bit32 as described in your link. I did the same for Solarcharger 0x01 and DC/DC Converter 0x04. This value is for my almost full charged battery -check the screenshot FFFF9D0500000000030000000080FE

2024-11-13_21-22-45

Save this code in a html page and you can analyse the hex value as raw data with and without little endian.

<!DOCTYPE html>

Hexadecimal Bit Interpreter

Hexadecimal Bit Interpreter

Field Start Bit Bit Value (Binary) Little Endian (Binary) Decimal Value from Little Endian Decimal Value without Little Endian
TTG 0
Battery voltage 16
Alarm reason 32
Aux voltage Mid voltage Temperature 48
Aux input 64
Battery current 66
Consumed Ah 88
SOC 108
Unused 118
wcbonner commented 1 week ago

@schonstudent I just changed the structure to make more sense. The victron documentation is almost as bad as the structure itself. 

struct __attribute__((packed))
{
  unsigned int ttg : 16;
  unsigned int battery_voltage : 16;
  unsigned int alarm_reason : 16;
  unsigned int aux_voltage : 16;
  unsigned int mid_voltage : 16;
  unsigned int temperature : 16;
  unsigned int aux_input : 2;
  unsigned int battery_current : 22;
  unsigned int consumed_ah : 20;
  unsigned int soc : 10;
  //unsigned int unused : 10;
} BatteryMonitor; // 0x02

Now your data of 0E12740401000000AFF5FF2B0170F2 lines up as follows:

ttg 0E12 battery_voltage 7404 alarm_reason 0100 aux_voltage 0000 mid_voltage AFF5 temperature: FF2B But that only leaves 24 bits in your data stream: 01 70 F2

There should be 30 more bits. The offset Victron documentation gives seems strange to me. Can you supply a copy of the entire ManufacturerData string after you've decoded it that would include the first 8 bytes that specify the device?

schonstudent commented 1 week ago

I read all the data via ble (esp32) and i only post the "relevant" data

0x01 -> hex 0300110500000300000000FE 0x04 -> hex 0000DB04FF7F81000000

There the information looks as expected but there we dont truggle with the 2,10,20 or 22 bit long data

This you can ignore

unsigned int aux_voltage : 16; unsigned int mid_voltage : 16; unsigned int temperature : 16;

the smart shunt has another input which you can use as temperatur, mid voltage (in case of 24V system) or aux for e.g. to add the starter batterie.

wcbonner commented 1 week ago

I read all the data via ble (esp32) and i only post the "relevant" data

0x01 -> hex 0300110500000300000000FE 0x04 -> hex 0000DB04FF7F81000000

There the information looks as expected but there we dont truggle with the 2,10,20 or 22 bit long data

This you can ignore

unsigned int aux_voltage : 16; unsigned int mid_voltage : 16; unsigned int temperature : 16;

the smart shunt has another input which you can use as temperatur, mid voltage (in case of 24V system) or aux for e.g. to add the starter batterie.

@schonstudent I don't understand what you say above. I just now normalized my top level ReadMe documentation to have the start bits on manufacturer data all start with 0 so it makes sense as opposed to the way I'd originally copied from the Victron PDF file. Here's the relevant portions when decoding Bluetooth manufacturer data:

Manufacturer Data

Start Byte Byte Count Meaning Remark
0 1 Manufacturer Data Record type 0x10=Product Advertisement
1 2 model id
3 1 read out type 0xA0
4 1 record type Used to decide which bit packed structure to decode the extra data. e.g. 0x01=Solar Charger 0x05=SmartLithium
5 2 AES Initialization Vector two bytes, pad with 14 more 0x00 values to have a 16 byte array. sometimes called "nonce"
7 1 First Byte of AES Decryption Key If this doesn't match the stored key, skip attempting to decode
8 ? First Byte of encrypted data Extra Manufacturer Data

Battery Monitor (0x02)

Start Bit Nr of Bits Meaning Units Range NA Value Remark
0 16 TTG 1min 0 .. 45.5 days 0xFFFF VE_REG_TTG
16 16 Battery voltage 0.01V -327.68..327.66 V 0x7FFF VE_REG_DC_CHANNEL1_VOLTAGE
32 16 Alarm reason 0..0xFFFF VE_REG_ALARM_REASON
48 16 Aux voltage 0.01V -327.68..327.64 V VE_REG_DC_CHANNEL2_VOLTAGE
64 16 Mid voltage 0.01V 0..655.34 V VE_REG_BATTERY_MID_POINT_VOLTAGE
80 16 Temperature 0.01K 0..655.34 K VE_REG_BAT_TEMPERATURE
96 2 Aux input 0..3 0x3 VE_REG_BMV_AUX_INPUT 0 ⇒ Aux voltage : VE_REG_DC_CHANNEL2_VOLTAGE 1 ⇒ Mid voltage : VE_REG_BATTERY_MID_POINT_VOLTAGE 2 ⇒ Temperature : VE_REG_BAT_TEMPERATURE 3 ⇒ none
98 22 Battery current 0.001A -4194..4194 A 0x3FFFFF VE_REG_DC_CHANNEL1_CURRENT_MA
120 20 Consumed Ah 0.1Ah -104,857..0 Ah 0xFFFFF VE_REG_CAH Consumed Ah = -Record value
140 10 SOC 0.1% 0..100.0% 0x3FF VE_REG_SOC
150 10 Unused

If you can show my your entire manufacturer data string, both before and after you've done the AES decryption on it, and how it breaks into the initial structure values, it would simplify my understanding.

It should line up something like: Meaning Value
Manufacturer Data Record type 0x10
model id 0x????
read out type 0xA0
record type 0x02 = Battery Monitor
AES Initialization Vector 0x????
First Byte of AES Decryption Key 0x??
TTG 0x????
Battery voltage 0x????
Alarm reason 0x????
Aux voltage 0x????
Mid voltage 0x????
Temperature 0x????
Bits packed from here 2+22+20+10=54 0x???? ???? ???? ??
Aux input
Battery current
Consumed Ah
SOC
schonstudent commented 1 week ago

Hi again, sorry if I wasn't clear - lets have a look on some raw data 02e1100289a302f5ff17ffff6b0500000000030000000080fe433f

wcbonner commented 1 week ago

Hi again, sorry if I wasn't clear - lets have a look on some raw data 02e1100289a302f5ff17ffff6b0500000000030000000080fe433f

I finally understand, looking back at my old Bluetooth code that used the HCI interface. BlueZ separates the first two bytes into ManufacturerID and calls the rest ManufacturerData. 02e1 is the registered number for 'Victron Energy BV' in the well known numbers.

Bits Meaning Value
16 ManufacturerID 0x02e1 = 'Victron Energy BV'
8 Manufacturer Data Record type 0x10
16 model id 0x0289
8 read out type 0xa3
8 record type 0x02 = Battery Monitor
16 AES Initialization Vector 0xf5ff
8 First Byte of AES Decryption Key 0x17
16 TTG 0xffff
16 Battery voltage 0x6b05 = 17.17V
16 Alarm reason 0x0000
16 Aux voltage 0x0000
16 Mid voltage 0x0300 = 7.68V
16 Temperature 0x0000
Bits packed from here 2+22+20+10=54 0x???? ???? ???? ??
your remaining data =40 0x0080 fe43 3f
2 Aux input
22 Battery current
20 Consumed Ah
10 SOC

@schonstudent: You have only 40 bits left when the only documentation I've seen says I should expect 54.

Your read out type is 0xA3 where the documentation I have expects 0xA0. Perhaps that is a clue that the format has changed on more recent firmware. If you can find documentation on the Victron forums I'll have further suggestions, but this lack of bits is beyond me.

schonstudent commented 1 week ago

This is clear to me 16 | ManufacturerID | 0x02e1 = 'Victron Energy BV' 8 | Manufacturer Data Record type | 0x10 16 | model id | 0x0289 8 | read out type | 0xa3 8 | record type | 0x02 = Battery Monitor 16 | AES Initialization Vector | 0xf5ff 8 | First Byte of AES Decryption Key | 0x17

This data is stored with little endian Battery voltage is stored like this 0x6b05 but you need to interpret like this 0x056b ->1387 in mV * 0,01 = 13,87V 16 | Alarm reason | 0x0000 16 | Aux voltage / Mid voltage / Temperature | 0x0000 -> This is one imput but you can choose what you wanna connect 2 | Aux input |   22 | Battery current |   20 | Consumed Ah |   10 | SOC

with 16bit it's easy but with 2bit you need to convert everything to bin code and than convert it because it's using little endian

wcbonner commented 1 week ago

This is clear to me 16 | ManufacturerID | 0x02e1 = 'Victron Energy BV' 8 | Manufacturer Data Record type | 0x10 16 | model id | 0x0289 8 | read out type | 0xa3 8 | record type | 0x02 = Battery Monitor 16 | AES Initialization Vector | 0xf5ff 8 | First Byte of AES Decryption Key | 0x17

This data is stored with little endian Battery voltage is stored like this 0x6b05 but you need to interpret like this 0x056b ->1387 in mV * 0,01 = 13,87V 16 | Alarm reason | 0x0000 16 | Aux voltage / Mid voltage / Temperature | 0x0000 -> This is one imput but you can choose what you wanna connect 2 | Aux input |   22 | Battery current |   20 | Consumed Ah |   10 | SOC

with 16bit it's easy but with 2bit you need to convert everything to bin code and than convert it because it's using little endian

I got the MSB/LSB issue wrong in my manual conversion, but you are still missing bits on the end of the structure, which leaves me questioning the alignment and definition of the entire structure.

schonstudent commented 1 week ago

Yeah will try to fing some more infos

image