Trying to decode signal from Marlec Solar Iboost+

[mehstg]

I have a Marlec Solar Iboost+ solar diverter which communicates to it's display unit on 868.3mhz. From a few threads online it appears to use a TI CC1100 tranceiver.

I have some sample cu8 files recorded and would love to understand how you determine the encoding/modulation etc. Running -a on a recording gives me output like below:

*** signal_start = 4294957301, signal_end = 59789, signal_len = 69784, pulses_found = 6
Iteration 1. t: 9201    min: 4104 (4)    max: 14298 (2)    delta 16102010
Iteration 2. t: 9201    min: 4104 (4)    max: 14298 (2)    delta 0
Pulse coding: Short pulse length 4104 - Long pulse length 14298

Short distance: 269, long distance: 487, packet distance: 1872

p_limit: 9201
bitbuffer:: Number of rows: 3 
[00] { 1} 80        : 1
[01] { 4} 10        : 0001
[02] { 1} 00        : 0

[dal31b05]

The problem with the method using the radio chip to do the work is it isn't easy to determine the packet length. The CC1100 protocol allows for variable packet lengths by inserting a packet length byte before the address. Reading the messages, it doesn't look like the first data byte is a packet length and therefore I am assuming it uses fixed packet length; problem is, I don't know how long the packets are but the radio happily fills anything that doesn't arrive as a modulated message with demodulated noise.

The other problem is that if the units are all set up for fixed packet length but transmit different length packets, they are not ideally set up to communicate with one another. It probably still works if the controller and the Buddy have the same packet length and can communicate tx/rx between themselves. The sensor is tx only so could be programmed with a shorter packet length that the receivers pad. As long as they never access the padded part of the packet, that wouldn't really matter.

Ideally I'd connect up to the radio chip in the iBoost and read all the config registers but the pins are 0.5mm pitch so hard to get on to.

I have assumed a packet length of 46 bytes for all messages so any that are shorter will be padded with random characters but hopefully this will be obvious. If there is a longer format, the messages will be truncated.

sensor.txt is only messages from the sensor so expect them all to be the same length, albeit probably not as much as 46 bytes.

CRC is optional in the CC1100 but if it is switched on, it occupies the last two bytes of the packet. Could use that as a means to find the end of the packet. It would also confirm whether the data with FEC on or off passes CRC. Not sure that check would detect all bits inverted though.

[real-bombinho]

I am not sure if I have a mental block, I had used https://crccalc.com/?crc=A7+9C+67+FD+FF+E5+FB+F9+EB+CD+BB+D1+C1+B1+CD+FF+F1+FF+FF+FF+FF+FF+FF+FF+FF+E3+FC+F8+F0+C1+98+C7+E1+D7+E1+FF+F8+FE+FF+FF+FF+FF+FF+FF+FE+00+&method=hex&outtype=hex My ModBus CRC16 works fine but my adaption of the CRC16 code in the swra111e.pdf results in 0xB1A7?

[zuckschwerdt]

Did you account for the bit reflection? That crcData & 0x80 is MSB first. But we'd need LSB first.

[real-bombinho]

Yes, but since my result is UInt16 too I did a mental (only) swap of the bytes anyway. Yet I could not find any matching result on that web site, which either means I did some mess or the algorithm used is not among the implemented ones on that site.

[real-bombinho]

Okay, I have confirmed the 0xBA17 to be the expected result. Meaning, the CRC in use seems something exotic.

[zuckschwerdt]

The CRC would be poly, init, and final xor. Using 0x8005, 0xffff you get 0xBA17 for every line?

[real-bombinho]

The CRC would be poly, init, and final xor. Using 0x8005, 0xffff you get 0xBA17 for every line?

No, it seems not to be. I confirmed that by using the original code from the provided documentation (Texas Instruments). The entire CC2400 CRC algorithm deviates from normal CRC16. Hence my insecurity.

Edit: Oops. I should learn to read: "CC1100... only support the normal mode CRC"

[dal31b05]

The data sheet doesn't contain much detail about the CRC, other than referring to it is CRC-16 (=2 x 8 bit bytes??).

For info, this is the packet format:

Just noticed, the CRC is stripped by the radio chip and not sent to the Rx FIFO so it won't be in the data examples I have provided. If we have the packet length, I can program the chip to indicate whether CRC is passed for each packet. Alternatively, I could go back to using async data mode because in that mode, the chip is not processing data as packets and just outputs everything on the GDOx line. Looks like you guys already have the equivalent of that through rtl_433.

[merbanan]

@dal31b05 if you have a logic analyser you sniff the spi lines of the CC110x chip. Sigrok can then decode the configuration commands. Then we know what parametrar to use.

[dal31b05]

Ideally I'd connect up to the radio chip in the iBoost and read all the config registers but the pins are 0.5mm pitch so hard to get on to.

Either way, I'd need to be able to connect on to the chip and I can't find a clip (under about £250) to do that. I could desolder the radio module off the main board in the Buddy but I'd have to remove the display first. The module in the sensor is conformally coated so would be harder to desolder and resolder.

[merbanan]

How about the mcu spi pins?

[zuckschwerdt]

referring to it is CRC-16 (=2 x 8 bit bytes??).

aaactually… CRC-16 is 17 bits. But you omit the MSB or LSB (Koopman), both are 1 anyway. Data ist single bits always. But I guess that is solved with the CC110x working fine to check and strip the CRCs

[dal31b05]

How about the mcu spi pins?

Worse, they are the same pitch and not even in the corners :-(

[real-bombinho]

Just updated the tool (BitStreamTool) mentioned earlier. Right click does now provide a few basic data formats.

[real-bombinho]

Starting column 7 there is a much higher likelihood to have a meaningful single (FP32) number than anywhere else, at least on a quick glance. Although still spanning a wide range.

[real-bombinho]

A manual random check does not produce any repeated lines. But the data is not random enough to be encrypted.

[real-bombinho]

Having the '00' coloured in there is a notable zig zag pattern, I think, there may be something off in the production of the data?

Although there are 4 equal bytes at the beginning and 3 non-zero bytes at the end.

(sensor.txt)

[zuckschwerdt]

That's a lot more 0's then we had before. Not even sure PCM can reliably deliver those long runs? Since those long runs are roughly the same length, isn't it more likely those are gaps between packets? I.e. neither mark nor space but no carrier.

[real-bombinho]

Just bit inverted (xor 0xff). Underneath is the original. The zig zag gets longer and shorter with random appearing phases in-between as if it is tuned in with a slight timing offset.

[zuckschwerdt]

Long runs of 1's also can't be transmitted with PCM coding. The only chance to recover clock is on edges. It's unlikely this is true data. The bit length is 20µs, 10 bytes of ff are 80*20 = 1600µs nominal. With per chance 19µs or 21µs time seen, thats already four bit off.

[dal31b05]

Just bit inverted (xor 0xff). Underneath is the original. The zig zag gets longer and shorter with random appearing phases in-between as if it is tuned in with a slight timing offset.

Looks very much like the output I see from the radio chip.

Are bytes 45 and 46 CRC? If so I can try setting a packet length of 45 and enabling CRC checking.

Hoping to find some time over the weekend to capture some messages with known current measurements.

[dal31b05]

That's a lot more 0's then we had before. Not even sure PCM can reliably deliver those long runs? Since those long runs are roughly the same length, isn't it more likely those are gaps between packets? I.e. neither mark nor space but no carrier.

I doubt it. As well as being similar to the data from the chip, I would think it likely that a gap between packets would be demodulated as more random rather than a constant stream of the lower sideband.

[real-bombinho]

Are bytes 45 and 46 CRC? If so I can try setting a packet length of 45 and enabling CRC checking.

Byte 46 is certainly not part of a CRC, it only varies significantly over time, usually it is only the last four bits that change from line to line.

[merbanan]

How about the mcu spi pins?

Worse, they are the same pitch and not even in the corners :-(

You only need clock and data in. That you can handle with 2 hands. Just add a signal for chip select that is always active and then you are good to go.

[merbanan]

@real-bombinho I really doubt that there are float values encoded in there.

And AFAIK we still dont have signal samples from only the power/energy meter.

[real-bombinho]

@merbanan None I have found yet, indeed. But it still helps to find regularities.

[dal31b05]

You only need clock and data in. That you can handle with 2 hands. Just add a signal for chip select that is always active and then you are good to go.

Depends what we're trying to do. To see the data that the MCU is sending to the radio, you only need two pins (and no CS because the MCU will handle that) but to read the config registers, I need to be able to select the chip, send the read request to the chip and see the reply. Even though I could permanently select the chip, I still need to be able to get on to the pin to do that.

Either way, I'm not sure I have steady enough hands to not move the tip of the probe, opposite to the end I'm holding, more than 0.5mm. What I really need is a test clip but I haven't found a reasonably priced one.

[dal31b05]

And AFAIK we still dont have signal samples from only the power/energy meter.

From what I have seen, each device sends messages with a different first byte. In the CC1100 protocol, that byte can be an address and the chip can be programmed only to receive data from a single address. I'm not convinced that is how it works here because in a long run of data, you get more than three addresses, although that could be due to errors.

As all of the packets in the sample above start with A7, I would assume they are all from the sensor. Also, messages between the other devices don't create the diagonal pattern, they are much more consistent.

[merbanan]

Well IIRC the RF module has the spi pins available on the pin header. It should be 1mm between the pins. And we just need to observe what the mcu sets. There is no need to read from the chip.

I did the same thing on an alarm system door switch.

[merbanan]

And regarding the messages, one source of data is from the energy/power meter. The focus should be to first figure out what data is in there.

[dal31b05]

The module in the sensor is conformally coated.

I can't get on the sensor one at all.

Since we've read the sensor data two ways (rtl_433 and CC1100) and it looks similar, I would say what we're seeing in the packets is worth taking to the next stage (decoding the protocol).

[zuckschwerdt]

@real-bombinho do have, or can you get a sample file for one of those transmission with the long runs of mark? I'd really like to look at the FSK.

[real-bombinho]

@zuckschwerdt unfortunately not, I just had picked up out of curiosity. I have no suitable device myself to experiment with. I rely on the published data here.

[real-bombinho]

all devices, filtered for first byte

no zig zag

zig zag

[dal31b05]

Agreed. Looks like only the sensor creates the zig zag pattern.

I have done some data captures against known measurements of power passing through the clamp. A few points: I haven't filtered for devices. The sensor and Buddy were close by but the controller wasn't. I'm not sure if it was too far away to have recorded any messages in the stream.

In each case, the file name describes the situation. The value shown on the Buddy doesn't change instantaneously, it is damped. The values in the filename is where the reading settled. Each file starts with a couple of messages before the load was switched on. The load is then present until the end of the file.

One strange thing I noticed is that the Buddy shows -- when the load is switched off. Once a load is switched on, whatever it's value, it seems to start with the last known value. This was particularly noticeable on the first export file (33w) because even though it was exporting, the Buddy shown in importing flow, which counted down and then counted up in the export direction. The order of tests can be seen from the timestamps.

646w import.txt 1286w export.txt 1320w import.txt 1954w export.txt 2003w import.txt 33w export.txt 33w import.txt 631w export.txt

[bluey6600]

Any further progress on this? I have an iboost so can provide additional data if needed, although new to using rtl_433. Looking to get the data into home assistant alongside the solar generation to track usage.

[merbanan]

IIRC we need sample signals that are only from the power/energy meter. Right now there are signals that cover bidirectional communication. Right now we need to be able to figure out one party at the time. So I suggest you power down all other components and then grab a few recordings. That way we should be able to id its messages. After that we can grab files and match it against values decoded by the panel.

[whiteduck22]

I have the same issue with the iBoost. Oddly device support was added with 188 but mine isn't receiving any data so do I need to pass any other command line than 868.3M? Thanks

[whiskerp]

I am getting more reliable capture and decoding with a pulse length of 10: rtl_433 -R 0 -f 868.298M -X 'n=Marlec,m=FSK_PCM,s=10,l=10,g=1000,r=2000,preamble=aad391d391' -s 1000000

I changed the marlec_solar.c module locally to reflect this and it now CRCs OK too:

time : 2022-09-07 16:30:45 model : Marlec-Solar Raw data : e3940100000000000000040d1929200f1d1c110900000000000000000000140c01001e10211a120700000000 Integrity : CRC

---

time : 2022-09-07 16:30:55 model : Marlec-Solar Raw data : e39401000000000000000000030e192b25122522170b0000000000000000122821181c2d23112621170b0000 Integrity : CRC

Code change:

r_device marlec_solar = { .name = "Marlec Solar iBoost+ sensors", .modulation = FSK_PULSE_PCM, .short_width = 10, .long_width = 10, .gap_limit = 1000, .reset_limit = 2000, .decode_fn = &marlec_solar_decode, .fields = output_fields, };

Much shorter gap and reset limits seems to chop up the bitstream.

[c3iq]

Hello, I compiled the latest git about a week ago. I've just recorded a .bin file while doing a screen capture of the output with a webcam image of the iBoost Buddy+ and an OWL energy meter.

The video and bin file are here:

http://c3iq.co.uk/Marlec/

There is an xz tar archive of both saving 400MB.

The command I used was:

rtl_433 -W Marlec.bin -vv -f 868.298M -R 188

I hope this is helpful.

[dc42]

I am trying to integrate my iBoost+ version 1.0 with an energy management system to manage it along with my newly installed additional PV, battery and EV charger. In order to control when the iBoost+ turns on and how much power it uses, I will either have to fake the RF protocol from the sender, or insert a voltage in series with the CT (the CT has a built-in burden resistor). I will also need to capture the heating power being used by the iBoost+, perhaps by decoding the RF protocol.

After reading this thread, I decided that in order to understand the protocol I should capture the SPI commands to the CC1100 RF transceiver module using a logic analyser. I have now done that, and come to the following conclusions:

• The sender unit I have is transmit-only. All data bytes read over SPI have the value 0x0F. There is no change in the data with the main iBoost+ unit switched on or off.
• The PIC microcontroller on the sender sends a preamble of 0x8D,0x2D,0x01,0x00 followed by a 40-byte data packet
• Each byte of the data packet is an 8-bit ADC reading of a value from the CT. Only positive values are represented, so negative values from the CT are transmitted as zero. This explains why there are so many runs of zeros and a zigzag pattern in the RF data recorded by other users.
• The 40 bytes of current data cover two whole cycles of mains @ 50Hz.
• A transmitted instantaneous current value of 140 (decimal) corresponds to a reading on the iBoost Buddy of about 2600W. So each transmitted byte is approx. 9.1 times the instantaneous current in amps measured by the CT, which means the resolution is about 110mA.
• The 0x01 0x00 bytes may be the sender version number
• The whole burst lasts about 2.75ms. Therefore the sender must capture and store readings for 40ms and then send them all as a burst.
• The iBoost main unit needs to know the phase of the current relative to mains voltage. My guess is that the final current sample is taken shortly before sending the burst, and the iBoost main unit assumes this.

The following plot shows the values of the bytes sent to the CC100 over SPI, including 4 bytes at the start and 1 at the end to set up the CC1100.

[dc42]

PS using this command: rtl_433_64bit_static.exe -f 868.268M -X "n=Marlec,m=FSK_PCM,s=10,l=10,g=1000,r=2000,preamble=aad391d3912c" when there is a load I see the following data: 8d2d0100000000000c386177848b7f64441d0000000000000000000009345e76838b7d614119000000000000c3af8 which I think is more or less in accordance with the above (the rtl_433 Windows binary I am using is old and doesn't appear to support CRC checking). When there is no load, rtl_433 displays a shorter message containing a long string of zeros, which I think is because PCM sync has been lost.

[dc42]

I've made further progress on this and I think I have decoded the main parts of the protocol. Basically the Buddy sends a message beginning 0x2C (in place of the 0x1d sent by the sender) and a pile of zeros periodically, to tell the main unit that it is running. Having seen this, after the main unit receives a reading from the sensor it transmits a message beginning 0x25 to the Buddy. This message contains a lot of zeros, a few ones, a 16-bit little endian unsigned integer representing the heating power, and a 32-bit little endian signed integer representing the import or export power (negative means export). Details at https://miscsolutions.wordpress.com/2022/10/02/home-energy-management-system-part-2-hacking-the-iboost-protocol/.

[mehstg]

I am extremely excited about your progress on this. I no longer have an iboost, but this has been dormant for a long time!

[no1knows]

+1 following your progress with interest, @dc42! Would love to tie my iBoost into HomeAssistant, perhaps with a CC1100 attached to one of the Pis dotted around the house.

[dal31b05]

This is looking much more promising. The 1ms reading of instantaneous current certainly explains the pattern seen from the sender. I’m wondering if the conversion from measured value to transmitted value is linear. If a peak reading of 140 corresponds to a displayed value of 2.6kW, the maximum range is going to be quite limited compared to the values of current that could be seen at the meter.

To be able to fake the signal, we will need to know the set up of the CC1100. Have you managed to capture any initialisation messages from the MCU to the radio? Alternatively, can connect an external MCU to your logic analyser wiring to read the config registers?

[whiskerp]

This is great work. I don't have an iBoost Buddy, just the sender so I am primarily interested in the import/export readings.

I'll look at my data and see if I can now make sense of it.

Thanks for your efforts. Marlec have written to me recently promising to make the spec available at some time I'm the future (see below)

Peter

Dear Mr Whisker

Many thanks for your email and glad to hear your thoughts about the Solar iBoost+. It is a simple little product but we get some nice feedback about it's performance.

At present, the information you requested is not something I can supply due to the fact that we work with a third party for the communication side of the product. However, we are working on the next version of the Solar iBoost and it will probably use the same communication protocols but we are planning to have more information available at the time for people such as you who want to extract more information from the clamp / sender device.

I have not got a date of the next version Solar iBoost but I can happily give you the information you need once it is released.

I hope this information is useful and once again I hope to be in a position to supply you the information at a later date.

Kind regards

https://g.page/Marlec/review?rc Stuart James - Sales Director Tel:+44 (0) 1536 201588 Mobile: +44(0)7918 704322

 <https://www.facebook.com/marlecengineering>

https://www.linkedin.com/company/marlec-engineering-co-ltd/ https://www.instagram.com/marlecrenewables/ https://twitter.com/MarlecRenewable www.marlec.co.uk

On Mon, 3 Oct 2022, 08:16 dal31b05, @.***> wrote:

This is looking much more promising. The 1ms reading of instantaneous current certainly explains the pattern seen from the sender. I’m wondering if the conversion from measured value to transmitted value is linear. If a peak reading of 140 corresponds to a displayed value of 2.6kW, the maximum range is going to be quite limited compared to the values of current that could be seen at the meter.

To be able to fake the signal, we will need to know the set up of the CC1100. Have you managed to capture any initialisation messages from the MCU to the radio? Alternatively, can connect an external MCU to your logic analyser wiring to read the config registers?

— Reply to this email directly, view it on GitHub https://github.com/merbanan/rtl_433/issues/1739#issuecomment-1265032173, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFDKVDYYCQYDILOHOO6INLWBKB5DANCNFSM46UYY6LA . You are receiving this because you commented.Message ID: @.***>

[zuckschwerdt]

using ...-X "n=Marlec,m=FSK_PCM,s=10,l=10,g=1000,r=2000,preamble=aad391d3912c" ... containing a long string of zeros, which I think is because PCM sync has been lost.

@dc42 The gap-limit of 1000 with pulse width of 10 already allows for 100 bits of zero (25 0 nibbles) which means the timing needs to be stable to less than a percent, which does not seem reasonable to assume ;)

But here is an idea: we could (for this one protocol, not in general) read with a very long gap/reset limit and then fix the buffer latter by knowning where the next 1-bit should be -- that's if we can guess the alignment of those lonely values in the sea of 0's.

[dal31b05]

This might be useful:

It's from the data sheet for the radio module.

Some of the packet structure is dependent on configuration parameters so it would be useful to understand how the radio is configured.

[dc42]

@whiskerp that's interesting! @dal31b05 I'll look at capturing the sender SPI comms when the battery is first connected. @zuckschwerdt the bit rate is (I believe) controlled by the crystal on the wifi module, so better than 1% timing stability may be achieved. However, the fact that when the current is zero the Buddy displays '--' and the main unit transmits -1 for both the heating power and the import/export power suggests to me that the main unit is detecting the message with all-zeros data, but losing sync and failing the CRC check. I am surprised that data whitening isn't turned on in the CC1000, as that would have avoided problems with repeated zeros (or repeated ones).