merbanan / rtl_433

Program to decode radio transmissions from devices on the ISM bands (and other frequencies)
GNU General Public License v2.0
6k stars 1.3k forks source link

Add support for TFA Stratos 30.3151 #2538

Closed jonnycastaway closed 9 months ago

jonnycastaway commented 1 year ago

Got an TFA Stratos without Head Unit. But all sensors are there an working. See description.txt Default device protocols and enabled (default disabled) protocols shows nothing.

Samples, links and little description are in the Repo rtl_433_tests PR #454 as mentioned here.

zuckschwerdt commented 1 year ago

Looks like PCM 60 µs with the usual 2DD4 sync word.

The 6 repeats don't seem to have a separator though. A code should be something like AAAAAA 2DD4 51E0 F13E 0000 0621 3C00 2 but you will see 6 of those joined into a single line.

This should work

rtl_433 -R 0 -X 'n=name,m=FSK_PCM,s=60,l=60,g=1500,r=2500,bits>=10,preamble=aaaa2dd4'

But to see the repeats you would rather need the raw output of rtl_433 -R 0 -X 'n=name,m=FSK_PCM,s=60,l=60,g=500,r=2500'

jonnycastaway commented 1 year ago

Ok, i've run these commands and add the output to the repo. But what's to do with these output? I haven't experience with decoding these data. But i'm willing to learn.

zuckschwerdt commented 1 year ago

Table the codes with the measured data, e.g.

51E0 F13E 0000 0621 3C00 2 [12.3 C ...]

Then we can figure out where the temperature is in that data. Also with enough codes we can figure out the checksum -- should be somewhere in the 6213C part.

ProfBoc75 commented 1 year ago

Hi, My analysis : The message is 9 bytes, I confirm that the last bytes is crc8, poly 0x31, init 0x00, so 8 bytes of useful data. The message is repeated 6 times into the row, we have a small gap between them but unfortunately flex decoder split the message because of zeros bigger than the gap, so not good to use it. The message data are every 123 bits with preamble{40} = aaaaaa2dd4. After 1st preamble{40} =aaaaaa2dd4 : Message1{72} = from bit 0 to bit 71 , gap1{11} = from bit 72 to bit 82. preamble2{40} = from bit 83 to 122, Message2{72} = from bit 123 to 194 gap2{11} = from bit 195 to 205 ... / ... Message6{72} = bit 615 to 686

But search the preamble into the row within a loop will be better and will avoid any shift of bit positions.

Message with crc8 from source
51 e0 e1 49 00 00 03 12 67 PCM_60_µs-2DD4_output.txt
51 e0 f1 3e 00 00 03 10 9e g007 & g008.cu8
51 e0 f0 3e 00 00 03 10 c9 g009 & g010.cu8
51 e0 f0 3f 00 00 03 10 1a g011, 12, 13.cu8

Now I can't say what is what , I need to compare with existing decoders like ambientweather_wh31e very close to this one ... to be continued ...

ProfBoc75 commented 1 year ago

Hi,

I'm not able to find more information, can you capture more payloads with :

rtl_433 -f 868.3M -R 0 -X 'n=TFA303151,m=FSK_PCM,s=60,l=60,r=2500,bits>=10,preamble=aaaa2dd4'

Also, can you check at the same time, with another sensor, the temp & humidity, and put them with the captured messages.

Did you have also the Wind and Rain sensors connected to it ?

if yes, add some water, 1 Litter and note if any changes in the payload, also measure the funnel size (top internal diameter). The goal is to get the mm by m² of rain for 1 counter change/pulse, so we need the exact surface size of the funnel and how much the counter increased for exact 1 Litter (each "clac" noise = 1 pulse) . Example the funnel diameter is 9.21 cm, its surface = 66.666 cm² so 1/150 m² , and you have 150 pulses increased into the counter with 1 litter, this means that 1 Litter for 66.66 cm² = 150 Litters for 1 m², divided by 150 pulses = 1 Litter by pulse / m². 1 litter is 1 mm / m² , so here 1 pulse = 1 mm / m² . If you have 750 pulses , then 1 pulse = 0.2 mm/m² ... it's mathematical...

Also move the wind elements to get some changes into the payload and see the position of wind.

jonnycastaway commented 1 year ago

Hi, ans sorry for my late answer. I was on a climbing holiday a few days. I will collect new data this afternoon with some reference data for temp and humidity with a second, other labeled sensor. And yes, the wind and rain sensors are connected. So i can also collect data of them but without reference data. What i found in the manual is the presence of an DCF77 radio clock also in this unit. So it's possible that time and date is also transmitted.

jonnycastaway commented 1 year ago

Ok, one step further: In these file there are 3 datagrams. Additionally there's the values from an TFA Dostmann 30.3126 (Hideki-TS04) also. As you can see, only the Humidity changed. Because the wind and rain sensor are also not used (but connected) the only data that changed is humidity and maybe some time/date from DCF77. Today in the afternoon i'll do the same with temperature, wind and rain one by one.

zuckschwerdt commented 1 year ago

Format data into a BitBench like this to make analyzing easier.

jonnycastaway commented 1 year ago

Hi, i think i found Temp.

51e0bd3f000003231a0015555545ba8a3c17a7e0000064634002

If i use bitbench with following format line: 8b 8h TEMP_C:8d it seems that after first TEMP_C: the next 3 digits are Temp: 01010001 e0 TEMP_C:18601000000 00 TEMP_C:00000000011 23 TEMP_C:17400000000 15 TEMP_C:08501010101 45 TEMP_C:18610001010 3c TEMP_C:02301001000 00 TEMP_C:00000000000 64 TEMP_C:11711000000 02

It also seems that aaaaa8b75 is also an preamble.

ProfBoc75 commented 1 year ago

Hi @jonnycastaway

As I explained in my previous post, the same message is repeated 6 times with a small gap between each. The reason why it's look like you have another preamble, it's because all the bits are shown in hexa values (multiple of 4 bits) in one row, including the gap that should be excluded to align the bytes.

bytes not aligned :

51e0da3a00000312fb0015555545ba8a3c1b47400000625f6002aaaaa8b751478368e800000c4bec0055555516ea28f0 ....
| Message         |gap|preamble|Message         |gap|preamble|Message        |gap|preamble|Message ....

The gap is around 11 bits long and therefore not a multiple of 4 bits (hexa nibble) so there is an offset in the hexadecimal values. This is why we are using the preamble feature to find the good position of the bytes of the message payload, then the bytes are aligned (gap excluded):

51e0da3a00000312fb [gap] aaaaaa2dd451e0da3a00000312fb [gap] aaaaaa2dd451e0da3a00000312fb [gap] aaaaaa2dd451e0da3a00000312fb [gap] aaaaaa2dd451e0da3a00000312fb [gap] aaaaaa2dd451e0da3a00000312fb

Without the preambles, you have 6 times the same useful data

51e0da3a00000312fb [gap] [preamble] 51e0da3a00000312fb [gap] [preamble] 51e0da3a00000312fb [gap] [preamble] 51e0da3a00000312fb [gap] [preamble] 51e0da3a00000312fb [gap] [preamble] 51e0da3a00000312fb

Now looking to the message data , looks like this : Bitbench with your sample values

preamble{40} message{72} gap{11}

byte    00 01 02 03 04 05 06 07 08
message II IT TT HH ?? ?? RR RR CC
sample  51 e0 da 3a 00 00 03 12 fb [21.8C 58%]    -> very close to the 22.1C 52% from other sensor

I : 12 bits Model = 0x51 (8 bits) , 0xe = ID / Battery ? T: 12 bits Signed Temp in Celcius * 10 H: 8 bits Humidity % ?? : 16 bits probably for Wind Speed and Direction ? RR: 16 bits probably for rain gauge since the value has little increased since your previous samples.

ID:8h4h TEMP_C:12s HUM:8d 8h8h RAIN:16d CRC:8h

This is my estimation, we don't know the battery signal if any, and why nothing at Wind level ?

jonnycastaway commented 1 year ago

Hi @jonnycastaway

As I explained in my previous post, the same message is repeated 6 times with a small gap between each. The reason why it's look like you have another preamble, it's because all the bits are shown in hexa values (multiple of 4 bits) in one row, including the gap that should be excluded to align the bytes.

bytes not aligned :

51e0da3a00000312fb0015555545ba8a3c1b47400000625f6002aaaaa8b751478368e800000c4bec0055555516ea28f0 ....
| Message         |gap|preamble|Message         |gap|preamble|Message        |gap|preamble|Message ....

The gap is around 11 bits long and therefore not a multiple of 4 bits (hexa nibble) so there is an offset in the hexadecimal values. This is why we are using the preamble feature to find the good position of the bytes of the message payload, then the bytes are aligned (gap excluded):

51e0da3a00000312fb [gap] aaaaaa2dd451e0da3a00000312fb [gap] aaaaaa2dd451e0da3a00000312fb [gap] aaaaaa2dd451e0da3a00000312fb [gap] aaaaaa2dd451e0da3a00000312fb [gap] aaaaaa2dd451e0da3a00000312fb

Without the preambles, you have 6 times the same useful data

51e0da3a00000312fb [gap] [preamble] 51e0da3a00000312fb [gap] [preamble] 51e0da3a00000312fb [gap] [preamble] 51e0da3a00000312fb [gap] [preamble] 51e0da3a00000312fb [gap] [preamble] 51e0da3a00000312fb

Now looking to the message data , looks like this : Bitbench with your sample values

preamble{40} message{72} gap{11}

byte    00 01 02 03 04 05 06 07 08
message II IT TT HH ?? ?? RR RR CC
sample  51 e0 da 3a 00 00 03 12 fb [21.8C 58%]    -> very close to the 22.1C 52% from other sensor

I : 12 bits Model = 0x51 (8 bits) , 0xe = ID / Battery ? T: 12 bits Signed Temp in Celcius * 10 H: 8 bits Humidity % ?? : 16 bits probably for Wind Speed and Direction ? RR: 16 bits probably for rain gauge since the value has little increased since your previous samples.

ID:8h4h TEMP_C:12s HUM:8d 8h8h RAIN:16d CRC:8h

This is my estimation, we don't know the battery signal if any, and why nothing at Wind level ?

Hi @ProfBoc75 thanks for this detailed description. That clarified a lot to me. I'm not familar with such investigation and learn a lot at this point. So i will further investigate the data as you explained.

jonnycastaway commented 1 year ago

Ok, after @ProfBoc75 clarification i played a bit. You are right, the ?? is possibly the wind speed. Thats beside temp & hum the only moving part at moment because there's no rain ;-). I struggle a bit with the unit. It seems like WIND *100 in knots. My investigations are here: file

ProfBoc75 commented 1 year ago

Yes , we have some answers !

I reviewed the TFA STRATOS 35.1077 pdf (link into your README) and there's no wind vane, only wind speed sensor. And in fact, you have 2 Wind information :

And we can see that the RAIN minimum resolution is 0.3 mm, this means that 1 pulse = 0.3 mm/m²

So we have this bitbench :

ID:8h4h TEMP_C:12s HUM:8d WIND:8d GUST:8d RAIN:16d CRC:8h
byte    00 01 02 03 04 05 06 07 08
message II IT TT HH WW GG RR RR CC
sample  51 e0 c2 40 01 04 03 23 0e [19.4C 64% Wind Speed = 1 ? , Gust Speed = 4 ?, Rain = 803 ? ]  

We need to compare the Wind information from other sensor with the same kind of sensor to have the correct conversion into km/h or m/s ... not sure it is knots , I checked rtl_433 devices, none use this unit.

We have enough information to draft a new device driver.

Here the new Bitbench

ProfBoc75 commented 1 year ago

One more thing,

I saw also into the pdf guide that the DCF information is sent by the thermo-hygro sensor, this means that , you can also receive another kind of message payload than those ones you already shared. And from this guide, you need to wait 5 minutes to get the DCF information during the initial setup. So, you have to monitor during 5 mn all the message payload and check if one is different than other (don't reset the device, just collect for 5 mn, hope you will receive DCF data) . The data are sent every 24 seconds, so you need to collect around 10 or 12 messages to get DCF information, hope so.

If I'm not wrong , 1 time by hour ( around minute 59 ) a full DCF signal is sent with all information (date , time ... ) , so you may have to collect at that time, 1 or 2 minutes before a new hour .. And only if you're living in Europe and within 1500 Km around Frankfurt.

ProfBoc75 commented 1 year ago

@zuckschwerdt ; In fact, I was looking into existing devices, and I noticed that we are facing to the FSK PCM version of this sensor already decoded but OOK PPWM : fineoffset_wh1050.c , which also refers to the same station TFA Stratos 35.1077 !

Except preamble, everything else is consistent with my findings.

jonnycastaway commented 1 year ago

Your right, with the informations in your link i found the DCF Data! I live ~80km north of Frankfurt (Mainflingen) where the DCF77 is located. Here ist the bitbench The Shown Time in the bitbench ist the time when rtl_433 showed the data. So second in decoded data is -1

ProfBoc75 commented 1 year ago

I started to work on the fineoffset_wh1050.c in order to add this new modulation, this will take some time (one or 2 days). Notice that the Temp is not offset by 400 with the FSK version, but easy to manage.

To be continued ...

jonnycastaway commented 1 year ago

Is it still necessary to make measurements on the rain sensor? Or is there anything i can inspect? Sadly i don't have another anemometer here. Maybe someone in my circle of friends has one. Preamble for Time is the same as Weatherdata: aaaaaa2dd4

ProfBoc75 commented 1 year ago

@jonnycastaway up to you, no emergency , when my updated device is ready I will check with you. Anyway , the guide says that rain resolution is 0.3 mm and this is already decoded like this into ook version.

Then later the device can be corrected if any errors are found by others who own same TFA 30.3151 FSK version with the station 35.1077.

ProfBoc75 commented 1 year ago

Hi ,

My update is ready, if you want to test: git clone my fork compile/make it run rtl_433 from build/src folder to test.

For me, it works with the message type 5 samples (cu8 files) from you and from the Fineoffset wh1050 samples too.

But we don't have message type 6 with DFC information to test this kind of message. If you can capture such message it will be nice.

Let us know your result and if positive, @zuckschwerdt will merge my request/commit (hope so ;-) ).

jonnycastaway commented 1 year ago

Hi,

I'm not able to find more information, can you capture more payloads with :

rtl_433 -f 868.3M -R 0 -X 'n=TFA303151,m=FSK_PCM,s=60,l=60,r=2500,bits>=10,preamble=aaaa2dd4'

Also, can you check at the same time, with another sensor, the temp & humidity, and put them with the captured messages.

Did you have also the Wind and Rain sensors connected to it ?

if yes, add some water, 1 Litter and note if any changes in the payload, also measure the funnel size (top internal diameter). The goal is to get the mm by m² of rain for 1 counter change/pulse, so we need the exact surface size of the funnel and how much the counter increased for exact 1 Litter (each "clac" noise = 1 pulse) . Example the funnel diameter is 9.21 cm, its surface = 66.666 cm² so 1/150 m² , and you have 150 pulses increased into the counter with 1 litter, this means that 1 Litter for 66.66 cm² = 150 Litters for 1 m², divided by 150 pulses = 1 Litter by pulse / m². 1 litter is 1 mm / m² , so here 1 pulse = 1 mm / m² . If you have 750 pulses , then 1 pulse = 0.2 mm/m² ... it's mathematical...

Also move the wind elements to get some changes into the payload and see the position of wind.

Puh, lot's of math but i'll try.

The Rainsensor is a rectangle with 5cm * 11cm.. That's 55cm² but the surface is bigger because it's a funnel. Is this not relevant?

If it isn't i investigated further: 1l (hope i was exact enough) is 344 pulses.

1m² = 10000cm² 55cm² = 1l 10000cm² = 181.81l

181.81l / 344pulses = 0,528l by pulse / m²

1l = 1mm/m2 0,528l/m² = 0,528mm/m²

Should this be correct? I don't know...

jonnycastaway commented 1 year ago

Hi ,

My update is ready, if you want to test: git clone my fork compile/make it run rtl_433 from build/src folder to test.

For me, it works with the message type 5 samples (cu8 files) from you and from the Fineoffset wh1050 samples too.

But we don't have message type 6 with DFC information to test this kind of message. If you can capture such message it will be nice.

Let us know your result and if positive, @zuckschwerdt will merge my request/commit (hope so ;-) ).

No Problem, here some DCF77 Data: file

Or did you need it in another format?

ProfBoc75 commented 1 year ago

About the Rain, your numbers sound good, but not what is defined into the current device.

Yes, the funnel collects as much as much water from its top, so here classic 5x11 cm, so 55 cm². so around 181.81 less than 1m². And yes 1 L into 55 cm² = 181.81 L into 1 m² , so 344 pulses = 181.81 L so 1 pulse = 0.5285 L = mm/m², you're right.

Looking other sensors with the same kind of rain gauge, the conversion is 0.5f or 0.5180f so very close.

For DCF, we need the cu8 file to be able to decode it from a raw rf signal (replay) , same you already captured and put in rtl test repository.

Edit: Did you put the water very carefully and slowly and not regularly, I mean a little water, stop, again little water stop, and random pause between to let the water reach the tipping bucket. It's because for this model, I can't find 0.5 mm/m² but 0.3 mm or 0.1 mm and into the guide is refer to 0.3mm, I would like to trust them.

jonnycastaway commented 1 year ago

Hi ,

My update is ready, if you want to test: git clone my fork compile/make it run rtl_433 from build/src folder to test.

For me, it works with the message type 5 samples (cu8 files) from you and from the Fineoffset wh1050 samples too.

But we don't have message type 6 with DFC information to test this kind of message. If you can capture such message it will be nice.

Let us know your result and if positive, @zuckschwerdt will merge my request/commit (hope so ;-) ).

Done and yes, your code is working.

rtl_433 -f 868.3M -R 246 gives

looking good. so only need the units for wind, gust and rain :-/

ProfBoc75 commented 1 year ago

You're right, I will add the units ... my mistake.

jonnycastaway commented 1 year ago

For me, it works with the message type 5 samples (cu8 files) from you and from the Fineoffset wh1050 samples too.

But we don't have message type 6 with DFC information to test this kind of message. If you can capture such message it will be nice.

I catched ~500 cu8 files over a larger timespan (20 minutes, over 1 GB) to hopefully catch the dcf77 signal. But can you tell me how to sort out? Nobody will showing such an amount of files by hand. ;-)

jonnycastaway commented 1 year ago

You're right, I will add the units ... my mistake.

cloned and compiled it again, now units are present. 👍

ProfBoc75 commented 1 year ago

Yes, Done, my updated fork gives the unit , (play also with -C option to change the units).

About the cu8 files, you can replay them with my fork version. (install it first with sudo make install from build folder) Move into the folder you stored them :

rtl_433 -r *.cu8

Note the names of each cu8 files that will give you the time and upload into rtl test few of them (2 / 3 files is enough ) to be able to replay them later. (message type = 1)

If my version is not showing the time, you can also replay them with the flex options :

rtl_433 -r *.cu8 -X 'n=TFA303151,m=FSK_PCM,s=60,l=60,r=2500,bits>=10,preamble=aaaa2dd4'

And note cu8 file where the message is starting with 61e ( instead of 51e...) , they are DCF messages, and upload those ones to be able to decode them.

jonnycastaway commented 1 year ago

Puh, that was tough but i uploaded some files with DCF77 Data in it. show here

ProfBoc75 commented 1 year ago

Well done ! and it works fine. I made a little correction at the Station ID level, the format was not the same between weather sensor (ID was hexa) and time data (ID was decimal) messages. Now both the Station ID is hexa value.

All is good.

If you have time to retest the rain gauge , expected around 606 pulses for 1 L , then 1 pulse = 0.3 mm/m² If you still have around 360 pulses, this means that we have 0.5 mm/m² and I will correct the device accordingly.

For the wind, it's very difficult to estimate, it's best to wait for a windy period with weather forecasts on wind speed and possible gusts, then compare with your results. With Open Weather Map you may have such forecast information.

jonnycastaway commented 1 year ago

I will screw the station on its's place tomorrow. I will do another test with the rain sensor then. For wind i ordered a cheap anemometer then i'll also test this.

jonnycastaway commented 1 year ago

Hi, checked the Rain Sensor again. 322 pulses this time. So i think, 0.5mm/m² is the nearly right value for this sensor. Maybe I threw the water a little bit too fast. But the funnel did not overflow.

ProfBoc75 commented 1 year ago

Hi,

I made the correction and pushed the update, since the both tests confirm that it's more 0.5 than 0.3 mm.

@zuckschwerdt , if you please, sounds all are ok now, can you merge my PR #2549 please.

jonnycastaway commented 1 year ago

The Anemometer arrived at me today and i checked the units. Seems that km/h is correct. ;-) It's a bit gusty today so i'll measure it next days again.

jonnycastaway commented 1 year ago

Checked windspeed with the anemometer again today. km/h as unit is correct. All is working perfect since weeks. So many THANKS to all 👍

gdt commented 11 months ago

Is there a PR on the table, or is this merged? If this should remain open, please summarize current status and what needs to be done.

jonnycastaway commented 10 months ago

Is there a PR on the table, or is this merged? If this should remain open, please summarize current status and what needs to be done.

As you can see in the Comment from ProfBoc75 yes there is a PR. It's tested and functional.

jonnycastaway commented 10 months ago

It's merged so i close this issue.

jonnycastaway commented 9 months ago

Damn Had to reopen the issue because i found a unnormal behaviour. The Temperature is never going below 0°C. Think there's something to find in the protocol? @ProfBoc75 Anything we/i ca do?

jonnycastaway commented 9 months ago

The radio clock also gives wired data: 2000-05-108T25:64:00

zuckschwerdt commented 9 months ago

Can you grab codes below zero and a weird date with -R 246:vv?

jonnycastaway commented 9 months ago

Of course. Her it is. There are some points where the radio clock is ok. Ithink that's the original radio clock packet. The other packets, with false radio clock values, are originally temp/hum packets i think because they comes more often. And there aren't fine packets with temp/hum anymore.

below_0.txt

zuckschwerdt commented 9 months ago

We seem to have gotten the "weather or radio" bit wrong. It might rather be a temp-sign bit. The strange radio clock packets are actually weather packets. They show a temp of 0.5 C -- are you sure it was below zero (-0.5 C)?

jonnycastaway commented 9 months ago

We seem to have gotten the "weather or radio" bit wrong. It might rather be a temp-sign bit. The strange radio clock packets are actually weather packets. They show a temp of 0.5 C -- are you sure it was below zero (-0.5 C)?

Pretty sure but don't nail me down. A temperature of 0.1°C is reported correctly. I record the temp/hum and visualize the data in grafana. The line stops moving down exactly at 0°C. I couldn't grab packages in range 1°C to -1°C at the moment because actually it's cold and below 0 all the day. But if it's a bit warmer at day i can try to catch packages around 0°C.

zuckschwerdt commented 9 months ago

The negative temperatures and wrong weather/clock packet detection is fixed now.

jonnycastaway commented 9 months ago

Ok, will test it tomorrow. Thanks for your quick help.

jonnycastaway commented 9 months ago

The negative temperatures and wrong weather/clock packet detection is fixed now.

Testes -> functional! Good work!

Many thanks for your superfast support!