Add Klimalogg/30.3180.IT Support

[devZer0]

is there somebody willing on supporting adding support for these sensors? i have them running with tfrec but would like to see support in rtl_433 because it is a more capable tool. rtl_433 does not seem to decoce anything...

i can provide any input... i have access to 5 sensors. could need some help. could also donate a sensor to the project if somebody likes...

# tfrec -D -T 1
Registering demod for TFA_1 KlimaLoggPro
Found Rafael Micro R820T tuner
AUTO GAIN
Frequency 868.2500MHz
Samplerate 1536000
START READ THREAD
#000 1548583063  2d d4 6f 44 86 44 1e 60 e0 56 7d           ID 6f44 +24.4 30%  seq e lowbat 0 RSSI 78
#001 1548583063  2d d4 39 96 85 84 2c 60 a0 56 0c           ID 3996 +18.4 44%  seq a lowbat 0 RSSI 81
#002 1548583067  2d d4 3f 14 85 22 34 60 c0 56 66           ID 3f14 +12.2 52%  seq c lowbat 0 RSSI 81
#003 1548583068  2d d4 63 41 86 26 26 60 60 56 85           ID 6341 +22.6 38%  seq 6 lowbat 0 RSSI 74
#004 1548583069  2d d4 43 56 85 76 2e 60 b0 56 c3           ID 4356 +17.6 46%  seq b lowbat 0 RSSI 81
#005 1548583073  2d d4 6f 44 86 44 1e 60 f0 56 13           ID 6f44 +24.4 30%  seq f lowbat 0 RSSI 77
#006 1548583073  2d d4 39 96 85 84 2c 60 b0 56 62           ID 3996 +18.4 44%  seq b lowbat 0 RSSI 81

# rtl_433 -f 868.25M -s 1536000 -G
rtl_433 version 18.12-71-gb8578dc branch master at 201901231237 inputs file rtl_tcp RTL-SDR
Trying conf file at "rtl_433.conf"...
Trying conf file at "/root/.rtl_433/rtl_433.conf"...
Trying conf file at "/usr/local/etc/rtl_433/rtl_433.conf"...
Trying conf file at "/etc/rtl_433/rtl_433.conf"...
Registered 114 out of 120 device decoding protocols [ 1-4 6-8 10-26 29-64 67-120 ]
rtl_433: warning: 104 "Wireless M-Bus, Mode C&T, 100kbps (-f 868950000 -s 1200000)" does not support CSV output
rtl_433: warning: 105 "Wireless M-Bus, Mode S, 32.768kbps (-f 868300000 -s 1000000)" does not support CSV output
rtl_433: warning: 106 "Wireless M-Bus, Mode R, 4.8kbps (-f 868330000)" does not support CSV output
rtl_433: warning: 107 "Wireless M-Bus, Mode F, 2.4kbps" does not support CSV output
Found Rafael Micro R820T tuner
[R82XX] PLL not locked!
Sample rate set to 1536000 S/s.
Tuner gain set to Auto.
Tuned to 868.250MHz.

[zuckschwerdt]

Sure, I'll take a look. I contacted the tfrec guys on joining efforts some time ago but the project seems somewhat dormant.

[devZer0]

i would rename the issue title of https://github.com/baycom/tfrec/issues/7 to better reflect the intention

[zuckschwerdt]

I've added NRZ-S decoding (cce293c) so support should be simple. Can you grab some (3-5) signals and add them along with a readme describing the sensor and the approx. grabbed values. S.a. https://github.com/merbanan/rtl_433_tests#contributing

[devZer0]

great. how to submit? should i fork into my own repo and then add that files and create a pull request - or better send via email to you ? (unsure, because seeing unmerged pull requests at https://github.com/merbanan/rtl_433_tests/pulls )

[zuckschwerdt]

PR to rtl_433_tests is prefered. Some PRs there are pending on various updates and changes, but generally we always merge directly.

[anse1]

devZer0 writes:

is there somebody willing on supporting adding support for these sensors?

I do have a working gnuradio receiver for this sensor: https://github.com/anse1/klimalogg/

I tried getting it into rtl_433 but got frustrated when the ad-hoc manual decoder was extremely sensitive to manual level adjustment for it to keep synchronization. I might give it another try now that there's more than one person interested…

regards, Andreas

$ for f in *cu8; do ./rtl_433 -r $f -l 773 -X name=klimalogg,modulation=OOK_PCM,short=48,long=49,reset=65,match=0x55df755dd; done Test mode active. Reading samples from file: g001_868.3M_1000k.cu8 time : @0.267816s model : klimalogg count : 1 num_rows : 1 rows : len : 553 data : fdd555555555555555555555555555555555555555555555555555555555555555555555555555555555df755ddff755fddf7d5775f555f5557d55557ddd5ddf55555555550 codes : {553}fdd555555555555555555555555555555555555555555555555555555555555555555555555555555555df755ddff755fddf7d5775f555f5557d55557ddd5ddf55555555550 Test mode active. Reading samples from file: g002_868.3M_1000k.cu8 time : @0.075392s model : klimalogg count : 1 num_rows : 1 rows : len : 553 data : fdd555555555555555555555555555555555555555555555555555555555555555555555555555555555df755ddfd755fddf7d5775f555f5557d55d57ddd77f755555555550 codes : {553}fdd555555555555555555555555555555555555555555555555555555555555555555555555555555555df755ddfd755fddf7d5775f555f5557d55d57ddd77f755555555550 Test mode active. Reading samples from file: g003_868.3M_1000k.cu8 time : @0.325256s model : klimalogg count : 1 num_rows : 1 rows : len : 553 data : fdd555555555555555555555555555555555555555555555555555555555555555555555555555555555df755ddff755fddf7d57fdf555f5557d55757dddf7df55555555550 codes : {553}fdd555555555555555555555555555555555555555555555555555555555555555555555555555555555df755ddff755fddf7d57fdf555f5557d55757dddf7df55555555550

[zuckschwerdt]

For OOK_PCM with NRZ(S) short and long should equal, otherwise it's RZ-code. Also reset=65 says "if there is a 65 µs gap then end the packet" you rather want something like 480 there (for bursts of up to 10 equal bits).

[devZer0]

i collected testdata for klimalogg and have done a pull-request.

isn't that a little bit too large binary repo? checking out >1gig for adding some megabytes is rather inefficient and afair, git is not made for storing large amounts of binary data

[zuckschwerdt]

Nearly 5000 files, each around 200KB. Git is still well suited for that (large files wouldn't be too good, many files are ok). But you are right, adding samples should be easier and better structured (guided, example Readme).

Ideas anyone?

[merbanan]

We could rebuild the git. That should help getting rid of some history. But other then that the only way is to thin out the sample archive. One could try to check out a subset. But I'm not sure how that would work.

[zuckschwerdt]

I just tested recreating the repo and it isn't any smaller. We are mostly only ever adding stuff so that sounds about right. We could perhaps shave of 5% by removing files. But there are no obvious offenders:

find tests -type d | while read d ; do echo $(find "$d" -type f -maxdepth 1 |wc -l) $(du -sh "$d") ; done | sort -g

Even the over stuffed sample collections (the last lines there) don't weigh too heavy.

Perhaps we could set up a mostly empty branch to pull (shallow clone) and commit to? Not sure how much hassle it would be to move the new commits.

[zuckschwerdt]

Note that for this signal the PD_MIN_PULSE_SAMPLES needs tuning down to about 5. Even then the detection isn't very good. We really need to rework the pulse detect to work better for GFSK / CPFSK.

[merbanan]

As far as I understood the code logic the demodulated signal is low pass filtered. Then 2 estimators of the low and high level needs to converge (the symbol threshold is then just the mid point between these levels). I think that is the hardest demand of this generic demodulator. These level estimators are just set by some initial heuristics that might not be optimal for this type of signal. How to make it better is not really clear to me but in theory because we have the complete signal we can analyse the complete signal in one pass to recover optimal parameters that we use in a second bit recovery pass.

Maybe we need to move the problem to the the frequency domain. There was someone who did an OOK demod via fftw.

[zuckschwerdt]

I guess FFT would be really expensive. I made a note to look into wavelets for cheaper FSK, but so far I havn't delved into that. Also someday I want to try a FIR lowpass filter instead of the PD_MIN_PULSE_SAMPLES counter.

For the estimators we can perhaps use a proper mean+variance band and better see where transitions are. That would apply to (non-gaussian) OOK only. For GFSK we probably need another detector -- measuring the direction/slope of the frequency shift looks more reliable than looking at the level. I'll try to finish the modularization soon, so people can mess around with (many) different detectors easily.

[MichaelRenner]

Is there any news here? I would like to use the KlimaLOGG too. Apparently rtl_433 does not know the sensor yet.

[merbanan]

I am slowly working on a new FSK capable demodulator. If/when that is in place we will be able to better support more types of FSK signals.

[MichaelRenner]

Thanks for this info, good to know. And thank you for writing such an useful application!

[merbanan]

Try with changing PD_MIN_PULSE_SAMPLES to 5. If that works we can start figuring out the payload structure. And try to record a clear set of sample signals.

[MichaelRenner]

I gave up. Regardless of the gain settings, a lot of crap was received, sometimes with only one bit - even with a dummyload instead of an antenna. And even worse: Even with tfrec, which actually supports this device, no decode was possible. Now I read the signals of the neighbors :-)

[pjenglund]

I am also interested in getting support for this sensor in rtl_433. I changed the PD_MIN_PULSE_SAMPLES to 5 but I could not see any noticeable difference in the output. I have six different sensors of which tfrec finds 4 continuously, one only occasionally (very rarely) and one not at all. However, they are all detected and displayed correctly by the KlimaLogg main unit. I can try to record some clean samples with different settings if there is still a need for that?

[merbanan]

Well adding a few signals (from each sensor) should be a fair contribution.

[MichaelRenner]

Well, that motivated me to keep looking for it. I found the sensor 100kHz higher than expected. I collect the data soon.

[merbanan]

The samples in the samples repository are distorted. We now have a wip fsk demod that might work with this sensor.

https://github.com/merbanan/rtl_433/tree/feat-fsk_demod_new

[tophee]

How can I test this? (Sorry, if this is a dumb question. I am just getting started with rtl_433...)

[merbanan]

This signal is ASK modulated, not a FSK based one. Looks like a demodulator rework is needed to be able to decode this signal.

[merbanan]

@tophee nothing to test yet.

[merbanan]

This is highly suspicious. The tfrec project uses a FSK demod for this. The samples are all distorted, I need better samples before I do any further work.

[tophee]

How can I provide you with those samples?

[merbanan]

Upload them to the issue.

[tophee]

Yes, but how do I obtain samples from my Klimalogg devices and ensure that they are usable (i.e. not distorted)?

[merbanan]

rtl_433 -S unknown -g 30, then open them with examples/sigrok-open.sh and look at the waveform. The bits should be decoded properly and the waveform should not have any clipping. You will probably need to adapt the gain parameter (-g).

[alxlo]

Here are some samples recorded with rtl_433 -S unknown -g 42 -f 868250000 I hope I did this right and it helps. tfa_30.3180_samples.zip

If it would be any help, I'm very willing to donate one of these sensors to you.

CAVE: I can not be sure, that these are really my TFA1 sensors (ID 356a or 364f), there are some strong TFA2 sensors around here and I have no way to cross-check. If required I will have to get out to a place with less noise.

[merbanan]

The signals look ok, but they are ASK modulated. And the current code misses the whole preamble and a few bits in the payload. Anyway rtl_433_tests/tests/TFA-KlimaLogg-30.3180.IT/02/g006_868.25M_1536k.cu8 demodulated correctly.

src/rtl_433 -A -r ../rtl_433_tests/tests/TFA-KlimaLogg-30.3180.IT/02/g006_868.25M_1536k.cu8 -X name=klimalogg,modulation=OOK_PCM,short=80,long=80,reset=6500,preamble=0x55df755dd

[mueckinger]

I love this sensors. They are cheap and extremely energy effiicent. Battery life is more than 3-5 years. I operate 6 of them and look forward to read them with rtl_433.

[merbanan]

src/rtl_433 -l 15000 -s 1536k -A -r ../rtl_433_tests/tests/TFA-KlimaLogg-30.3180.IT/01/g001_868.25M_1536k.cu8

Adding -l 15000 gives a proper pulse recovery and this sample can be decoded by tfrec.

./tfrec -L ../../rtl_433_tests/tests/TFA-KlimaLogg-30.3180.IT/01/g001_868.25M_1536k.cu8
Registering demod for TFA_1 KlimaLoggPro
Registering demod for TFA_2 sensors, 17240 bit/s
type 0x1: Samples per bit: 22.3
Registering demod for TFA_3 sensors, 9600 bit/s
type 0x2: Samples per bit: 40.0
Dumpmode -1 (LOAD), dumpfile ../../rtl_433_tests/tests/TFA-KlimaLogg-30.3180.IT/01/g001_868.25M_1536k.cu8
TFA1 ID 3f14 +18.9 43% seq 9 lowbat 0 RSSI 82
done reading dump

[alxlo]

Seriously, I would be more than happy to let Amazon ship one or two of these sensors to you or help you in any other way.

[merbanan]

There is now a PR for this sensor https://github.com/merbanan/rtl_433/pull/1319. Try it out you might need to play with the -l parameter.

[merbanan]

@alxlo can you ship me some development time? :) Anyway these sensors does not fit into the current generic dsp model. The current code might work with a samples rate of -s 1536k and -l in the range of 5000-15000.

[alxlo]

With e. g. -g 30 -f 868250000 -l 15000 -s 1536k -R 150 #1319 provides quite consistent results with a sensor in the same room as long as -g is not to high. The outdoor sensor (15m and 3 brick walls away) gives no results, even with max gain and any amount of fiddling with -l

@merbanan If you find a means for shipping dev time I gladly would :-) The effort and time you invest in this project is greatly appreciated!

[mueckinger]

With -l 15000 I receive 2 out of 6 sensors (wooden wall). With -l 24000 I receive 2 others (brick wall and wooden ceiling) but not the ones above. (Documentation says -I 16384 is maximum !?!) Any idea why I cannot receive 4 sensors with same settings? Actually my Klimalogg base station is able to receive 5/6 sensors. So I am missing one sensor. -g was set to 20 (19.2dB seems to be the max gain of my tuner).

[zuckschwerdt]

@mueckinger using RTL-SDR (8 bit) the maximum is 32768 with full scale at 16384 (amplitude, i.e. linear), for SoapySDR the maximum is 22144, full scale 16384 (magnitude, i.e. log). Everything above full scale is a distorted signal. (think of full scale as sine wave at full amplitude).

[merbanan]

As I said the current dsp code is not suited at all for this sensor. I am quite sure this is not the proper way to demodulate this signal. Ther must be a better way.

And I have no idea on how to get rid of the -l option.

[zuckschwerdt]

The Spectrogram looks like FSK with those two dominat frequencies, which is strange. Looking at Sigrok Pulseview the samples look like OOK (or ASK). A simple moving average threshold seems to cut the levels well. Look at .e.g. 02/g013 -- that's FSK, two frequencies and two strong harmonics plain visible. But the amplitude carries a clean preamble where the frequency looks random. The sample is clipped badly (too much gain) but the FSK looks more like a clock than data? Not sure I understand the signal.

All samples hit the DC, which is not great. Can we get a sample at lower gain and with a frequency offset? E.g. -f 868M. 02/g006 has the right gain -- in https://triq.org/iqs/ white pixels need to be rare.

[zuckschwerdt]

@merbanan I've added an old demod evaluation example, update for the native sigrok writer with f5e5a6a. I find it useful to quickly try some algorithms there and visualize the output.

[merbanan]

@zuckschwerdt I mainly prototype things in Octave. Anyway the tfrec code has this:

//-------------------------------------------------------------------------
// Vastly reduced mixed demodulator for FM-NRZS:
// It doesn't need to be linear, nor do we care about frequency shift direction
//-------------------------------------------------------------------------
int fm_dev_nrzs(int ar, int aj, int br, int bj)
{
    int cr=ar*br+aj*bj;

    // This limits also the max RSSI
    if (cr>1e9)
        cr=1e9;
    if (cr<-1e9)
        cr=-1e9;
    return cr;
}

ar = I[n] aj = Q[n] br = I[n-1] bj = Q[n-1]

I'm not sure if it makes sense to do any further work. At least without the real sensor. And currently I'd rather work on enabling 2 fsk demods and some other parts that I need so I'm not sure sending a sensor to me is the best use of resources.

The signal could be dual tone modulated. But this puzzles me much. We might have to look at the transceiver chip or ask an expert.

[zuckschwerdt]

That's a very fast phase difference estimator. But it's amplitude sensitive. Usually you want arg(x[n] × conj(x[n-1])), e.g. here atan2(aj×br-ar×bj, ar×br+aj×bj)

So they are looking at the FSK. I can't seem to get anything from that. At least with the DC hitting samples. I wouldn't expect a OOK to be visible? Maybe an artifact of DC correction or such. We do need samples at lower gain and with a frequency offset.

[zuckschwerdt]

Ah, wait, with that crude estimation they (unintentionally?) do look at the OOK…

[merbanan]

The calling code calculates how many 1-bit duration's fit between the "OOK" pulses. I do the same in my code but this is unfiltered whole rtl_433 uses and IIR filter.

cr=arbr+ajbj; -> i[n]i[n-1] + q[n]q[n-1] -> ~ I^2 + Q^2

That is the same as rtl_433 uses to demodulate OOK.

For some reason that works, but looking at a frequency spectrum one can see 2 frequencies present. But looking at the waveform both frequencies are present at the same time at least when one of the bits are transmitted.

[zuckschwerdt]

If you use x[n] and x[n-1] then you are looking at (FSK) phase differences. If you want a (OOK) amplitude that's xr[n]^2+xj[n]^2, and for the magnitude it's sqrt(xr[n]^2+xj[n]^2).

[merbanan]

I know I just thought that at 1.5Ms that n~=n-1. But the difference might be 4. Anyway I figured out the signal. Octave code follows:

m=0:0.0003:3*2*pi;
l=0:0.0001:2*pi;
sig=sin(l)+sin(m);
plot(l,sig)

Klimalogg_signal.pdf

This matches what is seen when looking at the signal visually. And it makes sense for this type of device. There is a carrier always transmitted at Y Hz when a one symbol is transmitted and 3*YHz+YHz (actually modulated 2 times) when a zero symbol is transmitted (with a slowly drifting phase). What can this be called, maybe OOK added with a carrier. From the Audacity frequency analyzer I get 555Hz on the lower frequency and 1670Hz on the high frequency. This confirms the frequency difference of 3 times.

So what to do from here?