search

merbanan / rtl_433

Program to decode radio transmissions from devices on the ISM bands (and other frequencies)
GNU General Public License v2.0
6.11k stars 1.32k forks source link

Writing basic flex decoders #1929

Closed frevoli closed 4 months ago

frevoli commented 2 years ago

Hello all. Having written and used a few basic flex decoders via the -W flag I thought it would be good to add them to the main list of decoders properly. I followed an excellent online guide which was going perfectly until it told me to add my decoder to the list in src/Makefile.am but I can't find this directory anywhere. If someone could tell me what I'm missing it would be most appreciated! Many thanks, Matt

zuckschwerdt commented 2 years ago

Just write flex decoders in .conf files, like we have here: https://github.com/merbanan/rtl_433/tree/master/conf Then load with -c or using config_file from the main .conf (example here: https://github.com/merbanan/rtl_433/blob/master/conf/rtl_433.example.conf )

frevoli commented 2 years ago

Excellent thank you. I think the guide I was following was attempting to integrate the decoders with the main list but if this works too then that is also great. Much simpler! One more question - if I have several of these flex decoders I will end up with a VERY long command. Is there any way to roll them all into one conf file and just call that single one? Thanks again

zuckschwerdt commented 2 years ago

Yes, you can create multiple flex decoders in one conf. The example conf has a few one-liners at the bottom, but best if you use the style of the dedicated examples. E.g. sections of

decoder {
    name=...
}
...
frevoli commented 2 years ago

Thanks Christian. Great advice from an excellent community. Cheers

zuckschwerdt commented 2 years ago

Glad I could help, and nice to see the flex decoders are a working tool to help on projects!

At some point I hope to add more features to the flex syntax, e.g. checksums, some specific decodings, and maybe branching.

frevoli commented 2 years ago

I find they are a really good way of getting a custom decoder onto the tool quickly without having to submit for official repo contribution. Having said that, I'm more than happy to contribute if the decoders will benefit the community. I'm still new to rtl_433 and all I've managed to do with my flex decoders so far is recognise and detect my devices. My next task is to fully decode so that I can see ID, data etc. I expect that will be easier said than done but theres some great documentation on the wiki.

zuckschwerdt commented 2 years ago

We always welcome adding more flex confs, either PR (so you get credit) or just paste the file. Documenting the conf is probably the most helpful for other users to quickly adapt something.

Once you got the raw bits from some transmission it's mostly a trial and error to figure out which bits do what. If you want to share a number of codes to discuss decoding maybe try BitBench.

frevoli commented 2 years ago

Does the callback and r_device code go into the flex conf file in the same way that it does a regular ".c" device file? If that makes sense! Thanks again

zuckschwerdt commented 2 years ago

Just what you see in the examples and with the help from rtl_433 -X Specifically not the callback (decode_fn), disabled, and fields.

frevoli commented 2 years ago

Hi Christian, This is a daunting task for the newcomer but it looks as if once you have reversed one device the process will be easier to do more. I will be using URH to work out what all the data fields are. For my application I only need to know the unique ID of a transmitting device so that should be possible. Although I'm not sure if this is possible fir devices with rolling codes. I will keep you posted with my progress! Thanks again Matt, UK

zuckschwerdt commented 2 years ago

If you have the codes, or the pdv link from -A or a good sample file from -S unknown then just post here and we'll help out.

frevoli commented 2 years ago

Brilliant thank you. I'll be collecting some samples next week. I'll put one or two on here as any guidance would be greatly appreciated. Once I've been shown once I hope to be ok to do more. Thank you! : )

frevoli commented 2 years ago

Actually Christian, I have one device here that I could quickly get a few PDV links for. Bear with me and I'll see if I can get some info across in a bit 👍

frevoli commented 2 years ago

Hello again mate.

Ok... I have a Visonic sd-304 c pg2, which I believe is an alarm shock sensor. 868MHz. I ran an -s unknown capture and inserted the battery. It recorded about 10 files. I ran -A on one single file and these are the pdv links it produced...

https://triq.org/pdv/#AAB0190701000A001D00310099005B004327118191A59292B581818455+AAB0180701000A001D00310099005B00432711819281819191C1A655

https://triq.org/pdv/#AAB10311A386F6B07381818255 https://triq.org/pdv/#AAB0180701000A001D009800450032006E2711818391818181848555+AAB0130701000A001D009800450032006E27118383A655

https://triq.org/pdv/#AAB10311A286F7B0738181818255

https://triq.org/pdv/#AAB0200701000B0048001D00310064009927118282828282828282828282828282829455+AAB0140701000B0048001D0031006400992711A38282B455+AAB0130701000B0048001D0031006400992711A3828155+AAB0120701000B0048001D0031006400992711838155+AAB0140701000B0048001D0031006400992711C1B383A455+AAB0330701000B0048001D0031006400992711A38391A3A3828282C28283928182A2B1A3A3D1828281D282A2A2C2B28382A38381A3A655

https://triq.org/pdv/#AAB0200701000A0048001D0031006500A027118282828282828282828282828282829455+AAB0140701000A0048001D0031006500A02711A38282B455+AAB0130701000A0048001D0031006500A02711A3828155+AAB0120701000A0048001D0031006500A02711838155+AAB0140701000A0048001D0031006500A02711C1B383A455+AAB0300701000A0048001D0031006500A02711A38391A3A3828282C28283928182A2B1A3A3D1828281B3828282A2A2C2B2838555+AAB0120701000A0048001D0031006500A0271181C655

https://triq.org/pdv/#AAB0200701000B004B001D00310099006B27118282828282828282828282828282829555+AAB0140701000B004B001D00310099006B2711A38282B555+AAB0130701000B004B001D00310099006B2711A3828155+AAB0120701000B004B001D00310099006B2711838155+AAB0140701000B004B001D00310099006B271191B383A555+AAB0320701000B004B001D00310099006B2711A38391A3A3828282928283928182A2B1A3A3C1828281B1A282A2A292B283839381D655

https://triq.org/pdv/#AAB0200701000B0048001D00310065009927118282828282828282828282828282829455+AAB0140701000B0048001D0031006500992711A38282B455+AAB0130701000B0048001D0031006500992711A3828155+AAB0120701000B0048001D0031006500992711838155+AAB0140701000B0048001D0031006500992711C1B383A455+AAB0330701000B0048001D0031006500992711A38391A3A3828282C28283928182A2B1A3A3D1828281B2838282A2A2C2B283829181C655

https://triq.org/pdv/#AAB0200701000B0048001D00310063009827118282828282828282828282828282829455+AAB0140701000B0048001D0031006300982711A38282B455+AAB0130701000B0048001D0031006300982711A3828455+AAB0120701000B0048001D0031006300982711838155+AAB0140701000B0048001D0031006300982711C1B383A455+AAB0110701000B0048001D0031006300982711A455+AAB0300701000B0048001D003100630098271191A3A3828282C28283928182A2B1A3A3D182828182B38282A2A2C2B28381A3A155+AAB0110701000B0048001D00310063009827118655

https://triq.org/pdv/#AAB0230801000E005B00220032008C001900412711828585858585858582858585858585859155+AAB0160801000E005B00220032008C001900412711A38585B155+AAB0150801000E005B00220032008C001900412711A3858155+AAB0140801000E005B00220032008C001900412711838155+AAB0160801000E005B00220032008C00190041271193B383A155+AAB0290801000E005B00220032008C00190041271186A3B3A3A3858585958583958682A5B6A3A3C68585868155+AAB01D0801000E005B00220032008C001900412711858585A5A595B58385A58455+AAB0150801000E005B00220032008C0019004127118585B755

https://triq.org/pdv/#AAB0230801000D0046002400310066009D00192711828686868686868686868686868686869455+AAB0160801000D0046002400310066009D00192711A38686B455+AAB0150801000D0046002400310066009D00192711A2868155+AAB0140801000D0046002400310066009D00192711838155+AAB0160801000D0046002400310066009D00192711C1B383A455+AAB0290801000D0046002400310066009D0019271181A3B1A2A3868686C68683968186A6B1A3A2D18686818455+AAB01D0801000D0046002400310066009D00192711A686A6A6C6B6838686868455+AAB0150801000D0046002400310066009D001927118686B755

There will obviously be many more pdv links from the other files but I didnt want to send you all of them as that would be a lot of stuff to look at! Are you able to give me any pointers as to where to start with analysing this lot? As I said, I'm really only interested in building a flex decoders that will detect the transmission and tell me the ID of the device.

Any help greatly appreciated.

Thanks again Matt

zuckschwerdt commented 2 years ago

That does not much look like the data we'd expect. The first ones are short enough for a alarm sensor but seem like random interference. The later ones have too much data and look more like perhaps TPMS.

Best to capture without the antenna and then upload zipped .cu8 https://triq.org/rtl_433/ANALYZE.html

zuckschwerdt commented 2 years ago

You want something that looks like this: https://triq.org/pdv/#AAB10405180228039033BC828282829282928282828282829282928282928282929282829292929282828282929282829292928282929355

or more likely like this:

https://triq.org/pdv/#AAB021031B045801442CC48190908190909081908190819081819081909090818181819255+AAB0200301045801442CC481909081909090819081908190818190819090908181818255 (roughly the same as above but repeated, as those sensors do.)

frevoli commented 2 years ago

Hiya. I didn't have the antenna connected. I placed the tx directly next to the rx. And TPMS wouldn't be present at 868 would it? I'm wondering if the device needs to be paired with an alarm panel for it to transmit anything meaningful. This one is simply new out of box with me inserting a battery. What do you think?

merbanan commented 2 years ago

That is probably needed.

frevoli commented 2 years ago

Ok great thanks. In that case I'll dig out an accompanying panel and see what I can capture. Appreciate the support. Have a good day. Matt

frevoli commented 2 years ago

Hello again, hope you're having a good start to 2022. I was thinking it would probably be smart to learn to walk before I can run, so I've put the alarm sensor devices on hold and targeted a simple door bell instead. The following is a link to the pdv output. I have written a flex decoders for this based on my limited understanding. I would be very interested to see what your flex decoders would look like for this device. If you are happy to share what you would have done for it with explanations that would be very useful to help me. Thanks again Matt

https://triq.org/pdv/#AAB02B0801009C0254005402FC01041900045C271483949494819494839481949483A181839494A194818194948555+AAB02B0802009C0254005402FC01041900045C27148194949481949481948194948181818194948194818194948555+AAB0170801009C0254005402FC01041900045C2714819494969755

zuckschwerdt commented 2 years ago

I guess that would just be PWM with 150/600 µs, e.g. -X 'n=name,m=OOK_PWM,s=150,l=600,r=1000'

frevoli commented 2 years ago

Perfect thanks. And do you think there's a device ID buried in there somewhere?

zuckschwerdt commented 2 years ago

Yes, it's all ID in fact. That's a EV1527 encoder (or similar) with 24 bit ID. The last bit is sync (code length is {25}).

frevoli commented 2 years ago

Oh really. Ok. Is there an EV1527 decoder built into the main repo? Perhaps it will detect my doorbell

zuckschwerdt commented 2 years ago

The timings vary in a wide range, and you really want to match your ID. See the conf folder for some examples.

frevoli commented 2 years ago

Will do. Thanks again

frevoli commented 2 years ago

Hello again,

The conf directory is very helpful as it shows me that getting ID is possible. Many decoders resolve the device ID with a line similar to the example below...

get = ID:@20:{4},

What is this line doing and what does it mean? Is it referring to a section of the binary data string in the bit buffer? I think that if I can understand this then I will be able to make good progress with my custom flex decoders.

Many thanks Matt

zuckschwerdt commented 2 years ago

get = ID:@20:{4},

In the bitbuffer start at 20 and read length 4. So you'd want get=id:@0:{24},

But really all the bits are the ID, so the code {25}1234568 is already the ID. Minus the 8 at the end perhaps, which is the sync bit and will always shows as an 8 (binary 1000).

frevoli commented 2 years ago

Thank you that has helped a lot. I have written a conf file to decode an alarm door contact and am successfully able to output the alarm state, tamper state and ID. Very happy! However, each received burst contains 6 rows and so I end up with 6 duplicated lines on the screen. Can I change anything so that I only see one row of data for each capture? Thank you Matt

zuckschwerdt commented 2 years ago

These might help,

frevoli commented 2 years ago

How strange. I was going to try those today! Good to get confirmation that they are the correct commands to remove the duplicates. Many thanks my friend!

frevoli commented 2 years ago

Just thought I'd give you an update... the suggestion you made regarding unique has worked to hide duplicate print lines thank you. Having "cracked" that alarm device I have moved onto another. I have used the -A flag to observe the output and have worked out that the unique device ID is in rows 00-02 of the bit buffer. How can I command my decoder to look at JUST these rows? Thanks again Matt

zuckschwerdt commented 2 years ago

Usually rows should be repeats of the same packet for reliability, like you observed before. Only complex protocols will use different rows. What device is that and what are example rows?

frevoli commented 2 years ago

I get several rows. In fact it reaches 49 and then prints "... Maximum number of rows reached. Message is likely truncated".

frevoli commented 2 years ago

On closer inspection I am getting six rows with different binary output, repeated over and over. So I guess you could say I am just dealing with six rows. However the question remains, how do I get at data from specific rows?

zuckschwerdt commented 2 years ago

Different rows generally means your timings are broken. It should have been a single row.

frevoli commented 2 years ago

Hmmm ok. I used the timings suggested for a flex decoder from the -A output. Do you think I should experiment with them?

zuckschwerdt commented 2 years ago

Inspect in pdv and choose proper ones, don't just guess.

frevoli commented 2 years ago

Christian, looks like you've assisted with this particular device before...

https://groups.google.com/g/rtl_433/c/1yVOs7yoLw0/m/dkC3xLFfBwAJ?pli=1

frevoli commented 2 years ago

You see how the other chap received several rows too?

zuckschwerdt commented 2 years ago

Ah ok, a "Yale 6010 door/window sensor". I guess we never figured out what that stream of maybe-bytes means. Drop that 6010.ook in pdv, should look like the data you have. Put that in a BitBench and it won't reveal much, there is a pattern but not sure what it means.

No single row will have a meaning. If you want to experiment with the data, then read it with e.g. a Python script. We some examples.

zuckschwerdt commented 2 years ago

Here is the BitBench data from the older example. Looks like a message is made up of 6 packets. The first 3 (or 3.5) bytes are maybe the ID, then the data "thins out", finally there is a high entropy blob. The thinning indicates state bits, the blob is a checksum.

See decoded in this BitBench.

edit: The checksum is just remainder of adding the 5 messages bytes, i.e. adding 6 bytes checks to zero.

Try to get similar data and confirm the pattern or post the codes (send to me by mail if you don't want them public). Then I'll code support with a proper decoder module.

frevoli commented 2 years ago

Interesting. I'll take a deeper look at your suggestions. Although it looked like Peter managed to ascertain that rows 00-02 were the device ID, and that will be pretty much adequate for my application. I was wondering how to extract those rows of bits in my flex decoder? I will also take a look at your bit bench findings too

zuckschwerdt commented 2 years ago

Row 0+1 are the ID, Row 2 should be the device type. ac1 and ad1 are a window sensor, 153 is a PIR.

zuckschwerdt commented 2 years ago

There is now a experimental decoder. Please check different sensors and events and report what you get.

Would be interesting if the guess on sensor type and event is correct, and what's in state (battery?).

Guessed data so far:

And really interesting would be data from

Send me privately by mail if you like.

frevoli commented 2 years ago

I have added the support for Yale HSA files but when I execute rtl_433 -R 210 I get "Protocol number specified (210) is larger than number of protocols". My protocol list appears to stop at [207] SmartFire ProFlame 2 RC

Edit - I think I might know why this is! Will update...

zuckschwerdt commented 2 years ago

Fresh compile and install needed, see https://triq.org/rtl_433/BUILDING.html#linux-mac-os-x

frevoli commented 2 years ago

Yes that was the issue. All ok now. I can confirm your guesses are correct - Event 01=trigger, 04=tamper. Unfortunately I do not have any more hardware other than a door contact. But thank you for adding support for this device so quickly.

frevoli commented 2 years ago

Is it possible to write a Yale flex decoder that will print ID, Type, State, Event much like your proper decoder? I am struggling to work out how to tell it what row to look at. For example if I wanted the first 8 bits from row 3, could I do something like this?...

get=State:[3]:@0:{8}

zuckschwerdt commented 2 years ago

Sorry, not with just the flex spec. The rows are not really numbered, the transmission could start at any point. You'd need to look at the end-of-message bit, then use the previous 6 rows, then compute and verify the checksum. You can do that in a script, but not with just a flex spec.