bemasher / rtlamr

An rtl-sdr receiver for Itron ERT compatible smart meters operating in the 900MHz ISM band.
GNU Affero General Public License v3.0
2.19k stars 249 forks source link

Support for Badger ORION RF water meters? #163

Closed CFSworks closed 3 years ago

CFSworks commented 3 years ago

Hi! I have a Badger Meter smart water meter (FCC ID GIF2002A). I reverse-engineered the radio protocol; it transmits at 916.45MHz±100KHz ("DC-balanced" FSK at 100Ksym/s). I realize this requires pretty different DSP from the OOK of the ERT protocol.

The transmissions occur every 4-5 seconds. The interval seems to be randomized in that range, probably as a means of collision avoidance. The data consists of a 32-bit sender ID (at least, I think it's an ID - it's unchanging for my meter, but it might include type/version information) followed by a 32-bit consumption value (equal to the gallons displayed on the meter's mechanical face). There's a 16-bit CRC and no error correction.

Rather than describe the rest of the protocol, I wrote a Python script that can be used to generate the baseband signal for a given sender/consumption message (and with -v, prints the encoding steps). The output is in the HackRF IQ format (each sample is 2 bytes, sint8 i,q;) rather than RTL-SDR's uint8 format. The sample rate is 1 Msps.

Would this be something of interest in rtlamr, or does the different modulation scheme put this too out-of-scope?

bemasher commented 3 years ago

Would you be willing to provide signal captures of your meter?

It might be too far out of scope since rtlamr is designed to be relatively agnostic to frequency. Users will almost certainly have to limit the sample rate and listen on a specific frequency for their meter. Helping users determine this frequency and appropriate bandwidth costs a lot to support as well.

CFSworks commented 3 years ago

Would you be willing to provide signal captures of your meter?

Use the -o option with the Python script I provided above. This will provide you with a signal capture (1Msps HackRF format) for any sender ID and consumption value (in gallons) that you desire. You can zero-pad the capture and apply AWGN, CFO, SRO, fading, ... if you want to test with modeled error.

I can synthesize and post some example .iq files (w/ error) for testing purposes if you'd like. I'm reluctant to provide captures from my meter specifically, but I don't think they're necessary.

Helping users determine this frequency and appropriate bandwidth costs a lot to support as well.

The frequency doesn't have to be determined. These meters always transmit at 916.45MHz, per the FCC filings. While decoding the protocol, I have been using a fixed bandwidth of about 200KHz per sideband which seems to result in a pretty reliable decode.

Adminiuga commented 3 years ago

@CFSworks do you have a decoder python script you are willing to share? I've just got Hackrf One and was looking into reading my water meter too.

@bemasher my water meter is https://fccid.io/GIF2006B and per FCC it can be programmed to use frequency hopping. I assume most installation would use channel hopping vs a single operating frequency, but need to confirm. Having FSK decoder still could be beneficial and rtlamr can still stay agnostic to frequency. I could provide the recordings, but would prefer sending those to you privately.

CFSworks commented 3 years ago

@Adminiuga It's not a Python script, but I've updated the Gist linked in my first post with the GNURadio flowgraph (orion.grc) that I was using to study the protocol. I didn't share it initially because it makes a few assumptions that I haven't verified (such as treating sender as a single little-endian uint32) so please don't study it if you're trying to learn the protocol. :)

You'll probably want to replace the input to the DC Blocker with the osmocom source block to drive the HackRF directly; otherwise it will read /tmp/baseband.iq (which you can capture ahead-of-time with hackrf_transfer).

Since/if your meter is frequency-hopping, you'll need to do some further modification to get the flowgraph to cooperate. My Orion receiver block doesn't assume a nice centered frequency (it accepts a preamble at any offset), so you might be able to get away with just increasing the sample rate, and widening the filter and PLL bandwidth, at the expense of poorer noise tolerance. If that doesn't work, another thing to try might be a carrier-tracking PLL with a relatively low loop bandwidth (to search for the carrier frequency the meter chooses to use), and multiplying the conjugate of the recovered carrier into a delayed version of the original signal, prior to the low-pass filter. Let me know if you have trouble and I'll see if I can modify the flowgraph to tolerate frequency hops.

Adminiuga commented 3 years ago

Thanks! Apparently i'm not there yet, but SDR tutorial seems like a good place to start, and your flow would be a great help.

CFSworks commented 3 years ago

Assuming your meter is picking a random frequency on each transmit (and the protocol is otherwise the same as what I have), here are the modifications to make to the flowgraph:

  1. Increase samp_rate to 11M, and decim to 22 (i.e. so both go up by a factor of 11). This is necessary to capture a wide enough band that you can pick up the meter on any frequency in the 911.62-921.25 MHz range.
  2. Make sure your HackRF source block is tuned to a center frequency of 916.435 MHz (i.e. in the middle of that range) and connected to the input of the DC Blocker.
  3. Move the PLL Freq Det before the low pass filter block (but after the DC Blocker) and change its loop bandwidth to 0.05, also remove the /decim from the frequency limits and replace fsk_deviation_hz with 4.815e6 (to accommodate the whole range).
  4. Put the low pass filter block after the PLL Freq Det. You will also have to change its type from Complex->Complex to Float->Float.
  5. Lower the threshold of the Orion receiver to something like 0.05

Then try placing the antenna somewhat near to the meter (noise tolerance will be poor), keep lowering the Orion receiver threshold until you get some output every ~4-8 seconds, and adjust the PLL's loop bandwidth to achieve minimum error rate.

Adminiuga commented 3 years ago

Thank you for all your help, I really appreciate it. I still need a bit more time, before I could ask intelligent question. I've tried the modifications as you described in details https://gist.github.com/Adminiuga/6ea120d198d915729cc99833d37d5ac8 but i'm getting this error and some spurious "OOO"

OOOOOOOOOO/usr/lib/python3/dist-packages/numpy/core/fromnumeric.py:3372: RuntimeWarning: Mean of empty slice.
  return _methods._mean(a, axis=axis, dtype=dtype,
/usr/lib/python3/dist-packages/numpy/core/_methods.py:170: RuntimeWarning: invalid value encountered in true_divide
  ret = ret.dtype.type(ret / rcount)
OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
CFSworks commented 3 years ago

O means the HackRF source block detected an overflow (and hence discarded some samples). Since it's happening only occasionally, it probably just means that your system isn't quite fast enough to process 11 Msamples/sec consistently - it shouldn't cause the flowgraph to break outright, though.

If you aren't seeing anything even after several minutes, there's a good chance your meter uses a protocol different enough that my blocks don't recognize it. I'd recommend either investigating in URH or sharing a signal capture (here is my GPG/PGP public key if you want to keep the capture private).

Adminiuga commented 3 years ago

I put my hackrf next to my water meter in the basement, and I'm a bit confused. First of all, I do get some readings now and

  1. It is always the same sender
  2. consumption increases, for now I only got a delta of 3 in one hour, would need to check again tomorrow/later

BUT:

  1. Nor the "Sender" nor "consumption" matches the reader ID. I'm going to try to corelate the delta between a few days. I know the actual transmitter was replaced, but meter stayed the same.
  2. I've added a FFT sink and what really confuses me: the time of getting a reading does not coincide with the time I'm getting a signal on FFT like at all.

the #2 makes me think that I'm peaking some other meter, because until the HackRF was moved next to the transmitter, I wasn't getting such a strong signal in fft sink and per FFT it does look like an FSK signal, cause I do get two distinct spikes.

But I also have another, somewhat unrelated question. I did a IQ recording with osmocom_fft using 916.45MHz as the center frequency and got something which looked like PSK + FSK modulation image

So then I did another recording with an offset center freq around 916.1MHz and got a normal looking FSK image

Can you confirm if that phenomena was because of the center frequency?

Initially I thought it was cause by the center_freq selection, but I'm in doubt now, because on 916.45MHz I do get two distinct "signatures". One looks like this:

  1. image

  2. the second one (the one with PSK?) looks like image

So do I really get two different signals?

PS: my apologies @ bemasher for stealing the thread. I hope you don't mind. Feel free to unsubscribe from this one, if you didn't do so already :)

PPS: @ CFSworks Thanks for all your feedback and help. I really appreciate it. This is more of a "curiosity" project for me and I might be asking too many questions, since I'm really a novice in SDR. I'll try my best not to overburden you with too many questions, but feel free to ignore at any time

Adminiuga commented 3 years ago

I've compared the delta in consumption reporting by "orion" and actual meter reading and it matches 👍 But I think the "sender id" is not computed correctly. let's say the sender id is 1104737137 ==0x41dddddd, but the actual meter id (i have mine labeled on the wire) is0x4dddddd` then the decimal representation matches the actual meter ID labeled on the transmitter

Other than that: I still don't see the signal in live spectrogram which triggers the reading and apparently I have two different much stronger signals on 916.45MHz and now I wonder were those are coming from. One of the signals is FSK modulated, so it cannot be my Electric meter, but it could be the Gas meter. But what is the second signal I have no ideas.

Adminiuga commented 3 years ago

and actually the reported consumption does match the dials on the meter, but the most significant bytes. My meter has 6 digits, so maybe for my meter the consumption is 24bit and the significant byte is the number of overflows? It is going to be a while till my meter overflows again.

It would be lovely to see this decoder in RTLAMR

CFSworks commented 3 years ago

My blocks don't assign any numerical meaning to "sender" - it's just a field that grabs the first 32 bits of the transmission, which appears to be fixed for a given meter, and can hence uniquely identify which meter sent the message. I'm not even sure if all 32 are meant as a "serial" as some might be type/model/version info.

I doubt that the high bits of "consumption" are overflows. Those bits are 0 for my meter, which has likely been installed for longer than yours as it's an older model. This might instead be a tamper counter or other similar, miscellaneous "system health" info.

I'm wondering if a good way to get this supported in rtlamr is to use FFT as a coarse frequency acquisition method, then go back and use frequency-translating, a low-pass filter, and downsample prior to demodulation? Should help boost SNR (even in the OOK cases) while keeping the configuration frequency-agnostic!

On Tue, Jan 19, 2021, 09:18 Alexei Chetroi notifications@github.com wrote:

and actually the reported consumption does match the dials on the meter, but the most significant bytes. My meter has 6 digits, so maybe for my meter the consumption is 24bit and the significant byte is the number of overflows? It is going to be a while till my meter overflows again.

It would be lovely to see this decoder in RTLAMR

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bemasher/rtlamr/issues/163#issuecomment-762951954, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFLCYH5NYUXUQG4GZAADETS2WWFXANCNFSM4VOCT4HQ .

Adminiuga commented 3 years ago

Last question: should I be able to see the transmission in the "QT Frequency Sink"? I'm now confident your decoder does display data from my meter, but why I'm unable to catch the transmission in FFT sink? Is it too short or am I doing anything wrong? It didn't make a different for hackrf being 1ft or 3ft away from the meter. When I get the reading, the FFT doesn't show anything above the noise floor. And whenever I see the strong signal in FFT, there's no output from the decoder

CFSworks commented 3 years ago

If you have questions about SDR in general, I'm happy to answer them - but you should probably email me (my username, gmail) to avoid off-topic chatter on this issue ticket. I think the ticket should pertain only to figuring out the Orion protocol, and discussing ways to receive it effectively.

To answer your question: the Orion transmissions are only about 1.6ms long, and the FFT sink updates at a rate of about 10/sec by default. If you want to see the transmissions, I suggest a waterfall sink running at 100/sec with a window size large enough to include 10ms worth of samples.

(PS in addition to an SDR tutorial, check out my "wavebird-reversing" repo on my GH user page)

On Tue, Jan 19, 2021, 10:14 Alexei Chetroi notifications@github.com wrote:

Last question: should I be able to see the transmission in the "QT Frequency Sink"? I'm now confident your decoder does display data from my meter, but why I'm unable to catch the transmission in FFT sink? Is it too short or am I doing anything wrong? It didn't make a different for hackrf being 1ft or 3ft away from the meter. When I get the reading, the FFT doesn't show anything above the noise floor. And whenever I see the strong signal in FFT, there's no output from the decoder

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bemasher/rtlamr/issues/163#issuecomment-762988905, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFLCYFOQTGYMUZBVFG7RN3S2W4XLANCNFSM4VOCT4HQ .

mexdfr commented 3 years ago

Hi, I have an orion water meter https://fccid.io/GIF2006B[](url) and a RTL-SDR Blog V3 R820T2 RTL2832U https://www.amazon.com/gp/product/B011HVUEME/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1 in a pi4 trying to read the meter transmmisions.

I follow @CFSworks in https://github.com/bemasher/rtlamr/issues/163#issuecomment-761560072 and I get a error when running:

Invalid sample rate: 11000000 Hz

My graph is:

image

Version: GNU Radio Companion 3.8.2.0 (Python 3.7.3)

Can I get help with this error? Im really new with radio

Thanks

CFSworks commented 3 years ago

Hi @mexdfr,

I really don't think that this issue ticket is the right venue for help with those flowgraphs, since the primary focus here is whether or not to include support for the ORION protocol in rtlamr. (The secondary focus is in getting the protocol better understood.) Your enthusiasm about decoding your water meter is great (and I hope @bemasher takes note of that, hint hint), but this question is too general. Please either email me or use the comments section on the gist I posted instead. :)

That being said, the error Invalid sample rate: 11000000 Hz is the RTL-SDR driver rejecting the configured sample rate. The RTL-SDR supports sample rates up to 3.2 Msps (but in practice is only stable up to about 2.5 Msps) so 11 Msps is just too fast for that particular device. (I might recommend investing in a LimeSDR-Mini if you really need the increased bandwidth.)

But back on topic: It sounds like these multi-frequency (GIF2006B) meters are fairly common, and to be generally useful, we'd need to be able to decode them with only the bandwidth provided by a basic RTL-SDR. The FCC test report says "The test item could be programmed to operate in one of the following modes: transmit at 911.65MHz, transmit at 916.45MHz, transmit at 921.25MHz, or frequency hopping enabled." So maybe instead use the flowgraph I posted (with none of the modifications I suggested) to sniff on each of those frequencies for a few minutes?

And even if it's set to frequency-hopping mode, there are only 25 hopping channels (page 51 shows them clearly) so if frequency-hopping is enabled, waiting patiently on one channel for several minutes ought to pick up a transmission or two.

bemasher commented 3 years ago

There are no plans to support frequency modulated messages.