Motorola Type II SmartZone decoding improvements

[ilyacodes]

Is your feature request related to a problem? Please describe. Motorola Type II SmartZone functionality currently decodes only a small subset of control channel messages. This is sufficient for basic trunk tracking, but it could be much better, both for debugging live systems and for potential future features that require a deeper understanding of system state (e.g. patches, etc).

Describe the solution you'd like I am currently working on setting up my own Type II IntelliRepeater "site" on the bench, and will be using it to identify opcodes and add parsing for them. At a minimum, this will enable them to be logged for debugging and manual inspection.

I plan to contribute the findings back to the project, but have the following question: what is the most efficient way for me to contribute?

• One PR per new message discovered?
• One large PR, but each message as its own commit?
• One giant PR when I'm all done?

I would definitely like to make this the least headache for everyone involved, so I'm asking before I get started.

Describe alternatives you've considered This isn't really applicable, I just want to know the most efficient way to contribute.

Additional context To be clear, I'm not asking you to implement anything, just asking how I can contribute to the project most efficiently.

[boatbod]

Ilya,Right now all development goes through the 'dev' branch,  then gets merged to 'master' and finally down to 'gr310'.My preference would be for you to clone from 'dev' and do your work there, then when a substantial block of work is tested and ready to publish, make a pull request and I can roll it up into the code base from there. Thanks, GrahamOn Mar 28, 2024 3:23 PM, ilyacodes @.***> wrote: Is your feature request related to a problem? Please describe. Motorola Type II SmartZone functionality currently decodes only a small subset of control channel messages. This is sufficient for basic trunk tracking, but it could be much better, both for debugging live systems and for potential future features that require a deeper understanding of system state (e.g. patches, etc). Describe the solution you'd like I am currently working on setting up my own Type II IntelliRepeater "site" on the bench, and will be using it to identify opcodes and add parsing for them. At a minimum, this will enable them to be logged for debugging and manual inspection. I plan to contribute the findings back to the project, but have the following question: what is the most efficient way for me to contribute? One PR per new message discovered? One large PR, but each message as its own commit? One giant PR when I'm all done? Describe alternatives you've considered This isn't really applicable, I just want to know the most efficient way to contribute. Additional context To be clear, I'm not asking you to implement anything, just asking how I can contribute to the project most efficiently.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

[ilyacodes]

Hi Graham - thanks, I've sent #209 your way for review. Sorry it's so big, but I just waited for a substantial block of tested changes as requested. Future PRs should be more manageable as they will be building on the many refactorings that took place in this first PR.

EDIT: moved to #210 because I was cleaning up branches on my end.

[boatbod]

I have merged the smartnet and mi/algid/keyid logging changes into branch "dev". I'd appreciate if you could give it some exercise to make sure it's good before I propagate forward into "master" and "gr310" branches. My ability to test smartnet locally is severely limited because the only system we have basically runs with zero traffic (it's a hot standby for our P25 system, but sees no usage).

I'll be honest that I'm not sure I like the changes to the terminal yet. Let me run it for a while and see if I can get used to it. The "control" and "secondary" stuff is rather distracting.

[ilyacodes]

Perfect, thank you for reviewing and merging!

I have some wideband SDRs (hackrf and Pluto) on the way so I’ll get some more data on it too, especially P25 and SmartZone running at the same time. I’ve also been running it nearly continuously on SmartZone (in various iterations) for the last couple weeks.

The biggest risk I see currently is the IDLE messages – I’m still trying to make sense of how they get interleaved with other multi-OSW messages, though I think it may be a matter of just living with an infrequent unknown OSW. It happens more on some systems than others.

One question: I don’t know how familiar you are with DSD-FME, but one option it has is to use a pre-recorded file as input instead of an SDR. Does op25 have anything like this? If so I can get some recordings which we can use as test cases, as it gets merged into production branches.

Re: the terminal, perhaps we can have that be a flag in the config file? I find it handy to see more info even when not using the web interface, but I agree it can be clutter if you don’t care about it.

[boatbod]

op25 has the capability to take input from sdr devices, native symbol files, .IQ and .wav files.

The symbol files are the easiest, and utilize "raw_input" and "raw_output" parameters in the channel section of the .json. The benefit is you can start/stop the capture from the terminal very easily and they bypass the whole demodulator chain on playback. The limitation is you can't tune, and it really doesn't work well in an environment like smartzone where you expect voice on one channel and the control chan on another.

.IQ and .wav file handling are included for compatibility with DSD+FL/DSD-FME recordings. Depending how the recordings were made depends how you need to configure op25 for playback. Check the iq_example and wav_example Readme's for more info.

[ilyacodes]

Thank you, I will read up on that and try it.

[boatbod]

For .wav the main challenge is adjusting the "wav_gain" parameter to get a nice signal. Recommend looking at the Datascope plot (eye diagram) to tweak that adjustment.

For .IQ the problem is knowing/identifying the tuning offset in the .iq file. FMPP does things one way, while FMPP24 does it differently. Some iq files have a header signature while others don't...
Take a peek in lib/iqfile_source_impl.cc for more hints. I usually fire up the fft plot and/or mixer plot to try to verify if I've got the "offset" parameter correct. The "iq_seek" parameter allows you to skip ahead in the file as necessary.

[boatbod]

All changes have been pulled and integrated into dev, master & gr310 branches.

[ilyacodes]

Is it possible to replay raw_input symbol files at faster than real time - i.e. disable the self.throttle on the channels? I notice that's a difference from DSD-FME that is a little annoying when I've been doing iterative testing over the same (large) raw_output data set.

I'm also curious whether you have looked at ISWs at all? I have been looking at mottrunk.txt as well as US patents [1] [2] [3] (among others) and am finding that there is a little bit of information about the ISW format, but the details of deinterleaving and CRC are basically documented as "For some unknown reason you must [do x]. Don't ask us why.". Is it fair to say that's basically the state of the art for Type II at this time?

I was able to get a recording of some ISWs on the bench though, so I've demodulated that and am looking at how to make sense of the resulting bitstream now.

[boatbod]

You could probably speed up the throttle, but I think disabling it completely might lead to timing problems where channel signaling arrives at a faster rate than it can be passed to python and processed by the trunking code. Best thing would be to try and see what happens!

I've not done anything with ISWs. My local smartzone system is for the most part dead (i.e. just a control channel idling it's time away) so having any traffic to monitor and debug would be a rarity.

[ilyacodes]

I've been able to successfully decode some ISWs!

Demodulation

Demodulation is fairly straightforward, I was able to do it in Universal Radio Hacker once I was able to get a decent recording [1].

Modulated data format

Bits	Single-word	Dual-word
10	1010101010 preamble	1010101010 preamble
78	ISW interleaved data	ISW1 interleaved data
6		010101 sync [2: Fig 11b]
78		ISW2 interleaved data

Fortunately we can correlate this against the relevant patent [3: Fig 11a] to confirm relevant bit counts.

Once the data is demodulated, we can just follow this flow chart in reverse, from bottom to top, to get the original 21-bit ISW data.

Deinterleaving

The same deinterleaving function that is used for OSWs works for ISWs, with a minor change: use 4 groups of 20 elements rather than 4 groups of 19 elements.

Incoming	0	1	2	3	4	5	6	7	8	9	...
Deinterleaved	0	20	40	60	1	21	41	61	2	22	...

I'm still not sure what to do with the final few elements where 78 obviously doesn't divide cleanly into 20... right now I'm trying the following (non-standard emphasised).

Incoming	...	70	71	72	73	74	75	76	77
Deinterleaved	...	57	8	18	38	58	19	39	59

Error correction

Now we have 78 bits in their original order - we can now check parity bits and perform error correction. The same error correction function used for OSWs works here too.

One exception: bit 37 (0-indexed) always reports as bad for me, even though my signal is known to be basically perfect. This leads me to wonder whether my assumption of how to de-interleave the last few bits was correct.

Error-corrected data format

Once we have error-correction out of the way, the data is a fairly straightforward set of 38 bits:

Bits	Meaning
16	Address
5	Command
17	CRC / error detection code

Auto-sync

Now we have 37 bits that have been error-corrected. At this point we face the auto-synchronization pattern that was added after the bits were encoded. My reading suggests that this is just an xor of the data with a pre-shared pattern known to both the transmitter and receiver [4].

Note: For this section, assume the data is left-aligned as shown below.

Based on known inputs and the helpful fact that the 5-bit command/opcode segment is known for ISW1 of a dual-word ISW [5: Fig 4b], I've determined the mask to be ACE328 for the 21-bit data segment. This is not quite a Barker-7 code [6] but shares similarities to combining several Barker codes in various orders and polarities:

Bits	0-15	16-20	21-37
Meaning	Address	Command	CRC / error detection code
Layout	AAAA_AAAA_AAAA_AAAA	CCCC_C	EEE_EEEE_EEEE_EEEE_EE
Mask	1010_1100_1110_0011	0010_1	Unknown

I've not yet experimented with extending the mask to apply to the 16-bit CRC portion of the ISW.

Error detection

Theoretically, once we finish applying the auto-sync mask, we can then do a CRC or CRC-like operation on the 21-bit data and compare it to the 17-bit error-detection value.

Since this is a different size from the CRC on the OSW, it is likely to be a different polynomial, and will require further research.

Adding automatic demodulation (op25 / DSD-FME)

I've dabbled at changing the SmartNet sync from AC to AA in both op25 and in DSD-FME, but so neither appears to sync and demodulate my actual ISWs. It does increase the noise of "sync" being "found" and getting garbage data though.

I'm hardly an expert in the demodulation logic - if there's something obvious you think I may be missing, please let me know!

Notes

[1] For future reference: this seems to be the sweet spot in SDR++ for recording signals that are usable for URH. Of course, adjust gain to match your setup.

[Carter121]

Hi, sorry for replying to an old thread but I couldn't find a discord link or anything in the wiki about what I'm looking for.

I was just curious how this performs compared to Unitrunker. I'm currently using Unitrunker and am looking for newer, open-source alternatives. Also, the readme says SmartZone requires two dongles, but can I use more than two? The list of talk groups I scan is quite large so I currently have 5 SDRs to avoid missing anything.

Thanks in advance!

[ilyacodes]

I'll have to leave that to @boatbod to reply on since I've mostly been focused on the technical documentation and implementation of new OSW opcodes :)

Welcome to the project regardless!

[boatbod]

Hi, sorry for replying to an old thread but I couldn't find a discord link or anything in the wiki about what I'm looking for.

I was just curious how this performs compared to Unitrunker. I'm currently using Unitrunker and am looking for newer, open-source alternatives. Also, the readme says SmartZone requires two dongles, but can I use more than two? The list of talk groups I scan is quite large so I currently have 5 SDRs to avoid missing anything.

Thanks in advance! You can certainly use more dongles, but the minimum is 2 (one for control channel, one or more for voice). Basically define however many voice channels you want separate streams and give each it's own set of whitelisted tgids.

I'll leave it to you to compare with Unitrunker as it's been years since I experimented with that. It does support both analog and digital talkgroups.

[Carter121]

Thank you! You guys are amazing! @boatbod @ilyacodes

[romanremus]

Hello Ilyacodes, Boatbod, and everyone, unfortunately in my country, especially in my area I can't enjoy too many DMR signals, but there are a lot of TETRA networks, what do you think about implementing TETRA in this OP25 code, I saw a code at Midnightblue tetra, Osmo-tetra, they even have TEA 1, 2, 3 encryptions, but there are no updates in these codes, if TETRA decoding can also be implemented in OP25, I will definitely forget about Windows and switch only to Linux - OP25. Thank you very much

[boatbod]

I'm open to accepting code contributions that add functionality (e.g. the recent DES-OFB support), but I don't presently have time to work on it myself.  More importantly, I'm not aware of any local tetra systems, which would make debugging very challenging.

Graham

On 11/29/24 09:25, romanremus wrote:

Hello Ilyacodes, Boatbod, and everyone, unfortunately in my country, especially in my area I can't enjoy too many DMR signals, but there are a lot of TETRA networks, what do you think about implementing TETRA in this OP25 code, I saw a code at Midnightblue tetra, Osmo-tetra, they even have TEA 1, 2, 3 encryptions, but there are no updates in these codes, if TETRA decoding can also be implemented in OP25, I will definitely forget about Windows and switch only to Linux

• OP25. Thank you very much

— Reply to this email directly, view it on GitHub https://github.com/boatbod/op25/issues/207#issuecomment-2507921239, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHS3SNV33BUNOVI5WF6XZCL2DB2ODAVCNFSM6AAAAABFNLL23SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBXHEZDCMRTHE. You are receiving this because you were mentioned.Message ID: @.***>

[romanremus]

OK, thank you very much, whenever you want I can provide RAW files, or BASEBAND recordings with tetra, unfortunately many tetra channels are also encrypted, when you have time please look into these codes https://github.com/osmocom/osmo-tetra, https://github.com/MidnightBlueLabs/TETRA_crypto

[romanremus]

DES-OFB is RC4 or AES256 ?

[boatbod]

OP25 currently understand RC4 and DES-OFB (with known keys), but not yet AES256G.On Nov 30, 2024 1:24 AM, romanremus @.***> wrote: DES-OFB is RC4 or AES256 ?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>