lwvmobile / dsd-fme

Digital Speech Decoder - Florida Man Edition
Other
159 stars 30 forks source link

NXDN96 ISSUE #70

Closed iz4tow closed 2 years ago

iz4tow commented 2 years ago

Hi I think there is an issue with NXDN96. I tried to decode many tracks without success, the output is confused. I don't think the tracks are encrypted because I can listen voices, but they are distorted (too fast maybe?)

lwvmobile commented 2 years ago

Hi,

Which Github build version are you using? Which branch? Also, which OS/Distro are you running? Can you tell me which command you are using to start up with?

iz4tow commented 2 years ago

Hi, I'm using your last commit with hash 171c71ddcbb085d1d2f2e659cf476e3139c8f6db on Ubuntu 20.04 Desktop. Here the command from wav file: cat encodedNXDN.wav | dsd -w decoded.wav -n -i -

Here the command from live input (like GQRX): padsp -- dsd -i /dev/dsp -n -w test.wav

lwvmobile commented 2 years ago

I suspected that might be the case. The 'main' branch is woefully behind, it is only there at this point to redirect users to the Pulse Audio branch.

https://github.com/lwvmobile/dsd-fme/tree/pulseaudio

With the pulse audio branch, you can do a lot more with the audio, including just play wv files in a media player and listen to them back. I suggest you take a look over the 'pulseaudio' branch readme file or take a look at this video on an update I'll be publishing tomorrow.

https://www.youtube.com/watch?v=TqTfAfaPJ4s

iz4tow commented 2 years ago

I'm trying with 10ed2ea but same results.

iz4tow commented 2 years ago

I tried also ncursesmenu branch. Same results

lwvmobile commented 2 years ago

Are you still trying to use the cat command method to push the wav file into the input? I usually don't recommend using that method, because it exhibits strange behavior. I would suggest setting up a virtual sink - see the script for that - and playing the wav file from a media playing (of the GQRX signal directly) into a virtual sink, and set dsd-fme up to listen to that virtual sink, and try using dsd-fme with the command.

dsd-fme -fn

Also, if you could, can you provide that wav sample to me for testing? I just want to rule out any issues with the quality of the signal. You can upload it to https://ufile.io/ or any hosting service of your choosing, if you could please.

iz4tow commented 2 years ago

Yes I'm using cat command to script an entire input folder. Tomorrow I'll try and I'll send you some samples.

lwvmobile commented 2 years ago

Again, using cat to push wav files usually has some bad results from my own testing. Sometimes it works, sometimes it doesn't, just depends on the system type and the sampling rate of the wav file.

In the meantime, this is how I usually playback wav files for DSD-FME to decode.

https://www.youtube.com/watch?v=Jm2emJkONXw

This is a demonstration using NXDN96 samples that I have.

Again, I highly advise reading over the entire README file on the pulse audio branch for information on the ways that DSD-FME can be used. I cannot and do not officially support using the cat wavfile | dsd-fme -i - -fi input method.

Listening from GQRX can be done with either piping the audio out into a virtual sink in the same manner as playing back wav in a media player for DSD-FME to listen to, or by using the UDP output from GQRX into a pipe into DSD-FME.

Be sure to first start UDP output sink in GQRX or SDR++ and set VFO appropriately, including squelch for NXDN

socat stdio udp-listen:7355 | dsd-fme -i - -fn

iz4tow commented 2 years ago

Hi, I tried with your instruction using virtualsink on pulseaudio and the result remain the same, but the software shows that maybe the conversation could be encrypted.

04:44:20 Sync: +DMR  [slot1]  slot2  | Color Code=01 | (CRC OK ) | VOICE LC Header  
 Slot 1   TGT=2  SRC=4801  Encrypted Call  
04:44:20 Sync: +DMR   slot1  [slot2] | Color Code=01 | (CRC OK ) | Slot idle    
04:44:20 Sync: +DMR  [slot1]  slot2  | Color Code=01 | (CRC OK ) | VOICE LC Header  
 Slot 1   TGT=2  SRC=4801  Encrypted Call  
04:44:20 Sync: +DMR   slot1  [slot2] | Color Code=01 | (CRC OK ) | Slot idle    
04:44:20 Sync: +DMR  [slot1]  slot2  | Color Code=01 | (CRC OK ) | VOICE LC Header  
 Slot 1   TGT=2  SRC=4801  Encrypted Call  
04:44:20 Sync: +DMR   slot1  [slot2] | Color Code=01 | (CRC OK ) | Slot idle    
04:44:20 Sync: +DMR  [slot1]  slot2  | Color Code=01 | (CRC OK ) | PI Header     
 Slot 1 DMR PI Header ALG ID: 0x02 KEY ID: 0x03 MI: 0x5956A57D 
04:44:21 Sync: +DMR   slot1  [slot2] | Color Code=01 | (CRC OK ) | Slot idle    
04:44:21 Sync: +DMR  [SLOT1]  slot2  | Color Code=01 | (CRC OK ) | VOICE 
04:44:21 Sync: +DMR  [SLOT1]  slot2  | Color Code=01 | (CRC OK ) | VOICE 
iz4tow commented 2 years ago

https://drive.google.com/drive/folders/16tOk_6slsBS0Gj2TqV7bhjN5dHJjPLwW?usp=sharing

lwvmobile commented 2 years ago

Hi, I tried with your instruction using virtualsink on pulseaudio and the result remain the same, but the software shows that maybe the conversation could be encrypted.

That particular sample would indeed appear to be encrypted, but seems to be a DMR system and not a NXDN96 system. I have a suspicion that you may have believed that system to be NXDN96 because older versions of DSD attempted to decode NXDN96 automatically and that leads to a lot of false positives.

iz4tow commented 2 years ago

Yes true. Looking on Internet I found that some users, like Luis Herve, claims that is possible to decrypt DMR tracks, especially if they are encrypted with "basic privacy". Are you planning to add decryption features?

LouisErigHerve commented 2 years ago

The track your are looking to decode is effectively encrypted DMR, and NOT NXDN.

The PI Header contains some interesting data : ALG ID = 0x02 = Hytera Full Encrypt (Hytera proprietary algorithm)

Another parameter to check is the FID (Feature Set ID), it is not displayed here but for Hytera its value should be equal to 0x68

For Motorola MotoTRBO systems, FID should be equal to 0x10 (in case of Basic Privacy and Enhanced Privacy RC4/AES use)

iz4tow commented 2 years ago

Are you planning to add decryption features? Can you please provide in the project any protocol (frames/packets) description available? Maybe a table with packet header useful information like what a TLC/slot idle/Voice/... packet is.

And most important can you write down a table with FID -> Brand/models and ALG ID -> algorithm name? I think these could be very important information to anyone.

Thanks

lwvmobile commented 2 years ago

Are you planning to add decryption features?

The short answer for the foreseeable future is no, I do not plan to add any sort of decryption features.

Can you please provide in the project any protocol (frames/packets) description available? Maybe a table with packet header useful information like what a TLC/slot idle/Voice/... packet is.

If I did that, it would quickly become a never ending list. I can't even say that I have all of them myself, just some here and there I've picked up. The best source for a lot of this type of information is the ETSI manuals, just try searching for ETSI DMR manuals and you will find them. And that is just for DMR, and not the other decoder types.

The short answer for those specifically mentioned.

Voice LC Header - Voice Transmission to Follow, here are the tgt and src info PI Header, Privacy Indicator, tells radios which alg and key to use and contain MI Voice: The duration of the voice call, has the voice frames and some embedded signalling TLC: Terminator, end of call. Slot Idle: Pseudorandom data to fill out the signal, just indicates system is idle.

And most important can you write down a table with FID -> Brand/models and ALG ID -> algorithm name?

Again, these can also be found with a web search, but just a couple off of the top of my head.

DMR ALG 0x21 RC4 0x22 DES-OFB

LouisErigHerve commented 2 years ago

If that can help a list of Manufacturer ID (FID) can be found here : http://www.etsi.org/images/files/DMRcodes/dmrs-mfid.xls

iz4tow commented 2 years ago

Thank you! Ill'try to read ETSI documentations and I'll provide you a table if you wish. I can also write a simple guide to attach to DSD project for newbie.

iz4tow commented 2 years ago

And I'm trying to port your project on Windows using Ubuntu WSL. I'm not a Windows fan but I think most of O.M. are more confident with Windows than Linux

lwvmobile commented 2 years ago

I'm trying to port your project on Windows using Ubuntu WSL

That will be interesting to hear how that turns out. I don't use Windows myself, but I attempted to do this once, to only realize you can't run WSL in Windows 11, when Windows 11 is running inside of a Virtual Machine. At least, I couldn't. I think there was some boxes I could have checked, but when I tried to run it, it made my entire system hang up and it messed up my Windows VM. Will be interesting to hear the results on real hardware.

You may have to skip the support for the rtl input stuff, don't know if that will build properly or be usable in WSL or not, but using it is purely optional for building.

For what its worth, I have had success with getting it to work inside of Cygwin, as have a few others, but building it with RTL support is a severe hassle, and there is some performance issues and also the inability as far as I know to route stereo slots out of the left and right channel since you can't open pulsemixer or pavucontrol in cygwin since its not available.

iz4tow commented 2 years ago

Hi. I don't have a RTL dongle to test with me, I'm using an HackRF as receiver, but I think with WSL could be possible to use RTL devices so there is no need to skip librtl. Also because in WSL it's very easy to install. I'll try in the future to use RTL in WSL, but I need to use a real hardware PC with WIN 10 to avoid issues related to VM hardware connection.

I'm trying in a VirtualBox WIN 10 machine with good results, the only problem it's that I can't run WSL 2 but only WSL 1, I think the issue are Hyper-V API - Virtualbox compatibility. I think that is your problem, maybe on WIN 11 you can only use WSL 2, you may try using VMWARE VMPlayer instead of qemu or VirtualBox.

But however...it's working quite well with wav and WSL 1

Next step I'll try to expose a udp port from WSL.

If you want I can provide my batch and bash script.

lwvmobile commented 2 years ago

1, I think the issue are Hyper-V API - Virtualbox compatibility.

I believe you are correct. When I enabled those features in Virtuabox and went into UEFI on my host computer and enabled those, when I rebooted the VM my entire host computer became unstable, and I ended up having to delete the VM and make a new one. Can't remember what all I did, but I gave up on it quickly. I was going to test it on real hardware, I just never got back around to doing so.

If you want I can provide my batch and bash script.

If you'd like to provide any finding you have, it may be better to make a new issue for those that is properly named for others who come along who want to contribute or see the information will be able to easily find it. People might not be able to find that information if its posted here since the title is NXDN96, so starting a new 'issue' might be best, in order to keep things semi organized.

lwvmobile commented 2 years ago

I'm going to go ahead and close this issue as well, since we already got the wiki up now and everything cleared up on this one as well I believe.