OneOfEleven / uv-k5-firmware-custom

A customized version of https://github.com/DualTachyon/uv-k5-firmware
Apache License 2.0
158 stars 72 forks source link

Request: Attempt real MDC-1200 #173

Closed Mason10198 closed 10 months ago

Mason10198 commented 10 months ago

The stock QS firmware contains the wonderful "fake" MDC-1200 EOT PTT ID. It does not sound like an actual 1200/1800 MDC Unit ID, and does not contain any actual information. Examples:

https://github.com/OneOfEleven/uv-k5-firmware-custom/assets/31994327/57138379-377a-4208-9a31-2ec81ffcf8c0

https://github.com/OneOfEleven/uv-k5-firmware-custom/assets/31994327/7f4317df-2c5c-4bb7-83c0-c5b58145548b

Encoding a proper MDC-1200 Unit ID should be doable, if someone with more skills than myself were to take this on. It should also be easy to add an option for this to be at BOT (beginning of TX) instead of EOT. Here is my shotty attempt at re-writing the current function:

// Define MDC-1200 Unit ID and function bits
#define MDC1200_UNIT_ID   0x0423    // Example Unit ID
#define MDC1200_FUNCTION  0x8A      // Example function

void BK4819_PlayRogerMDC(void)
{
    unsigned int i;

    BK4819_SetAF(BK4819_AF_MUTE);

    // Setting up the modulation and other settings remains the same
    BK4819_WriteRegister(BK4819_REG_58, 0x37C3);
    BK4819_WriteRegister(BK4819_REG_72, 0x3065);
    BK4819_WriteRegister(BK4819_REG_70, 0x00E0);
    BK4819_WriteRegister(BK4819_REG_5D, 0x0D00);
    BK4819_WriteRegister(BK4819_REG_59, 0x8068);
    BK4819_WriteRegister(BK4819_REG_59, 0x0068);
    BK4819_WriteRegister(BK4819_REG_5A, 0x5555);
    BK4819_WriteRegister(BK4819_REG_5B, 0x55AA);
    BK4819_WriteRegister(BK4819_REG_5C, 0xAA30);

    // Format MDC-1200 data
    uint16_t mdcData[4];
    mdcData[0] = 0x5555;  // Preamble (assuming 0x5555 for now)
    mdcData[1] = MDC1200_UNIT_ID;  // 16-bit Unit ID
    mdcData[2] = (MDC1200_UNIT_ID >> 8) | (MDC1200_FUNCTION << 8);  // Split Unit ID and function
    mdcData[3] = MDC1200_FUNCTION;  // 8-bit function code

    // Send MDC-1200 data
    for (i = 0; i < 4; i++)
        BK4819_WriteRegister(BK4819_REG_5F, mdcData[i]);

    SYSTEM_DelayMs(20);

    // 4 sync bytes, 6 byte preamble, Enable FSK TX
    BK4819_WriteRegister(BK4819_REG_59, 0x0868);

    SYSTEM_DelayMs(180);

    // Stop FSK TX, reset Tone-2, disable FSK
    BK4819_WriteRegister(BK4819_REG_59, 0x0068);
    BK4819_WriteRegister(BK4819_REG_70, 0x0000);
    BK4819_WriteRegister(BK4819_REG_58, 0x0000);
}

With my code, it sounds ever so slightly different to the ear, but it's still not correct. The pitch is too low, so maybe I'm missing something, or that's just the limitations of the baseband chip.

I also found this reference to a true MDC function in the BK4819 baseband chip, but I could not find any more information on implementing it. The reference document is full of usage data for FSK, but the MDC documentation is lacking.

Here's as much info as I could find on the actual MDC protocol. I own several MDC capable Motorola radios, so I can help test this if someone decides to pursue it. Wikipedia: MDC-1200 BATLABS: MDC-1200 Signalling SIGID: MDC-1200 DBPedia: MDC-1200 RadioReference: MDC-1200

OneOfEleven commented 10 months ago

Nothing like that is used here in the UK, at least not anymore, all comms are now digital, either DMR, TETRA or mobile phone network.

Apart from the AM aircraft bands, it's rare to now find analogue comms. You will find some, but not many these days.

Mason10198 commented 10 months ago

I agree, however amateur/private uses for analog ID methods (DTMF, ANI, MDC, Fleetsync, etc) do still exist, and since this seems to be halfway implemented already I thought it might be worth a shot if someone found the time to dig into it. Great work on the project BTW, and please don't take this "issue" as another "hey can you cater your project to me", because you have been getting plenty of those already. I just wanted to share what I was able to find on the topic in case it helps down the road.

OneOfEleven commented 10 months ago

It's a lot more than just adding a couple of ID values, it has to be encoded into a packet with CRC etc. I've got the code to do it. I've run out flash space for that amount of extra code, I need to optimize the firmware code to try and recover some space.

Lar-Sen commented 10 months ago

@Mason10198 Looks like your MOTO_REAL_MDC sample comes from MDC 2400. I explored a bit how its done at Quansheng's. What they call 'FFSK' in reference document (which should be MSK, all right?) is in fact AFSK with 1200/1800 mark/space tones. Which matches MDC protocol.

They did a stupid thing activating Tone2 via REG 70. This should make sense in the case of FSK to select baud rate, but here it just adds a stupid 1200Hz tone to the mix. Flipping a bit (put 0x60 to REG 70) resolves the problem. It sounds closer to MDC1200 now.

Anyway, the payload is confusing. In my opinion it translates to nothing. Maybe it's me, as I didn't have any sucess to find a snippet for de-interleaving the 'mdcData'. In my mods, I updated mdcData to play a PTTID EOT fixed to unit ID 0001. But it is not what you are looking for.

Mason10198 commented 10 months ago

The sample I provided was from an XTS3000.

If the EOT ID actually works and is decodable, that's perfect. The unit ID could be hard coded in the firmware build, changed at compile. I doubt there is space for a customizable unit ID menu entry.

Lar-Sen commented 10 months ago

Your sample decodes as PTTID BOT, ID#740 using WinMDCD. The payload I use in my mod was "stolen" from a great topic on Batlabs, and I have to admit I failed to forge a good packet myself :) I only adjusted preamble and sync bytes to better conform to the spec. Good thing to know is there is indeed 5 sync bytes available, whereas reference document from Beken only lists 4.

I don't know if it decodes well. I use WinMDCD locally listening to global downmix on my laptop. I miss a way to hook HT's speaker output to my sound card.

Mason10198 commented 10 months ago

I can test the decoding on my XTS and XPR radios, I also have the discriminator of an XPR wired directly into my sound card

Lar-Sen commented 10 months ago

As this is AFSK I don't see why a discriminator output would be needed; anyway I'm not an expert here :)

You could try my latest ROMpatched firmware, if you want to : https://github.com/Lar-Sen/Quansheng_UV-K5_Kitchen/blob/main/v2.01.31_mods/k5_v2.01.31_MODDED.bin

It has "my" MDC payload.

OneOfEleven commented 10 months ago

They did a stupid thing activating Tone2 via REG 70. This should make sense in the case of FSK to select baud rate, but here it just adds a stupid 1200Hz tone to the mix. Flipping a bit (put 0x60 to REG 70) resolves the problem. It sounds closer to MDC1200 now.

oh right !

I'll be back to ask you about the fix you've done, I'll add it in

Mason10198 commented 10 months ago

Discriminator isn't needed, it's just what I happened to have hooked up, it was an unnecessary specification in my comment on my part, lol.

I will test your payload this evening or sometime soon and report back.

OneOfEleven commented 10 months ago

This is currently how the K5's MDC1200 sounds, this is a direct recording in the SDR's RX ..

mdc1200_k5.zip

OneOfEleven commented 10 months ago

I did modify it a few days ago to include the initial 27ms of each of the FSK tones.

OneOfEleven commented 10 months ago

The c code to create a custom packet with your chosen ID I do have (and the decode c code), I'll add it as a compile option - assuming it's correct.

Lar-Sen commented 10 months ago

Interesting to ear pure signal of it, not filtered. It appears clearer than listening to it with another handheld. We can clearly notice a short burst of that continuous 1200Hz at the beginning. Anyway WinMDCD still says wtf :)

Trying to understand bit interleaving which Motorola makes use of, I wrote this decoding algorithm in a basic-like form (pls don't laugh xD ) :

pos=1
for i=1 to 16
{
    for j=0 to 6
    {
        k = (j*16) + i
        bit_dest(pos) = bit_source(k)
        pos=pos+1
    }
}

This is the only thing I found picky. Rest of the frame structure is well documented. Can't wait to study your implementation :)

OneOfEleven commented 10 months ago

If you like playing radio you should get yourself a cheap SDR RX only dongle such as one of the nooelecs. heres one ..

https://www.amazon.co.uk/Nooelec-NESDR-SMArt-SDR-R820T2-Based/dp/B01HA642SW/ref=sr_1_3

I use the blue mini 2 one for recording audio. Use the free SDRSharp (SDR#) with them.

Lar-Sen commented 10 months ago

I got one, it's just because I'm lazy :) A good old RTL-SDR which I use with HDSDR software. I prefer HDSDR to SDR# because it's lighter, as I run it on another (old) laptop. The NooElec looks it a an upgraded tuner over RTL-SDR v3. Good to now at that price.

BTW I was examining the updates you made to the PlayRogerMDC() function. Register init looks clearer and I noticed you disabled the undocumented 0b101 which was set to FSK RX mode. Dunno if this were one more quirk from QS or really useful; I did the same anyway. Do you think packet could take longer to xmit, as you watch interrupts for TX end in a limited loop? Or is this preliminary work for something better there?

Edit: Oh ok. You add 00000000FFFFFFFF to the MDC payload, making it longer. Scratching my head now :)

OneOfEleven commented 10 months ago

The loop that monitors the packet transmitting just adds a few more ms to the time limit so as not to interferer with it. The loop is to ensure the TX doesn't get stuck on for any reason (corrupt BK register while TX'ing or such like).

A stuck TX will upset a LOT of people, that's why I removed the ability to press the PTT permanently by always having a TX timeout setting. It happened a few days ago locally when someone accidently caused their transceiver to stay TX'ing (they had disabled the TX timeout) for an hour or more.

OneOfEleven commented 10 months ago

Edit: Oh ok. You add 00000000FFFFFFFF to the MDC payload, making it longer. Scratching my head now :)

That was how I added the 27ms low tone then 27ms high tone that I see on some MDC1200 packets. I don't have the MDC1200 details to look at, but I do have C code to encode and decode the data (not mine, so hoping it's error free).

Lar-Sen commented 10 months ago

Oh ok :) Thank you for the explanation. I have to listen more, but this traffic type is not so common here. These tones could be related to preamble and synchronization bytes. MDC sync is told to be 07092A446F, which cannot produce uniform tone. Preamble bytes can. Anyway. I took a look at the reverse engineering work by DualTachyon for the Radtel RT-890, especially the PTTID part. This unit share the same BK4819 core, they implemented a "send my device name" using same ASFK method, more or less. This is not intended to be interoperable with MDC1200, but I'm sure you can get inspiration to implement digital PTTID with a custom device name. Interestingly, register init is a bit different. They make use of said "FSK Scrambler" for it, same Tone2 "issue", and mysterious register 5C gets 0x5665 (CRC on, what else?). All that in a different sequence. Maybe we can get clues there. At least I've learned we have to fill FIFO MSB first :) Many of raw MDC frames found on the internets show as little-endian sequences of dwords.

OneOfEleven commented 10 months ago

I have no idea at all about what's going on with the unknown registers, it's all guess work without a datasheet.

Anyway, I've added an mdc1200 packet encoder, I have no idea if it works as I don't have an mdc1200 decoder of any kind. It's going to be in the hands of those that do now.

Lar-Sen commented 10 months ago

Converning the loop/interrupt method, Radtel apparently choosed the same way, only 1s delay and monitored interrupt is different (SQ detection).

You are lightning fast! :) You can easily decode MDC packets using a little windows app, WinMDCD. It's old but efficient.

https://www.dropbox.com/s/w332mfh4dw309xv/WinMDCD10.zip?dl=0

Lar-Sen commented 10 months ago

MDC codec looks promising, I'll take time to study it. BTW may I suggest you use real opcode and argument to test it? For example, PTT ID opcode is 01, its argument is 80. A more complete list is:

01 (80) is PTT ID 01 (00) is POST ID 11 (8A) is REMOTE MONITOR 22 (06) is STATUS REQ 2B (0C) is RADIO ENABLE 2B (00) is RADIO DISABLE 35 (89) is CALL ALERT 46 (XX) is STS XX 47 (XX) is MSG XX 63 (85) is RADIO CHECK

..found at https://batboard.batlabs.com/viewtopic.php?t=36459

OneOfEleven commented 10 months ago

That winmdcd program just sits there doing nothing on my PC. I'm guessing it's suppose to see the laptops audio interface, but it doesn't look like it is.

Image1

Lar-Sen commented 10 months ago

WinMDCD is listening to your default audio capture device. It has no builtin option to select a recording device. You may have to set it to "Stereo Mix" or "Local loopback" in windows audio control panel.

OneOfEleven commented 10 months ago

I forgot to say that aircopy uses direct FM mode rather than toned FSK, I forgot about that when you were talking about pocsag.

Anyway, I'm really close to getting MDC working, though I need an FSK demodulator to let me move forward. I've done a software 1200bd one somewhere in my other projects, I'll search it out as I need to see exactly whats being sent from the K5 as compared to the 2 MDC audio samples I've managed to find on the net.

wrcrooks commented 10 months ago

@OneOfEleven Do you happen to have a Raspberry Pi or a Linux box laying around? I just got decoding MDC1200 signals working with fsync-mdc1200-decode (Project link: https://github.com/russinnes/fsync-mdc1200-decode)

It can take microphone input, similar to WinMDCD. Just a head's up, I forgot to unmute my microphone on my laptop when running it so it had the right audio input but wasn't hearing anything.

OneOfEleven commented 10 months ago

I haven't no, don't have any nix, just windoze.

I'm just writing a test program (for windows) to decode and show an MDC1200 signal in all it's detail ..

Image1

wrcrooks commented 10 months ago

Wow, you have a lot of cool projects up in the air! I'll offer up my help to decode any MDC1200 WAV files in the meantime, if that helps with your testing at all.

OneOfEleven commented 10 months ago

ok, heres a WAV file I've done of the K5 MDC1200 as it currently is, I recorded it on the SDR RX so it's nice and clean and unfiltered. Let me know what it makes of it maybe ? ..

MDC1200_K5.zip

I have something still slightly amiss with the MDC1200 code in the K5, WinMDCD won't decode it at all, it gives no details about why (very unhelpful)

wrcrooks commented 10 months ago

Well it seems like I may have spoken out of turn. The fsync-mdc1200-decode utility doesn't offer any analysis of the MDC message, it only provides output if the message is properly decoded.

Doing some Google searching, there really aren't any tools that I can find out there that help to troubleshoot MDC if it's out of spec.

That got me wondering if there's something wrong with the 1200bps rate that the Quanshengs are producing because I can't decode anything FSK1200 at all. Lends to the theory that the timing is slightly off.

OneOfEleven commented 10 months ago

The MDC1200 TX is no longer fake. I've yet to add a menu option to set the radio ID but apart from that the MDC1200 roger should be good now.

If all is well I'll move the MDC1200 to the PTT-ID menu so that it can be set to BOT and/or EOT.

OneOfEleven commented 10 months ago

@Mason10198, please report what you now find, it's been a LOT of work.

Mason10198 commented 10 months ago

You all are legends. I need to play with it more, but at quick glance, the EOT is properly decoded by my XTS3000.

My newer MOTOTRBO radios however (2x XPR4550s and 1x SL300), are muting the MDC on RX for some reason. When done on purpose, this is called DOS (Data Operated Squelch), but I have DOS completely disabled on these radios so this shouldn't be happening. The decoded ID is not shown at all on the display either. Maybe the newer TRBO radios dislike something about this packet?

All of my radios are configured with DOS disabled and MDC ID at BOT. All Moto radios decode, audibly present, and display each others ID just fine, but only the older XTS will properly decode and display the ID from the K5.

Surely would not make a difference, but maybe try it at BOT instead?

I saw some RX decode stuff in your commits, I don't think you implemented it yet, but in case you did, I do not see any decoded IDs on the QS screen.

Your work is astounding. Let me know if I can do anything to assist!

wrcrooks commented 10 months ago

Decoded message via fsync-mdc1200-decode: {"type":"MDC1200","timestamp":"1698191676","op":"01","arg":"80","unitID":"b183","ex0":"00","ex1":"00","ex2":"00","ex3":"00"}

Recognized it like a charm!

mjwgeek commented 10 months ago

@Mason10198 did you see if Asterisk decoded it?

I've been following this and this is super exciting work!

Mason10198 commented 10 months ago

@Mason10198 did you see if Asterisk decoded it?

I just went to try it on my repeater, and noticed that the new MDC does not work with CTCSS TX enabled - it comes out garbled. TXes fine with CTCSS turned off.

OneOfEleven commented 10 months ago

I'll look at the CTCSS + MDC1200 problem.

Am hoping the BK radio chip can send the MDC while it's still sending the CTCSS tone, if not then I'll have to disable the CTCSS while it sends the burst of MDC, will see.

I'll do it later this morning, tiz 6am here, just having a quick nosey on the laptop, and still sleepy.

OneOfEleven commented 10 months ago

Interesting to ear pure signal of it, not filtered. It appears clearer than listening to it with another handheld. We can clearly notice a short burst of that continuous 1200Hz at the beginning. Anyway WinMDCD still says wtf :)

I've found that WinMDCD is very sensitive to the audio it hears in that it won't decode at all if it's not at the expected level and/or is somehow filtered, it's extremely fussy :( .. exactly not what you want from a decoder.

OneOfEleven commented 10 months ago

@Mason10198 did you see if Asterisk decoded it?

I've been following this and this is super exciting work!

Wonder what Asterisk is ?

I've had to create a new decoder to get MDC1200 working so that I could see every demodulated bit etc, this is Masons original MOTO_REAL_MDC.mp4 being fed into it ..

Image1

It tells me the packet is op 01 arg 80 ID 0740

OneOfEleven commented 10 months ago

can't have both, tones of one would corrupt the other

Mason10198 commented 10 months ago

Wonder what Asterisk is ?

Asterisk is an open source PBX software. Amateur radio operators use the term "Allstar" to refer to a modified version of Asterisk that is intended to interconnect radios over VOIP.

app_rpt is a module included in this modified Asterisk, that provides Asterisk with the digital functions for interfacing with a physical radio.

app_rpt has a built in MDC-1200 decoder, so that's what me and @mjwgeek are talking about here - we both have repeaters that run Asterisk/app_rpt controllers.

OneOfEleven commented 10 months ago

oh I see, very nice !

There is a fusion node about 500 yards from me that I sometimes listen too. Not heard of asrerisk, though allstar yes.

Try the new upload I've done, I've added a menu item. I've yet to add BOT & BOT just for MDC1200 though, but nearly there.

Mason10198 commented 10 months ago

You added a new menu option? I'm not seeing it on the latest commit.

I fixed the MOTOTRBO radios not decoding the ID. They are still muting it with DOS for some reason (maybe they always do that for an EOT ID regardless of DOS setting and I never noticed it since I always use BOT IDs) but at least now they decode and display properly. You are using 0x80 arg which is supposed to be for a BOT (pre) ID. Change this to a 0x00 for an EOT (post) ID.

Pulled this from old Batlabs thread:

OP (ARG)
01 (80) - PTT ID
01 (00) - POST ID
11 (8A) - REMOTE MONITOR
22 (06) - STATUS REQ
2B (0C) - RADIO ENABLE
2B (00) - RADIO DISABLE
35 (89) - CALL ALERT
46 (XX) - STS XX
47 (XX) - MSG XX
63 (85) - RADIO CHECK
    const uint8_t  op  = 0x01;
    const uint8_t  arg = 0x00;
    const uint16_t id  = 0x0740;
OneOfEleven commented 10 months ago

oops, got my bots n eots in a twiz.

Hopefully fixed now - let me know.

Have a look around menu number 47 (exact number varies depending on what you enable) .. MDC ID

Mason10198 commented 10 months ago

Interesting, I compiled the firmware about an hour ago and I don't have that menu option. I probably missed something. I'm away from home now so I'll have to wait until this evening to compile it again.

More observations: with CTCSS and DCS disabled, the TRBO radios will decode it but not allow it to come through the speaker. With CTCSS enabled, it's too garbled to be decoded and as such does come through the speaker. With DCS enabled, it comes through the speaker garbled but IS also properly decoded and displayed.

OneOfEleven commented 10 months ago

So what do you see on your moto radios when they hear an MDC burst of data I wonder ?

Show me a piccy if you can.

OneOfEleven commented 10 months ago

More observations: with CTCSS and DCS disabled, the TRBO radios will decode it but not allow it to come through the speaker. With CTCSS enabled, it's too garbled to be decoded and as such does come through the speaker. With DCS enabled, it comes through the speaker garbled but IS also properly decoded and displayed.

That shouldn't still be happening, I fixed the CTCSS+MDC problem ??

Mason10198 commented 10 months ago

I bet something happened with my git pull, I was in a rush out the door trying to load the latest.

This is what the display looks like on an XPR4550. It's decoding correctly, I just need to figure out why it's being DOS'ed. Need to set another radio to EOT and see if it does the same thing, or if it's something to do with our packet.

20231025_083240.jpg

OneOfEleven commented 10 months ago

lol, I see moto aren't big on showing useful info on there screens. Least they could do is say BOT 0740 or EOT 0740 ;)

wrcrooks commented 10 months ago

@OneOfEleven I like the new menu option a lot! And I hate to be 'that guy' but after spending a few minutes holding down the arrow key to get to the RADIO ID that I wanted, I got to thinking: "Could we put a Makefile parameter that sets the MDC1200 Radio ID's default value at compile time?"

What do you think?