search

df8oe / UHSDR

SDR firmware and bootloader with configuration files for use with Eclipse, EmBitz and Makefile
Other
360 stars 189 forks source link

RTTY character encoding / decoding #1858

Closed SP5QAD closed 4 years ago

SP5QAD commented 4 years ago

UHSDR Firmware Defect Issue (Bug) Template 1.1

Issue Data goes here (please remove text above):

Your firmware version: 2.11.84 Your bootloader version: 5.0.4 (Please note, we will generally not accept issue reports if your bootloader is not at least 5.0.1)

Hardware

Describe the issue:

RTTY transmission test: Text entered from a wireless USB keyboard: logitech k400 (works as expected) connected directly to mcHF, or played back with macro.

Transmissions decoded in the program "MULTIPSK v.4.43

noticed errors: instead of "0" -> "Ř" appears (in the "MULTIPSK" program it looks like 'R' with a bird at the top).

Characters he didn't give: '!', '@', '#', '$', '%', '^', '&', '*', '_' '[', ']', '{', '}' ';', '"', '', '|' '<', '>'

In my opinion, the signs: '!' and '@' could be broadcast.

Broadcasting from the MULTIPSK program, decoding to mcHF: characters incorrectly decoded: sent -> decoded '!', '@', '#' -> '.' ''(multiplication sign) -> '#' '' '(single quotation mark, apostrophe) ->', '(comma) '?' -> '7' signs: '%', '^', '&', '', '_', '[', ']', '{', '}', ';', '"', '', '| ' '<', '>' they were also not broadcast from the MULTIPSK program.

Greetings, Marcin SP5QAD.

db4ple commented 4 years ago

Have a look here:

https://github.com/df8oe/UHSDR/blob/6b97d917f471f438783831a69706fb050e88bb56/mchf-eclipse/drivers/audio/rtty.c#L26 This lists the ASCII to RTTY Code map. If there is 0 in a position it means this character cannot be translated. So anything we have 0 in a row, we simply don't transmit. This matches with your observations. The only strange thing is '0' which obviously should translate to '0' And in the other direction, MULTIPSK does translate characters before sending them out.

SP5QAD commented 4 years ago

There are still distortions in decoding by mcHF when receiving broadcasts, e.g. from MULTIPSK: characters: "!, @, #" have been decoded as: "." (dot), the multiplication sign (*) has been decoded as: "#" (hash) single quotation mark was decoded as: "," (comma) and what surprised me a lot: the question mark ("?") was decoded as: "7" (seven) Hi ...

Marcin

SP5QAD commented 4 years ago

I haven't checked what it looks like on BPSK broadcasts, but I'll catch up soon. ;-) However, I suppose the same procedures are used ...

db4ple commented 4 years ago

Hi Marcin, you are right, the back translation (Baudot to Ascii) has some interesting errors.

db4ple commented 4 years ago

BTW, it seems that MULTIPSK and FLDIGI do not entirely agree on the code table but since MULTIPSK is closed source and flgigi is open source it was much easier to use the fldigi table. What not has been checked is how our ASCII to Baudot differs from our Baudot to ASCII.

SP5QAD commented 4 years ago

Software test v.2.11.86:

  1. Broadcasting with mnHF

decoding program: MULTIPSK

RTTY mode BD = 45, SFT = 170/200:

Pressed / given: ! @ # $% ^ & * () _ + decoded: ! @ # $ () +

Pressed / given: 1234567890- = decoded: 123456789Ř-

instead of 0 it is still decoded: Ř equality sign (=) was not given

but the ';' sign was pressed, it was decoded as '=' - such a surprise ;-)

Letters from a to z - correctly.

Attention: 1.SFT = 425 is decoded in MULTIPSK as RTTY 50.

  1. Transmission with settings BD = 50, SFT = 170 I could not decode.

BPSK31 / 63/125 mode: (changing modes works, transmissions according to the selected mode, while the visual change in marker spacing is visible only after changing to a different mode and returning again)

Pressed / given: ! @ # $% ^ & () _ + decoded: ! @ # $% ^ & () _ +

Pressed / given: 1234567890- = decoded: 123456789Ř- =

instead of 0 the following is still decoded: Ř. '=' character set correctly (incorrectly in RTTY transmission)

Pressed / given: [] {}; '',. / <>? ~ decoded: [] {}; '',. / <>?~

Letters from a to z - correctly.

  1. Broadcasting from the MULTIPSK program decoding on mcHF

RTTY mode BD = 45, SFT = 170/200: given: 1234567890- = decoded: 1234567890-; (the '=' character is decoded as ';')

posted -> decoded:

! -> ! @ -> & "#" -> "#" $ -> $ % -> (nothing) ^ -> (nothing) & -> (nothing) "*" -> ' (single quotation mark) ( -> ( ) -> ) _ -> (nothing) "+" -> " (double quotation mark)

posted -> decoded: [] {} -> (nothing) ; -> (nothing) '-> probably a duplicate of single quotation marks, I'm not sure what it is. \ -> (nothing) : -> : "-> (nothing) | -> (nothing)

, ->, . ->. / -> /

"<" -> (nothing) ">" -> (nothing) "?" -> ?

  1. Broadcasting from the MULTIPSK program decoding on mcHF

BPSK31 / 63 mode: It's hard to tune ... something is decoding.

Marcin.

db4ple commented 4 years ago

Hi Marcin, thanks for testing. In order to not waste your time, please review the supported characters here: Even if you are not a programmer, you can see which characters we can transmit and receive. Any character not shown in these lines will neither be transmitted or received:

https://github.com/df8oe/UHSDR/blob/918d62160edca71d1d3bbc2064de0ea317b8916b/mchf-eclipse/drivers/audio/rtty.c#L27-L41

And please test also with fldigi. I used the code from fldigi and would like to know if you can decode RTTY correctly with fldigi. Especially the 0 is of interest.

K6ANI commented 4 years ago

Marcin,

In general, it appears that MULTIPSK is using ITA2 Baudot-Murray code for RTTY, and UHSDR (and FLDIGI by extension), is using US-TTY Baudot Murray code for RTTY.

As far as I can tell from researching the topic, both RTTY code tables are legal and common in Ham Radio worldwide, though I presume the ITA2, being a CCITT international standard, would be used more outside the U.S. than US-TTY.

You can see the differences in the two Code Tables at:

https://en.wikipedia.org/wiki/Baudot_code

There are several character differences and characters that are missing from one or the other of the tables. For instance, the ";" in the US-TTY table gets translated by MULTIPSK into an "=" in the ITA2 table, explaining some of these types of "errors" you were seeing.

Also in general, it appears that if a character is not in the US-TTY table and you type the character in UHSDR, it appears that UHSDR does not even process the character or send it out. So that would explain some of the missing characters in the "! @ # $% ^ & * () _ +" to "! @ # $ () +" sequence.

Jerry

K6ANI

K6ANI commented 4 years ago

DB4PLE,

It appears that MULTIPSK is using ITA2 Baudot-Murray code for RTTY, and UHSDR (and FLDIGI by extension), is using US-TTY Baudot Murray code for RTTY.

I compared the UHSDR code segment you referenced with the US-TTY Baudot-Murray code used by US hams. Both tables are identical, so any of the RTTY characters in the US-TTY Baudot-Murray code table should be sent out.

I looked specifically at the "0" and "&" characters in the ITA2 and US-TTY RTTY tables, and they both have the same code in both tables. So that appears not to be the problem.

I would assume that if there was a compatibility problem in either FLDIGI or MULTIPSK (ITA2 and US-TTY codes) with the "0" character, it would have been seen and fixed by now, as the "0" character would be used quite a bit by hams, and both MULTIPSK and FLDIGI are commonly used programs.

Jerry

K6ANI

On Thu, 21 May 2020 07:29:08 -0700, db4ple notifications@github.com wrote:

Hi Marcin, thanks for testing. In order to not waste your time, please review the supported characters here: Even if you are not a programmer, you can see which characters we can transmit and receive. Any character not shown in these lines will neither be transmitted or received:

https://github.com/df8oe/UHSDR/blob/918d62160edca71d1d3bbc2064de0ea317b8916b/mchf-eclipse/drivers/audio/rtty.c#L27-L41

And please test also with fldigi. I used the code from fldigi and would like to know if you can decode RTTY correctly with fldigi. Especially the 0 is of interest.

  • mcHF RTTY TX: Our RTTY code does not transmit any of these % ^ * =, they are not part of the RTTY baudot code, see above . However & should work in the latest release. So with the expection of & and 0 the results are as expected. See list above for reference.

  • mcHF RTTY RX: Also use the list above as reference. It is obvious, that for some special characters MultiPSK and fldigi disagree. We use the fldigi as reference, so if MultiPSK disagrees with this coding, so be it.

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/df8oe/UHSDR/issues/1858#issuecomment-632117708

SP5QAD commented 4 years ago

I checked the decoding in the fldigi program:

  1. all digits are decoded correctly (zero also),
  2. the following characters have been correctly decoded: ! # $ & () -:;,. /?
  3. when given '(single quotation mark) it has been decoded as the inscription '\<BEL>'
  4. "(double quotation mark) has not been assigned (further tests have shown that the added (+) character is decoded in fldiga just as double quotation marks (")).
SP5QAD commented 4 years ago

DB4PLE, It appears that MULTIPSK is using ITA2 Baudot-Murray code for RTTY, and UHSDR (and FLDIGI by extension), is using US-TTY Baudot Murray code for RTTY. I compared the UHSDR code segment you referenced with the US-TTY Baudot-Murray code used by US hams. Both tables are identical, so any of the RTTY characters in the US-TTY Baudot-Murray code table should be sent out. I looked specifically at the "0" and "&" characters in the ITA2 and US-TTY RTTY tables, and they both have the same code in both tables. So that appears not to be the problem. I would assume that if there was a compatibility problem in either FLDIGI or MULTIPSK (ITA2 and US-TTY codes) with the "0" character, it would have been seen and fixed by now, as the "0" character would be used quite a bit by hams, and both MULTIPSK and FLDIGI are commonly used programs. Jerry K6ANI … On Thu, 21 May 2020 07:29:08 -0700, db4ple @.**> wrote: Hi Marcin, thanks for testing. In order to not waste your time, please review the supported characters here: Even if you are not a programmer, you can see which characters we can transmit and receive. Any character not shown in these lines will neither be transmitted or received: https://github.com/df8oe/UHSDR/blob/918d62160edca71d1d3bbc2064de0ea317b8916b/mchf-eclipse/drivers/audio/rtty.c#L27-L41 And please test also with fldigi. I used the code from fldigi and would like to know if you can decode RTTY correctly with fldigi. Especially the 0 is of interest. - mcHF RTTY TX: Our RTTY code does not transmit any of these % ^ ``=, they are not part of the RTTY baudot code, see above . However & should work in the latest release. So with the expection of & and 0 the results are as expected. See list above for reference. - mcHF RTTY RX: Also use the list above as reference. It is obvious, that for some special characters MultiPSK and fldigi disagree. We use the fldigi as reference, so if MultiPSK disagrees with this coding, so be it. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #1858 (comment)

Yes, the only question is how many people broadcast directly from mcHF in RTTY mode ;-) I suspect that most people worked with the MultiPSK or fldigi or some other program, and mcHF was used only as a relay (broadcasting a ready RTTY signal from the program).

db4ple commented 4 years ago

@SP5QAD: Thanks for testing with fldigi. I fixed the " (now one has to press " , not +).

db4ple commented 4 years ago

@SP5QAD : I don't think not many use RTTTY TX straight from the keyboard. I agree, most will use fldigi or MultiPSK etc.

SP5QAD commented 4 years ago

My previous comment / question was directed to K6ANI. I have not yet figured out how to properly comment on specific entries ... provided that it can be done. I suspect that if more people were using the possibility of broadcasting in RTTY mode directly from mcHF, then many things would have been detected earlier.

SP5QAD commented 4 years ago

All characters from the list specified above, given from the program fldigi, have been correctly decoded in mcHF (fw v. 2.11.86)

SP5QAD commented 4 years ago

Quick test v. 2.18.87: broadcast mchf

reception:

  1. fldigi: digits: ok, letters: ok, characters: almost all: single quotation mark (\ ') is decoded as text: "\ <BEL >".

  2. MultiPSK: digits: 1-9 ok, 0 as Ř letters: ok, signs:

    • ";" as "=",
    • double quotation marks as "+".
    • single quotation mark decoded correctly.

-- Marcin

SP5QAD commented 4 years ago

It seems that the erroneous operation has been corrected. Thank you to everyone who contributed to this.