g4klx / MMDVMHost

The host program for the MMDVM
GNU General Public License v2.0
378 stars 275 forks source link

FM Network with RAW audio through UDP. #782

Closed EU4BNC closed 9 months ago

EU4BNC commented 1 year ago

Hello. Now I use mmdvm in fm mode in conjunction with SVXlink through USRP. But perhaps it is possible to use RAW data via UDP? Is it possible to do this?

sp2ong commented 1 year ago

SVXLink with UDP RX and TX works good with GnuRadio: https://www.pg540.org/wiki/index.php/LimeSDR_interface_to_SVXlink_using_GNUradio Maybe MDVMHost expects that the SOURCE port from svxlink will be the one specified in [FM Network] in LocalPort?? but svxlink uses source port random

g4klx commented 1 year ago

The message comes about because an incoming packet, from SVXLink, has arrived from an IP address and/or port that isn’t listed in the ini file as being where the gateway is. This is a security feature. Can you check to see what the correct address and port for SVXLink is, and is it the same for transmit and receive UDP packets?

Sent from Yahoo Mail for iPhone

On Wednesday, October 18, 2023, 18:56, Waldek @.***> wrote:

Hi, I have the same "problem" ? I don't know what this information in the log means.

I: 2023-10-18 17:47:40.360 MMDVMHost-20231017 is starting I: 2023-10-18 17:47:40.360 Built 17:19:49 Oct 18 2023 (GitID #e91f640) I: 2023-10-18 17:47:40.360 General Parameters I: 2023-10-18 17:47:40.360 Callsign: SP2ONG I: 2023-10-18 17:47:40.360 Id: 123456 I: 2023-10-18 17:47:40.361 Duplex: yes I: 2023-10-18 17:47:40.361 Timeout: 180s I: 2023-10-18 17:47:40.361 D-Star: disabled I: 2023-10-18 17:47:40.361 DMR: disabled I: 2023-10-18 17:47:40.361 YSF: disabled I: 2023-10-18 17:47:40.361 P25: disabled I: 2023-10-18 17:47:40.361 NXDN: disabled I: 2023-10-18 17:47:40.361 M17: disabled I: 2023-10-18 17:47:40.361 POCSAG: disabled I: 2023-10-18 17:47:40.361 FM: enabled I: 2023-10-18 17:47:40.361 AX.25: disabled I: 2023-10-18 17:47:40.361 Modem Parameters I: 2023-10-18 17:47:40.361 Protocol: uart I: 2023-10-18 17:47:40.361 UART Port: /dev/ttyACM0 I: 2023-10-18 17:47:40.361 UART Speed: 460800 I: 2023-10-18 17:47:40.361 RX Invert: no I: 2023-10-18 17:47:40.361 TX Invert: yes I: 2023-10-18 17:47:40.361 PTT Invert: no I: 2023-10-18 17:47:40.361 TX Delay: 100ms I: 2023-10-18 17:47:40.361 RX Offset: 0Hz I: 2023-10-18 17:47:40.361 TX Offset: 0Hz I: 2023-10-18 17:47:40.361 RX DC Offset: 0 I: 2023-10-18 17:47:40.361 TX DC Offset: 0 I: 2023-10-18 17:47:40.361 RF Level: 100.0% I: 2023-10-18 17:47:40.361 DMR Delay: 0 (0.0ms) I: 2023-10-18 17:47:40.361 RX Level: 50.0% I: 2023-10-18 17:47:40.361 CW Id TX Level: 50.0% I: 2023-10-18 17:47:40.361 D-Star TX Level: 50.0% I: 2023-10-18 17:47:40.361 DMR TX Level: 50.0% I: 2023-10-18 17:47:40.361 YSF TX Level: 50.0% I: 2023-10-18 17:47:40.361 P25 TX Level: 50.0% I: 2023-10-18 17:47:40.361 NXDN TX Level: 50.0% I: 2023-10-18 17:47:40.361 M17 TX Level: 50.0% I: 2023-10-18 17:47:40.361 POCSAG TX Level: 50.0% I: 2023-10-18 17:47:40.361 FM TX Level: 50.0% I: 2023-10-18 17:47:40.361 AX.25 TX Level: 50.0% I: 2023-10-18 17:47:40.361 TX Frequency: 435000000Hz (435000000Hz) I: 2023-10-18 17:47:40.361 Use COS as Lockout: no I: 2023-10-18 17:47:40.361 FM Parameters I: 2023-10-18 17:47:40.361 Callsign: SP2ONG I: 2023-10-18 17:47:40.362 Callsign Speed: 20WPM I: 2023-10-18 17:47:40.362 Callsign Frequency: 1000Hz I: 2023-10-18 17:47:40.362 Callsign Time: 10mins I: 2023-10-18 17:47:40.362 Callsign Holdoff: 1/0 I: 2023-10-18 17:47:40.362 Callsign High Level: 50.0% I: 2023-10-18 17:47:40.362 Callsign Low Level: 20.0% I: 2023-10-18 17:47:40.362 Callsign At Start: no I: 2023-10-18 17:47:40.362 Callsign At End: no I: 2023-10-18 17:47:40.362 Callsign At Latch: no I: 2023-10-18 17:47:40.362 RF Ack: K I: 2023-10-18 17:47:40.362 Ack Speed: 20WPM I: 2023-10-18 17:47:40.362 Ack Frequency: 1750Hz I: 2023-10-18 17:47:40.362 Ack Min Time: 4s I: 2023-10-18 17:47:40.362 Ack Delay: 1000ms I: 2023-10-18 17:47:40.362 Ack Level: 50.0% I: 2023-10-18 17:47:40.362 Timeout: 180s I: 2023-10-18 17:47:40.362 Timeout Level: 80.0% I: 2023-10-18 17:47:40.362 CTCSS Frequency: 88.5Hz I: 2023-10-18 17:47:40.362 CTCSS High Threshold: 30 I: 2023-10-18 17:47:40.362 CTCSS Low Threshold: 30 I: 2023-10-18 17:47:40.362 CTCSS Level: 20.0% I: 2023-10-18 17:47:40.362 Kerchunk Time: 30s I: 2023-10-18 17:47:40.362 Hang Time: 30s I: 2023-10-18 17:47:40.362 Access Mode: 1 I: 2023-10-18 17:47:40.362 Link Mode: no I: 2023-10-18 17:47:40.362 COS Invert: no I: 2023-10-18 17:47:40.362 Noise Squelch: no I: 2023-10-18 17:47:40.362 RF Audio Boost: x1 I: 2023-10-18 17:47:40.362 Max. Deviation Level: 90.0% I: 2023-10-18 17:47:40.362 Mode Hang: 10s I: 2023-10-18 17:47:40.362 Ext. Ack: N I: 2023-10-18 17:47:40.362 Ext. Audio Boost: x1 M: 2023-10-18 17:47:40.362 Opening the MMDVM I: 2023-10-18 17:47:42.377 MMDVM protocol version: 2, description: MMDVM 20221121 12.0000 MHz GitID #508018c I: 2023-10-18 17:47:42.377 CPU: ST-Micro ARM, UDID: 4B0025000351383032333330 I: 2023-10-18 17:47:42.377 Modes: DMR FM I: 2023-10-18 17:47:42.438 Display Parameters I: 2023-10-18 17:47:42.438 Type: None W: 2023-10-18 17:47:42.438 No valid display found, disabling I: 2023-10-18 17:47:42.438 Opening network connections I: 2023-10-18 17:47:42.438 FM Network Parameters I: 2023-10-18 17:47:42.438 Protocol: RAW I: 2023-10-18 17:47:42.438 Sample Rate: 48000 I: 2023-10-18 17:47:42.438 Gateway Address: 127.0.0.1 I: 2023-10-18 17:47:42.438 Gateway Port: 4810 I: 2023-10-18 17:47:42.438 Local Address: 127.0.0.1 I: 2023-10-18 17:47:42.438 Local Port: 3810 I: 2023-10-18 17:47:42.438 Pre-Emphasis: yes I: 2023-10-18 17:47:42.439 De-Emphasis: yes I: 2023-10-18 17:47:42.439 TX Audio Gain: 1.00 I: 2023-10-18 17:47:42.439 RX Audio Gain: 0.20 I: 2023-10-18 17:47:42.439 Mode Hang: 3s M: 2023-10-18 17:47:42.439 Opening FM network connection I: 2023-10-18 17:47:42.439 Opening UDP port on 3810 I: 2023-10-18 17:47:42.439 RSSI I: 2023-10-18 17:47:42.439 Mapping File: RSSI.dat I: 2023-10-18 17:47:42.439 Loaded 0 RSSI data mapping points from RSSI.dat I: 2023-10-18 17:47:42.439 Starting protocol handlers I: 2023-10-18 17:47:42.439 MMDVMHost-20231017 is running M: 2023-10-18 17:51:22.244 FM packet received from an invalid source M: 2023-10-18 17:51:22.265 FM packet received from an invalid source ......

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

EU4BNC commented 1 year ago

When analyzing the traffic, it is clear that data for MMDVMHost leaves from a different port (the port is different from the one specified in Gateway).

Here is an example package from MMDVMHost in SVXLink. Port 3811(mmdvmhost) sends data to port 4811(SVX). 19:18:40.398721 IP (tos 0x0, ttl 64, id 8543, offset 0, flags [DF], proto UDP (17), length 668) localhost.3811 > localhost.4811: [bad udp cksum 0x009c -> 0x6b7c!] UDP, length 640

Here is an example package from SVXLink at MMDVMHost.

Port !!!39605(SVX) sends data to port 3811(MMDVMHost)

20:41:36.843420 IP (tos 0x0, ttl 64, id 62422, offset 0, flags [DF], proto UDP (17), ### length 1052) localhost.39605 > localhost.3811: [bad udp cksum 0x021c -> 0xed42!] UDP, length 1024

SVXLink port is 4811, but outgoing data comes from port 39605. Outgoing port may be different for each transmission.

EU4BNC commented 1 year ago

Is it possible to make a function to disable protection?

g4klx commented 1 year ago

I have modified the host to allow more flexibility in RAW mode, but I won’t completely open it up as security is extremely important. Give the latest GitHub version a go and see how it goes please.

Sent from Yahoo Mail for iPhone

On Wednesday, October 18, 2023, 19:56, EU4BNC @.***> wrote:

Is it possible to make a function to disable protection?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

EU4BNC commented 1 year ago

Thank you very much, I'll try it now.

EU4BNC commented 1 year ago

Great! First success! The data went from SVXLink to MMDVMHost correctly, all that remains is to figure out why SVXLink does not respond to data from MMDVMHost.

sp2ong commented 1 year ago

Maybe the level of audio from MmdvmHost is too low to open Vox in svxlink try gain audio in MMDVMHost

EU4BNC commented 1 year ago

I would like to express my deep gratitude for the work done, the addition of the RAW protocol and Resampler.

EU4BNC commented 1 year ago

We can ask you to add one more modification. SVXLink incorrectly handles SQUELCH DETECTION when traffic on the incoming port is lost. This causes SQUELCH to open when traffic appears and not close when traffic disappears. SVXLink has the ability to control SQUELCH via PTY. If "O" is received on the PTY device (e.g. PTY_PATH=/tmp/sql), SQUELCH will be opened, if "Z" is sent to the device (e.g. PTY_PATH=/tmp/sql), SQUELCH will be closed. Is it possible to add external SQUELCH control via PTY to MMDVMHost? After the traffic starts broadcasting to the Gateway port, we will send O to PTY. Before the end of broadcasting packets to PTY, we will send Z.

GATEWAY_SQL_PTY=/tmp/sql GATEWAY_SQL_PTY_OPEN=O GATEWAY_SQL_PTY_CLOSE=Z

EU4BNC commented 1 year ago

This is an extract from the SVXLINK instructions: The PTY squelch expects a very easy protocol over a pseudo-tty device, created by SvxLink on runtime. An ’O’ over this pty device indicate an open squelch, a ’Z’ is a closed squelch. Define the slave pty in PTY_PATH (e.g. PTY_PATH=/tmp/sql) and SvxLink will create a link to the specified path from it’s pseudotty slave device (/dev/pts/X). This can be used by a script to interface custom devices, modems or other hardware to SvxLink. Look for nhrcx.pl to see an example.

shawnchain commented 1 year ago

I think this could be move to a SVXNetwork. BTW, I would like to ask if any have any doc about the FM 'USRP' protocol ? Currently I'm using the FMNetwork in another situation to communicate with my own implementation of FMReflector. I found the simple USRP is lacking of ping/keep-alive mechanism, nor could I find any clue about FM 'USRP' protocol.

sp2ong commented 1 year ago

We discussed this problem https://github.com/sm0svx/svxlink/issues/644 It looks like svxlink required incoming UDP audio with an option which is used in svxlink ASYNC_AUDIO_UDP_ZEROFILL _AudioDeviceUDP can now write zeros to the UDP connection on underflow. That is, when the "audio device" is open but there is no audio to write zeros will be written instead. Enable this behavior by setting the environment variable ASYNC_AUDIO_UDPZEROFILL=1

But how to do/implement this in a UDP stream in MDVMHost, I don't know.

We want to inerconnect MMDVMHost in FM MODE with svxlink via RAW UDP because at the current branch svxlink USRP not support decoding DTMF via USRP and does not send any audio message from modules etc. RAW UDP Audio will allow full integration MMDVMHost with the possibility svxlink. We use SVXReflector with svxlink which has a nice possibility in FM Network for example you can see on the example dashboard FM Network: https://svxportal.sm2ampr.net/ or http://www.fm-poland.pl/dashboard/

sp2ong commented 1 year ago

It appears that the audio from MDVMHost is being sent as much as it has and SVXLink is expecting a full buffer? and if the incoming sound does not fill the full buffer, it waits and therefore is not able to consider that "SQL" is closed. But I am just guessing that this is what happens

g4klx commented 1 year ago

That sounds extremely stupid of SVXLink. If you can find the buffer length that they're expecting, please let me know. On Tuesday, 24 October 2023 at 13:04:55 BST, Waldek @.***> wrote:

It appears that the audio from MDVMHost is being sent as much as it has and SVXLink is expecting a full buffer? and if the incoming sound does not fill the full buffer, it waits and therefore is not able to consider that "SQL" is closed. But I am just guessing that this is what happens

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

sp2ong commented 1 year ago

I don't know, but I assume that based on the behavior of receiving UDP RAW audio when the TX option ASYNC_AUDIO_UDP_ZEROFILL=1 is used and then it works. Opening and closing SQL and when I do not use the ASYNC_AUDIO_UDP_ZEROFILL option, i.e. I set it to 0, then when AUDIO comes there is information about opening SQL and after the UDP transmission is completed, there is no information about closing the SQL, which blocks further operation. Probably the author of svxlink would answer these doubts...

g4klx commented 1 year ago

I've added the squelch file handling. Can you give it a try and report back please?

EU4BNC commented 1 year ago

I've added the squelch file handling. Can you give it a try and report back please?

I'll try it ib evening and report here. Thanks to you!

EU4BNC commented 1 year ago

I've added the squelch file handling. Can you give it a try and report back please?

I'm sorry, could you please explain how this setting works and what should be in the sql file.

sp2ong commented 1 year ago

You must add in MMDVM.ini SquelchFile

[FM Network] SquelchFile=/tmp/sql ....

and in svxlink.conf in [Rx1]

SQL_DET=PTY PTY_PATH=/tmp/sql

Note that MDVMHost and svxlink should be run by the same user due to the /tmp/sql PTY device permissions

Tell us whether the new solutions added by Jonathan work for you now.

Thanks, Jonathan, for your support

EU4BNC commented 1 year ago

You must add in MMDVM.ini SquelchFile

[FM Network]

The squelch file is optional and only used in RAW mode

SquelchFile=/tmp/sql ....

and in svxlink.conf in [Rx1]

SQL_DET=PTY PTY_PATH=/tmp/sql

Note that MDVMHost and svxlink should be run by the same user due to the /tmp/sql PTY device permissions

Tell us whether the new solutions added by Jonathan work for you now.

Thanks, Jonathan, for your support Understood. Thanks, I'll try it now. Thanks to the developer for your support!

EU4BNC commented 1 year ago

I've added the squelch file handling. Can you give it a try and report back please?

Thanks a lot! It works!

g4klx commented 1 year ago

Wow! Let me know if you need any more changes for it.

Maybe you can make an announcement on the OpenDV group and maybe a small guide on how to configure it, and what you can use it for.

Would that be possible?

EU4BNC commented 1 year ago

Wow! Let me know if you need any more changes for it.

Maybe you can make an announcement on the OpenDV group and maybe a small guide on how to configure it, and what you can use it for.

Would that be possible?

Yes, I will prepare a text to present this project a little later. Once again, thank you very much for your support and improvements. I will also have one more question regarding oddities in the quality of transmission of the audio stream to the network. I'll describe it a little later, I think it's related to the STM firmware.

Currently, our repeater network has MMDVM equipment installed with MOTOROLLA GM and CM series radios. DMR switching is carried out via Brandmeister and XLX. Analog traffic is switched through the SVXLink server with the ability to control groups via DTMF. This is such an interesting symbiosis.

f4hlv commented 1 year ago

Hi, thank you once again for the work done. The audio works but it's still a little cut up. Managing squelch SQL_DET=PTY can be complicated to implement svxlink and mmdvmhost are not on the same machine. I noticed that when I trigger analog mode with mmdvmhost after several hours, the relay sends all the svxlink ID beacons that have not been transmitted. It's like buffering.

A good alternative would be to be able to link svxreflector with the OPUS codec.

Thanks again, this is a project I've been waiting for for a long time. Vincent

sp2ong commented 1 year ago

A good alternative would be to be able to link svxreflector with the OPUS codec.

I think that such a solution is not realistic because you would have to implement the entire mechanism for handling communication with svxreflector, which is in svxlink, and you would have to constantly monitor changes in the svxlnik code to have a current implementation of such a solution, and this is not the goal of MDVMHost. MDVMHost can send the analog audio stream to software such as svxlink and it takes care of the rest of the implementation of communication with svxrelfector, group management, etc. Rather, the author of svxlink could also show interest in solving any problems in the integration of this solution. We had a long discussion on github svxlink about this topic, but unfortunately there is no response from the author of svlink, even advice on how to approach this problem.

XATTA6 commented 1 year ago

hello everyone! I have svxlibk in analog mode, after refining mmdvmhost, I want to note that I really like the sound quality coming through udp, I have the same board myself and I think that the work done is just fine, we got dtmf and alerts plus the quality standards, which are controlled through mmdvmhost and svxlibk, so those who don't like the changes I suggest not to use them.