search

charlie-foxtrot / RTLSDR-Airband

Multichannel AM/NFM demodulator
GNU General Public License v3.0
748 stars 135 forks source link

[BUG] Random Disconnects from Broadcastify / Other Icecast Outputs #445

Open zigsphere opened 7 months ago

zigsphere commented 7 months ago

Using a Raspberry PI 3 and using the armv7-generic platform. 2 RTLSDR devices connected to a USB hub (powered)

I'm using the latest version 4.2.0 of rtl_airband. Raspbian GNU/Linux 12 (bookworm)

What happened?

I am getting random disconnects / reconnects to broadcastify and other icecast endpoints I am using. I am currently running 2 rtlsdr dongles and recently discoverd the multithreading - that is enabled. I am trying to understand why the disconnects happen. My internet has no issues - constantly pinging google as an example - no issues. Perhaps there is something I can adjust in my airband configuration? The connection drops 15 - 30 minutes after running. Load, CPU, memory are also fine on the PI.

What you expected to happen? Run rtl_airband and have no disconnects to broadcastify / other platforms for outputs.

Steps to Reproduce Start rtl_airband and just wait.

Additional context

     Loaded: loaded (/etc/systemd/system/rtl_airband.service; enabled;
 preset: enabled)
     Active: active (running) since Fri 2024-01-12 03:26:05 UTC; 57min
 ago
       Docs: https://github.com/charlie-foxtrot/RTLSDR-Airband/wiki
   Main PID: 1056 (rtl_airband)
      Tasks: 7 (limit: 1571)
        CPU: 40min 54.865s
     CGroup: /system.slice/rtl_airband.service
             └─1056 /usr/local/bin/rtl_airband -Fe

Jan 12 03:57:13 pi3 rtl_airband[1056]: Connecting to audio9.broadcasti
fy.com:80/XXXX...
Jan 12 03:57:14 pi3 rtl_airband[1056]: Connected to audio9.broadcastif
y.com:80/XXXX
Jan 12 04:07:10 pi3 rtl_airband[1056]: Lost connection to audio9.broad
castify.com:80/XXXX
Jan 12 04:07:14 pi3 rtl_airband[1056]: Trying to reconnect to audio9.b
roadcastify.com:80/XXXX...
Jan 12 04:07:14 pi3 rtl_airband[1056]: Connecting to audio9.broadcasti
fy.com:80/XXXX...
Jan 12 04:07:15 pi3 rtl_airband[1056]: Connected to audio9.broadcastif
y.com:80/XXXX
Jan 12 04:13:12 pi3 rtl_airband[1056]: Lost connection to audio1.broad
castify.com:80/XXXX
Jan 12 04:13:15 pi3 rtl_airband[1056]: Trying to reconnect to audio1.b
roadcastify.com:80/XXXX...
Jan 12 04:13:15 pi3 rtl_airband[1056]: Connecting to audio1.broadcasti
fy.com:80/XXXX...
Jan 12 04:13:16 pi3 rtl_airband[1056]: Connected to audio1.broadcastif
y.com:80/XXXX

Your rtl_airband.conf file

multiple_output_threads = true;
devices:
({
    type = "rtlsdr";
    index = 0;
    gain = 35;
    centerfreq = 121.520;
    correction = 0;
    sample_rate = 2.40;
    channels:
    (
      {
        freq = 122.800;
        modulation = "am";
        lowpass = -1;
        highpass = 300;
        label = "Roseburg Regional Airport (122.800)";
        bandwidth = 5000;
        ampfactor = 2.00;
        outputs: (
          {
            type = "icecast";
            server = "audio9.broadcastify.com";
            port = 80;
            mountpoint = "XXXX";
            name = "Roseburg Regional Airport Unicom/CTAF";
            genre = "Aviation";
            username = "source";
            password = "XXXX";
            },
          {
            type = "file";
            directory = "/tmp";
            filename_template = "airport";
          }
        );
      }
    );
  },
  {
    type = "rtlsdr";
    index = 1;
    gain = 49;
    centerfreq = 155.242;
    correction = 0;
    sample_rate = 2.40;
    channels:
    (
      {
        freq = 155.700;
        modulation = "nfm";
        #lowpass = 5000;
        highpass = 300;
        lowpass = -1;
        label = "Douglas County Sheriff Law 1 (155.700)";
        bandwidth = 5000;
        ctcss = 151.4;
        outputs: (
          {
            type = "file";
            directory = "/tmp";
            split_on_transmission = True;
            silence_release = 2.0;
            include_freq = true;
            filename_template = "law1";
          },
          {
            type = "icecast";
            server = "audio9.broadcastify.com";
            port = 80;
            mountpoint = "XXXX";
            name = "Douglas County Sheriff - Law 1";
            genre = "Public Safety";
            username = "source";
            password = "XXXX";
          }
        );
      },
      {
        freq = 155.550;
        modulation = "nfm";
        lowpass = -1;
        highpass = 300;
        label = "Douglas County Sheriff - Law 2";
        bandwidth = 5000;
        #ampfactor = 2.00;
        ctcss = 151.4;
        outputs: (
          {
            type = "file";
            directory = "/tmp";
            filename_template = "law2";
            split_on_transmission = True;
            silence_release = 2.0;
            include_freq = true;
          },
          {
            type = "icecast";
            server = "audio9.broadcastify.com";
            port = 80;
            mountpoint = "XXXX";
            name = "Douglas County Sheriff - Law 2";
            genre = "Public Safety";
            username = "source";
            password = "XXXX";
          }
        );
      },
      {
        freq = 155.775;
        modulation = "nfm";
        lowpass = -1;
        highpass = 300;
        label = "Oregon Department of Transportation District 7 (155.775)";
        bandwidth = 5000;
        ctcss = 100.0;
        outputs: (
          {
            type = "file";
            directory = "/tmp";
            filename_template = "odot";
            split_on_transmission = True;
            silence_release = 2.0;
            include_freq = true;
          },
          {
            type = "icecast";
            server = "audio9.broadcastify.com";
            port = 80;
            mountpoint = "XXXX";
            name = "Oregon Department of Transportation - District 7";
            genre = "Public Safety";
            username = "source";
            password = "XXXX";
          }
        );
      },
      {
        freq = 154.815;
        modulation = "nfm";
        lowpass = -1;
        highpass = 300;
        label = "Oregon State Police - Roseburg Station";
        bandwidth = 5000;
        ctcss = 131.8;
        outputs: (
          {
            type = "file";
            directory = "/tmp";
            filename_template = "osp";
            split_on_transmission = True;
            silence_release = 2.0;
            include_freq = true;
          },
          {
            type = "icecast";
            server = "audio9.broadcastify.com";
            port = 80;
            mountpoint = "XXXX";
            name = "Oregon State Police - Roseburg Station";
            genre = "Public Safety";
            username = "source";
            password = "XXXX";
          }
        );
      },
      {
        freq = 154.370;
        modulation = "nfm";
        lowpass = -1;
        highpass = 300;
        label = "Douglas County Fire Dispatch";
        bandwidth = 5000;
        ctcss = 151.4;
        outputs: (
          {
            type = "file";
            directory = "/tmp";
            filename_template = "fire";
            split_on_transmission = True;
            silence_release = 2.0;
            include_freq = true;
          },
          {
            type = "icecast";
            server = "audio1.broadcastify.com";
            port = 80;
            mountpoint = "XXXX";
            name = "Douglas County Fire Dispatch";
            genre = "Public Safety";
            username = "source";
            password = "XXXX";
          },
          {
            type = "icecast";
            server = "listen.theroseburgreceiver.com";
            port = 8015;
            mountpoint = "";
            name = "Douglas County Fire Dispatch - Direct";
            genre = "Public Safety";
            username = "source";
            password = "XXXX";
          }
        );
      }
    );
  }
 );
charlie-foxtrot commented 7 months ago

I have multiple units running both local icecast and broadcastify feeds and don't see this on a few that I checked. All of mine are running on pi 4s however, so I wonder if that is a factor . . .

In addition to multiple_output_threads = true you can also use multiple_demod_threads = true, this will run a different thread for each dongle and will help keep up with the raw data rate. I think on the pi 3 both ethernet and USB share the same bus, so perhaps a backlog there could cause issues?

If you recompile with native instead of armv7-generic that will tune to the hardware you're building on (presumably the pi itself), not sure if that will impact anything or not.

There are a bunch of stats captured and can be written to a file using stats_filepath = "/tmp/radio.prom", enabling that and watching for changes when the disconnects happen may be interesting.

zigsphere commented 7 months ago

Sounds good. Thank you. I also just received a new PI5. With that being the case, what would be the best platform to use when compiling? Ill give your suggestions a shot.

charlie-foxtrot commented 7 months ago

If you are compiling on the machine you will be running on, use native as the platform.

zigsphere commented 7 months ago

What I have noticed is a buffer overrun count:

# HELP channel_activity_counter Loops of output_thread with frequency active.
# TYPE channel_activity_counter counter
channel_activity_counter{freq="122.800",label="Roseburg Regional Airport (122.800)"}    0
channel_activity_counter{freq="155.700",label="Douglas County Sheriff Law 1 (155.700)"} 44
channel_activity_counter{freq="155.550",label="Douglas County Sheriff - Law 2"} 0
channel_activity_counter{freq="155.775",label="Oregon Department of Transportation District 7 (155.775)"}   0
channel_activity_counter{freq="154.815",label="Oregon State Police - Roseburg Station"} 0
channel_activity_counter{freq="154.370",label="Douglas County Fire Dispatch"}   0

# HELP channel_noise_level Raw squelch noise_level.
# TYPE channel_noise_level gauge
channel_noise_level{freq="122.800",label="Roseburg Regional Airport (122.800)"} 0.091
channel_noise_level{freq="155.700",label="Douglas County Sheriff Law 1 (155.700)"}  0.286
channel_noise_level{freq="155.550",label="Douglas County Sheriff - Law 2"}  0.286
channel_noise_level{freq="155.775",label="Oregon Department of Transportation District 7 (155.775)"}    0.285
channel_noise_level{freq="154.815",label="Oregon State Police - Roseburg Station"}  0.283
channel_noise_level{freq="154.370",label="Douglas County Fire Dispatch"}    0.250

# HELP channel_dbfs_noise_level Squelch noise_level as dBFS.
# TYPE channel_dbfs_noise_level gauge
channel_dbfs_noise_level{freq="122.800",label="Roseburg Regional Airport (122.800)"}    -48.739
channel_dbfs_noise_level{freq="155.700",label="Douglas County Sheriff Law 1 (155.700)"} -38.829
channel_dbfs_noise_level{freq="155.550",label="Douglas County Sheriff - Law 2"} -38.826
channel_dbfs_noise_level{freq="155.775",label="Oregon Department of Transportation District 7 (155.775)"}   -38.870
channel_dbfs_noise_level{freq="154.815",label="Oregon State Police - Roseburg Station"} -38.915
channel_dbfs_noise_level{freq="154.370",label="Douglas County Fire Dispatch"}   -39.979

# HELP channel_signal_level Raw squelch signal_level.
# TYPE channel_signal_level gauge
channel_signal_level{freq="122.800",label="Roseburg Regional Airport (122.800)"}    0.109
channel_signal_level{freq="155.700",label="Douglas County Sheriff Law 1 (155.700)"} 0.300
channel_signal_level{freq="155.550",label="Douglas County Sheriff - Law 2"} 0.317
channel_signal_level{freq="155.775",label="Oregon Department of Transportation District 7 (155.775)"}   0.309
channel_signal_level{freq="154.815",label="Oregon State Police - Roseburg Station"} 0.320
channel_signal_level{freq="154.370",label="Douglas County Fire Dispatch"}   0.300

# HELP channel_dbfs_signal_level Squelch signal_level as dBFS.
# TYPE channel_dbfs_signal_level gauge
channel_dbfs_signal_level{freq="122.800",label="Roseburg Regional Airport (122.800)"}   -47.194
channel_dbfs_signal_level{freq="155.700",label="Douglas County Sheriff Law 1 (155.700)"}    -38.404
channel_dbfs_signal_level{freq="155.550",label="Douglas County Sheriff - Law 2"}    -37.945
channel_dbfs_signal_level{freq="155.775",label="Oregon Department of Transportation District 7 (155.775)"}  -38.159
channel_dbfs_signal_level{freq="154.815",label="Oregon State Police - Roseburg Station"}    -37.853
channel_dbfs_signal_level{freq="154.370",label="Douglas County Fire Dispatch"}  -38.412

# HELP channel_squelch_counter Squelch open_count.
# TYPE channel_squelch_counter counter
channel_squelch_counter{freq="122.800",label="Roseburg Regional Airport (122.800)"} 0
channel_squelch_counter{freq="155.700",label="Douglas County Sheriff Law 1 (155.700)"}  1
channel_squelch_counter{freq="155.550",label="Douglas County Sheriff - Law 2"}  0
channel_squelch_counter{freq="155.775",label="Oregon Department of Transportation District 7 (155.775)"}    0
channel_squelch_counter{freq="154.815",label="Oregon State Police - Roseburg Station"}  0
channel_squelch_counter{freq="154.370",label="Douglas County Fire Dispatch"}    0

# HELP channel_squelch_level Squelch squelch_level.
# TYPE channel_squelch_level gauge
channel_squelch_level{freq="122.800",label="Roseburg Regional Airport (122.800)"}   0.274
channel_squelch_level{freq="155.700",label="Douglas County Sheriff Law 1 (155.700)"}    0.904
channel_squelch_level{freq="155.550",label="Douglas County Sheriff - Law 2"}    0.904
channel_squelch_level{freq="155.775",label="Oregon Department of Transportation District 7 (155.775)"}  0.900
channel_squelch_level{freq="154.815",label="Oregon State Police - Roseburg Station"}    0.895
channel_squelch_level{freq="154.370",label="Douglas County Fire Dispatch"}  0.792

# HELP channel_flappy_counter Squelch flappy_count.
# TYPE channel_flappy_counter counter
channel_flappy_counter{freq="122.800",label="Roseburg Regional Airport (122.800)"}  0
channel_flappy_counter{freq="155.700",label="Douglas County Sheriff Law 1 (155.700)"}   0
channel_flappy_counter{freq="155.550",label="Douglas County Sheriff - Law 2"}   0
channel_flappy_counter{freq="155.775",label="Oregon Department of Transportation District 7 (155.775)"} 0
channel_flappy_counter{freq="154.815",label="Oregon State Police - Roseburg Station"}   0
channel_flappy_counter{freq="154.370",label="Douglas County Fire Dispatch"} 0

# HELP channel_ctcss_counter count of windows with CTCSS detected.
# TYPE channel_ctcss_counter counter
channel_ctcss_counter{freq="122.800",label="Roseburg Regional Airport (122.800)"}   0
channel_ctcss_counter{freq="155.700",label="Douglas County Sheriff Law 1 (155.700)"}    13
channel_ctcss_counter{freq="155.550",label="Douglas County Sheriff - Law 2"}    0
channel_ctcss_counter{freq="155.775",label="Oregon Department of Transportation District 7 (155.775)"}  0
channel_ctcss_counter{freq="154.815",label="Oregon State Police - Roseburg Station"}    0
channel_ctcss_counter{freq="154.370",label="Douglas County Fire Dispatch"}  0

# HELP channel_no_ctcss_counter count of windows without CTCSS detected.
# TYPE channel_no_ctcss_counter counter
channel_no_ctcss_counter{freq="122.800",label="Roseburg Regional Airport (122.800)"}    0
channel_no_ctcss_counter{freq="155.700",label="Douglas County Sheriff Law 1 (155.700)"} 0
channel_no_ctcss_counter{freq="155.550",label="Douglas County Sheriff - Law 2"} 0
channel_no_ctcss_counter{freq="155.775",label="Oregon Department of Transportation District 7 (155.775)"}   0
channel_no_ctcss_counter{freq="154.815",label="Oregon State Police - Roseburg Station"} 0
channel_no_ctcss_counter{freq="154.370",label="Douglas County Fire Dispatch"}   0

# HELP buffer_overflow_count Number of times a device's buffer has overflowed.
# TYPE buffer_overflow_count counter
buffer_overflow_count{device="0"}   0
buffer_overflow_count{device="1"}   0

# HELP output_overrun_count Number of times a device or mixer output has overrun.
# TYPE output_overrun_count counter
output_overrun_count{device="0"}    0
output_overrun_count{device="1"}    9

The bandwidth for my feeds are set to 5000. Is that too big? There are only two dongles currently, but I plan to add a third.

Also, just as an update, this is now running on the PI5. Values multiple_output_threads = true and multiple_demod_threads = true have been set. Would the overrun count cause this?

When the drop occurred on icecast, I saw:

# HELP output_overrun_count Number of times a device or mixer output has overrun.
# TYPE output_overrun_count counter
output_overrun_count{device="0"}    0
output_overrun_count{device="1"}    48

Perhaps this is the culprit, but Im trying to understand why.

Thanks