Component stops working after a while if environment is too silent

[maxlaverse]

When there has been silence for a certain amount of time, the ffmpeg_noise component stops detecting any further noise. I'm using Home Assistant 0.117.5 with Python 3.8.2 on Linux 5.4.51-v7+. The input is a USB Microphone.

- platform: ffmpeg_noise
  name: Living room Noise
  input: -f alsa -i default:CARD=Microphone
  peak: -33

The ffmpeg_noise component of Home Assistant starts a ffmpeg process and parses its standard outputs looking for certain patterns. The vast majority of what ffmpeg outputs (when started by this components) can be divided in two categories:

• lines indicating that a silence has started or ended, e.g [silencedetect @ 0x1023f10] silence_start: 0.000291667
• progress information with the time since we started reading the input, size=N/A time=00:00:02.68 bitrate=N/A speed=0.997x

The loop inside haffmpeg processing ffmpeg's output is the following:

# https://github.com/home-assistant-libs/ha-ffmpeg/blob/master/haffmpeg/core.py#L222-L228
[...]
        while self.is_running:
            try:
                line = await self._input.readline()
                if not line:
                    break
                line = line.decode()
            except Exception:  # pylint: disable=broad-except
                break

            match = True if pattern is None else cmp.search(line)
            if match:
                _LOGGER.debug("Process: %s", line)
                await self._que.put(line)
[...]

I believe the problem is the following. The loop is using a readline() call that only returns when a new line separator is read from the output. A log line with a new line separator is printed when ffmpeg detects the start or the end of the silence. When the situation is not changing, meaning it keeps being noisy or it keeps being silent, the only output from ffmpeg are the progress information. Those are not using a new line separator but overwrite the same line with a carriage return character. As an effect, during long silences the main loop is waiting for readline() to return, waiting for a new line separator. In the meantime however, the buffer keeps growing with progress information. Eventually the buffer is saturated, an Exception is raised and the main loop exits. According to the documentation the buffer's size is supposed to be 64KiB.

I don't know which approach you'd like to take. Ideally, I would have wished for a buffer that is smart enough to detect the carriage return and erase the previous line, instead of just appending data. I'm not Python fluent and I searched a bit over Internet, but I couldn't find anything in that direction. As a workaround I patched my installation by emptying part of the buffer when it's full with the following additional exception handler.

[...]
                if not line:
                    break
                line = line.decode()
            except ValueError as err:
                await self._input.readexactly(1024)
                continue

            except Exception:  # pylint: disable=broad-except
                break
[...]

Since the meaningful payload is usually less than a 100 bytes, removing the first 1024 bytes of a 64KiB buffer full with most likely useless data sounds acceptable to me.

The following issues might be related: https://github.com/home-assistant-libs/ha-ffmpeg/issues/30#issuecomment-714265971, https://github.com/home-assistant/core/issues/42204

[maxlaverse]

By the way, to find out if Home Assistant is hanging in such a situation, you can try the following:

# Look for the PID of ffmepg
$ pidof ffmpeg
29188

# Watch for the writes of ffmepg to the standard outputs
$ strace -f -p 29188  -s1024 -e trace=write
[...]
[pid 29188] write(2, "size=N/A time=00:14:05.77 bitrate=N/A speed=   1x    \r", 54) = 54

# It should be hanging. I suppose at least haffmpeg's buffer is full, but another buffer in the
# process might be as well. In another window, read the standard output:
$ cat /proc/29188/fd/2

# That clears the other buffer and strace shows ffmpeg writing to the output again

As a side note, I experimented around that problem by artificially reducing the size of the buffer in https://github.com/home-assistant-libs/ha-ffmpeg/blob/master/haffmpeg/core.py#L175, replacing

reader = asyncio.StreamReader(loop=self._loop)

with

reader = asyncio.StreamReader(loop=self._loop, limit=2048)

(If I recall correctly)

[rjlee]

I can confirm I have experienced this issue as well and the above patch fixes it. Good job on diagnosing it, thanks.

[rjlee]

I've created a pull request for the above fix in https://github.com/home-assistant-libs/ha-ffmpeg/pull/33

[maxlaverse]

Actually, an even better fix was to add extra_arguments: -nostats to the ffmpeg_noise configuration in Home Assistant.

We could add this switch by default when starting the process if we don't expect any other component (e.g camera, sensor) to ever use this kind of information from the output. That would be a breaking change however.

[zandadoum]

hello.

this is still happening.

HA 2021.2.3 (docker on a Synology NAS) FFMPEG 4.3.1

apparently there seems to be a workaround mentioned above by @maxlaverse could anyone help me implement that into my installation?

[rjlee]

Add extra_arguments: -nostats to your ffmpeg_noise configuration, there's an example config here: https://github.com/rjlee/ha_noise_sensor

[zandadoum]

@rjlee added that last night. woke up and sensor is still working. simple. easy. brilliant! thank you a lot!

[maxlaverse]

Closing in favor of https://github.com/home-assistant/core/issues/49183