TqdmLoggingHandler requires multiprocessing

[Geekfish]

Hello there!

I am running ffmpeg-normalize on AWS lambda, python 3.8 runtime. I have run into the following issue with the logger:

Expected behavior

When I run normalization with default settings, I expect to see a warning printed out about the default sample rate, for example:

[WARNING]   The sample rate will automatically be set to 192 kHz by the loudnorm filter. Specify -ar/--sample-rate to override it.

Actual behavior Instead, I get back an error trace, such as this:

--- Logging error ---
Traceback (most recent call last):
File "/var/task/ffmpeg_normalize/_logger.py", line 18, in emit
tqdm.set_lock(Lock())
File "/var/lang/lib/python3.8/multiprocessing/context.py", line 68, in Lock
return Lock(ctx=self.get_context())
File "/var/lang/lib/python3.8/multiprocessing/synchronize.py", line 162, in __init__
SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
File "/var/lang/lib/python3.8/multiprocessing/synchronize.py", line 57, in __init__
sl = self._semlock = _multiprocessing.SemLock(
OSError: [Errno 38] Function not implemented
Call stack:
# .....
File "/var/task/ffmpeg_normalize/_media_file.py", line 162, in run_normalization
for _ in self._second_pass():
File "/var/task/ffmpeg_normalize/_media_file.py", line 304, in _second_pass
logger.warn(
File "/var/lang/lib/python3.8/logging/__init__.py", line 1463, in warn
self.warning(msg, *args, **kwargs)
File "/var/lang/lib/python3.8/logging/__init__.py", line 1458, in warning
self._log(WARNING, msg, args, **kwargs)
File "/var/lang/lib/python3.8/logging/__init__.py", line 1589, in _log
self.handle(record)
File "/var/lang/lib/python3.8/logging/__init__.py", line 1599, in handle
self.callHandlers(record)
File "/var/lang/lib/python3.8/logging/__init__.py", line 1661, in callHandlers
hdlr.handle(record)
File "/var/lang/lib/python3.8/logging/__init__.py", line 954, in handle
self.emit(record)
File "/var/task/ffmpeg_normalize/_logger.py", line 24, in emit
self.handleError(record)
Message: 'The sample rate will automatically be set to 192 kHz by the loudnorm filter. Specify -ar/--sample-rate to override it.'
Arguments: ()

This is just an example, but the issue can apply to other logging statements in this library.

Cause I have looked into the cause of it, and it seems to be the following:

• There is a custom logger in this library, namely TqdmLoggingHandler. I believe this was added so that we can log things without breaking the progress bar.
• The custom logger depends on multiprocessing.Lock (here's the commit that introduced it).
• Unfortunately, some environments, like the AWS lambda python runtime, lack multiprocessing support. In my specific case, the lambda container lacks shared memory (there is no /dev/shm). You can read more about this here.

Possible Fix

tqdm seems to already support environments with no multiprocessing support. This is their solution for lack of multiprocessing in Jython:

https://github.com/tqdm/tqdm/commit/d1187ac5464fd5896361d4f1c8a9ed91c9e54b57

I think we could use a similar approach here and only set the lock when multiprocessing is available?

If you agree with this solution or have any other ideas or insights on how to solve this, I would be happy to work on a PR!

Environment (please complete the following information):

• [X] Your operating system: Amazon Linux 2 (lambda with python 3.8 runtime)
• [X] Your Python version / distribution (python --version): 3.8.11
• [X] Your ffmpeg version (ffmpeg -version): 1.22.2

[slhck]

Thank you for this detailed error description.

I wasn't aware that this function wasn't available in some environments. Your proposal sounds like a good workaround for such platforms, and it shouldn't interfere with the existing behavior. If you have the time to create a PR that fixes this, I'd very much appreciate it!

[Geekfish]

Thank you for the superfast response @slhck ! I'll open a PR to address it.

[slhck]

Fixed by #157 and released in v1.22.3.