Deadlock from asyncio when a single audio file is read at the same time

[tae-jun]

Problematic behavior

If two identical audio files are read at the same time using asyncio, it gets deadlock.

To Reproduce the issue:

import asyncio
from pedalboard.io import ReadableAudioFile

PATH = "<ANY AUDIO FILE>"

async def read(f: ReadableAudioFile):
  loop = asyncio.get_running_loop()
  num_frames = int(f.samplerate)
  print(f'reading {f}')
  result = await loop.run_in_executor(None, f.read, num_frames)
  print(f'done {f}')
  return result

async def main():
  f1 = ReadableAudioFile(PATH)
  f2 = ReadableAudioFile(PATH)

  t1 = asyncio.create_task(read(f1))
  t2 = asyncio.create_task(read(f2))

  print('Waiting tasks...')
  await t1
  await t2
  print('Done!')

asyncio.run(main())

It prints:

Waiting tasks...
reading <pedalboard.io.ReadableAudioFile filename="PATH" samplerate=48000 num_channels=2 frames=2304000 file_dtype=int24 at 0x600000b24018>
reading <pedalboard.io.ReadableAudioFile filename="PATH" samplerate=48000 num_channels=2 frames=2304000 file_dtype=int24 at 0x600000b3c228>

and then the program got stuck. There is no error message and it doesn't end.

Though, you can fix it using asyncio.Lock:

lock = asyncio.Lock()

async def read(f: ReadableAudioFile):
  async with lock:
    loop = asyncio.get_running_loop()
    num_frames = int(f.samplerate)
    print(f'reading {f}')
    result = await loop.run_in_executor(None, f.read, num_frames)
    print(f'done {f}')
  return result

It works as expected.

Environments

• OS: macOS, also tested on Docker image python:3.10-slim-bullseye
• Python version: 3.10.4
• Pedalboard version: 0.6.2

Comments

Pedalboard is thread-safe so I thought it also works for coroutines, but it seems it's not in the example above. Did I miss something? Or is it a bug? It will be appreciated if you can help! 🙂

[psobot]

Wow, great find! Thanks for the bug report; I can confirm and reproduce this issue on my end.

The core issue seems to be a deadlock on the GIL when trying to create the NumPy array to return from ReadableAudioFile::read. Here's an amended stack trace:

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib                 0x1b1e7a270 __psynch_cvwait + 8
1   libsystem_pthread.dylib                0x1b1eb483c _pthread_cond_wait + 1236
2   Python                                 0x102ab375c take_gil + 556

Thread 3:
0   libsystem_kernel.dylib                 0x1b1e7a270 __psynch_cvwait + 8
1   libsystem_pthread.dylib                0x1b1eb483c _pthread_cond_wait + 1236
2   Python                                 0x102ab375c take_gil + 556
3   Python                                 0x102b44dfc posix_do_stat + 204
4   Python                                 0x102b3624c os_stat + 188
[snip]
65  Python                                 0x102ae8304 PyImport_Import + 300
66  Python                                 0x102ae817c PyImport_ImportModule + 60
67  pedalboard_native.cpython-310-darwin.so        0x103a3f6e8 pybind11::detail::npy_api::lookup() + 44

Thread 4:
0   libsystem_kernel.dylib                 0x1b1e7a270 __psynch_cvwait + 8
1   libsystem_pthread.dylib                0x1b1eb483c _pthread_cond_wait + 1236
2   libc++abi.dylib                        0x1b1e71c20 __cxa_guard_acquire + 140
3   pedalboard_native.cpython-310-darwin.so        0x103a9a4a8 pybind11::array::array<float>(pybind11::detail::any_container<long>, pybind11::detail::any_container<long>, float const*, pybind11::handle) + 292

I'm not familiar with asyncio, but I can confirm that doing import numpy in your program (before calling any Pedalboard functions) fixes the issue. I've opened #155 to resolve this issue in Pedalboard v0.6.3.

[psobot]

The fix for this issue has now been released as part of Pedalboard v0.6.3.

[tae-jun]

Thanks for your quick response and fix! After importing Numpy first, it works for me as well.

Thanks a lot!