gstreamer-superficial blocks libuv workers

[jamespepplinkhouse]

We had an issue when using many instances of gstreamer-superficial (e.g. new gstreamer.Pipeline) on a single Node.js instance, where dns and fs appear to stop working and never recover. I am sharing this as the problem was confusing and hard to track down. We use multiple gstreamer-superficial instances for the purposes of recording many incoming WebRTC streams.

TL;DR

As long as you set the environment variable UV_THREADPOOL_SIZE greater than the number of concurrent gstreamer-superficial pipelines, you'll have normal stable performance.

Explanation

(please correct me if this is not correct)

gstreamer-superficial uses nan AsyncWorker which consumes a thread in the libuv thread pool, so when you create many of them, they exhaust the thread pool leading to:

• Can't make dns lib requests - they never get sent to the network interface
• Can't make fs lib calls - nothing happens

The thread blocking may be related/limited to the use of pipeline.pollBus (unconfirmed). Anyway, as default UV_THREADPOOL_SIZE=4 so if you're going to expect 50 pipelines concurrently, set something like UV_THREADPOOL_SIZE=64 and you'll be fine. As stated here, the max value for newer Node.js versions is 1024 and the it uses approx. "~1MB for 128 threads", so it's a nice fix with little other impact. 👍

[dturing]

your reasoning looks right to me. thanks for writing this down here for others to find!

[pablito25sp]

I've just faced the exact same issue. Thanks for sharing @jamespepplinkhouse