versatica / mediasoup

Cutting Edge WebRTC Video Conferencing
https://mediasoup.org
ISC License
6.27k stars 1.13k forks source link

mediasoup-worker linux prebuilt with io-uring support (part 1) #1283

Closed ibc closed 10 months ago

ibc commented 10 months ago

This PR exposes kernel mayor number in the mediasoup-worker binary name to make it possible to have prebuilts for Linux with kernel 5 and 6 (the latter supports io-uring).

NOTE 1: This is related to issue #1282 but doesn't complete it.

TODO: In mediasoup-worker-prebuild.yaml we must include an Ubuntu version with kernel >= 6. Problem is that Ubuntu 22.04 still uses kernel 5.15 and there are no other more modern Ubuntu versions in GitHub CI.

NOTE 2: When that is done (in part 2 of this task), we must still keep Ubuntu 20.04 since, as documented in the job file, it uses an older version of GLib and otherwise we'd run into this issue again: https://github.com/versatica/mediasoup/issues/1089.

ibc commented 10 months ago

I'm not opposed to merging this, but major version doesn't really mean anything in Linux. And there is a pretty wild range of features available between various revisions of each major version.

I think this will have to become more sophisticated over time to make sure proper versions are accounted for.

I'm also wondering how hard it would be to compile a worker that can make decision about whether to enable io_uring in runtime.

Then it would be possible to install kernel headers and such from newer version even on older kernel and compile it with io_uring support on 20.04, while not using it in runtime if kernel is too old.

In summary:

  1. liburing can only be built if the host in which the worker is built has kernel > 6.
  2. Later that mediasoup-worker prebuilt could work in a kernel 5. In that case liburing is disabled in runtime.
  3. Bullet 1 is a requirement for bullet 2.
nazar-pc commented 10 months ago

liburing can only be built if the host in which the worker is built has kernel > 6.

I do not necessarily think that is the case. Headers from newer kernel are probably required, but the kernel running as such. It can be installed on a system running older kernel and with some tweaks to default library search paths compiler should be able to see it.

ibc commented 10 months ago

liburing can only be built if the host in which the worker is built has kernel > 6.

I do not necessarily think that is the case. Headers from newer kernel are probably required, but the kernel running as such. It can be installed on a system running older kernel and with some tweaks to default library search paths compiler should be able to see it.

Not sure why such a effort is needed. We only need a kernel >= 6 in CI which is not available yet.

ibc commented 10 months ago

NOTE: as a side effect of this PR part 1, those installing mediasoup node in Debian or Ubuntu or RedHat etc with kernel >= 6 will not have prebuilt worker available so will build worker from scratch, and hence with io-uring support, which IMHO is good. What do you think @jmillan?

jmillan commented 10 months ago

I think this should not be needed. Once GH action runners run kernel 6 liburing will be built automatically. Until that happens (unless we manage to compile it with proper headers from an older linux) then liburing will be disabled in the prebuilt binaries.

How will this PR help in any way?

ibc commented 10 months ago

I think this should not be needed. Once GH action runners run kernel 6 liburing will be built automatically. Until that happens (unless we manage to compile it with proper headers from an older linux) then liburing will be disabled in the prebuilt binaries.

How will this PR help in any way?

  1. We need a worker prebuilt made with Ubuntu 20.04 due to the GLib issue (see the link in the PR description), so it can work in not super recent Linux hosts, let's say those running kernel 5 (hopefully).
  2. This PR forces new Linux hosts (with kernel 6) to compile worker from scratch since there won't be (yet) prebuilt for them, so the worker will support io-uring.
jmillan commented 10 months ago

I think this should not be needed. Once GH action runners run kernel 6 liburing will be built automatically. Until that happens (unless we manage to compile it with proper headers from an older linux) then liburing will be disabled in the prebuilt binaries.

How will this PR help in any way?

  1. We need a worker prebuilt made with Ubuntu 20.04 due to the GLib issue (see the link in the PR description), so it can work in not super recent Linux hosts, let's say those running kernel 5 (hopefully).
  2. This PR forces new Linux hosts (with kernel 6) to compile worker from scratch since there won't be (yet) prebuilt for them, so the worker will support io-uring.

Gotcha, rather than renaming the binary with the kernel version I think a more elegant way is that the binary itself can tell it, via command argument. Ie: mediasoup-worker -v [--version] or --info. The output of that command being, among others, whether io_uring is built-in.

ibc commented 10 months ago

Gotcha, rather than renaming the binary with the kernel version I think a more elegant way is that the binary itself can tell it, via command argument. Ie: mediasoup-worker -v [--version] or --info. The output of that command being, among others, whether io_uring is built-in.

For what? How would that help? And how is that related to the prebuilt fetching?

jmillan commented 10 months ago

Anyway I'm OK with this change.

ibc commented 10 months ago

Imagine we had Ubuntu 24.04 with kernel 6 in GitHub actions so we can produce io-using capable worker. But we still need to produce a prebuilt worker for Linux with older GLib (due to the referenced issue). So we need TWO prebuilt binaries so two different names. This is not about having "npm install mediasoup" downloading a "universal" binary and then checking some command on it to see if it was valid or not.

nazar-pc commented 10 months ago

I agree, we do want to be able to make decision before downloading