docker-compose not working

[rosensvv]

I tried using the updated docker-compose file in compositor-proxy and I get this error in the logs while nothing happens onscreen: gstgl_gbm_utils.c:489:gst_gl_gbm_find_and_open_drm_node: Cannot open device node "/dev/dri/card0": Permission denied (13) 0:01:44.661486208 1 0x54ab000 ERROR gldisplay gstgldisplay_gbm.c:394:gst_gl_display_gbm_new: could not find or open DRM device 0:01:44.661815392 1 0x7f9a5c251680 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:<src:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id 0:01:44.666003528 1 0x7f9a5c2515e0 WARN glwindow gstglwindow.c:293:gst_gl_window_new: Could not create window. user specified gbm, creating dummy window {"level":30,"time":1648673918415,"pid":1,"hostname":"462ef0848770","name":"native-compositor-session","msg":"New websocket connected."} I have double-checked that the compositor-proxy is defined as privileged=true in the compose-file. What could be wrong?

[Zubnix]

What graphics card do you have? It is by any chance nvidia?

[rosensvv]

This is running in a machine that has an integrated intel GPU and an external nvidia graphics card (GTX 1060) over thunderbolt. I have removed the nouveau module using modprobe and the card is not used. I thought that greenfield doesn't support gpu acceleration yet? Even if it does, can it not use the intel GPU? I intend to have the external nvidia GPU not always connected to the server...

[Zubnix]

Greenfield does not support hardware acceleration (yet) for client applications, but it does use opengl to do efficient color conversion & transparency support, which is required for the video encoding step. Your error seems to come from gstreamer that is unable to create an egl display object using the GBM api. (An egl display object is needed so it can create an opengl rendering context without an underlying windowing system like X11 or Wayland).

The reason I was asking about the GPU is because the defacto standard on Linux for creating an egl display is by using the GBM api, which unfortunately is only very recently supported by nvidia (since driver 495). To hint gstreamer what API it should use, you can use the GST_GL_WINDOW environment variable. By default this is set to gbm, but you can also set it to egl-device in case you are using a GPU that does not support the GBM api (ie nvidia but afaik also works for intel cards).

Also very important is that nvidia also requires a modified docker setup to access the underlying hardware: https://github.com/NVIDIA/nvidia-docker

Anyway your error seems to indicate that the docker container can not open or find the /dev/dri/card0 path so I would suggest to first try without the nvidia card (as you are already doing 👍🏻 ), just to make sure it's not interfering with anything. Next step would be to ensure a working intel opengl setup without docker. If that's all working, than it's purely a docker permission/setup issue, which I unfortunately don't know how to help with. All I can think of is setting GST_GL_WINDOW to egl-device. 😞

[rosensvv]

Strange, I tried removing the nvidia, I tried adding a devices section in my docker-compose to add /dev/dri, yet still the same 0:00:00.774062774 1 0x6131c00 WARN gleglgbm gstgl_gbm_utils.c:489:gst_gl_gbm_find_and_open_drm_node: Cannot open device node "/dev/dri/card0": Permission denied (13) Setting GST_GL_WINDOW to egl-device seems to not have that error, simply says new websocket connected in the proxy logs. Am I missing something?

[Zubnix]

Did you ever got this working? (I'm developing on an nvidia+intel laptop myself without issues btw)

[rosensvv]

I just tried it after updating and I get this error on the compositor-proxy, I think unrelated to the previous issue:

Reading configuration from: /app/config.yaml
/app/config.js:22
    throw new Error(`Error validating configuration: ${JSON.stringify(validate.errors)}`);
    ^

Error: Error validating configuration: [{"instancePath":"/server","schemaPath":"#/properties/server/additionalProperties","keyword":"additionalProperties","params":{"additionalProperty":"allowOrigin"},"message":"must NOT have additional properties"}]
    at Object.<anonymous> (/app/config.js:22:11)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/app/index.js:9:18)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)

[Zubnix]

Normally when you run yarn generate in compositor-proxy, a json validation schema is generated base on https://github.com/udevbe/greenfield/blob/master/compositor-proxy/configschema.yaml

This json schema should be located at ./compositor-proxy/src/configschema.json. I suspect you still have an old version as the allowOrigin option was only recently added. You can try deleting this file and running yarn generate again.

[rosensvv]

I have no such file on this installation... My other installation runs fine with the same config....

[Zubnix]

Your configschema.json should look something like this:

{
  "description": "The main config file",
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "server": {
      "description": "General server settings",
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "bindIP": {
          "type": "string",
          "description": "The ip address to bind to"
        },
        "bindPort": {
          "type": "integer",
          "minimum": 1,
          "maximum": 65535,
          "description": "The port to bind to"
        },
        "allowOrigin": {
          "type": "string",
          "description": "The allowed origins during CORS checks."
        }
      },
      "required": [
        "bindIP",
        "bindPort",
        "allowOrigin"
      ]
    },
    "public": {
      "description": "Settings for the public endpoint",
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "baseURL": {
          "type": "string",
          "format": "uri",
          "description": "The base url to use when connecting to this endpoint. This is required to  inform other endpoints when doing direct endpoint to endpoint transfers."
        }
      },
      "required": [
        "baseURL"
      ]
    },
    "encoder": {
      "description": "Encoder settings",
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "h264Encoder": {
          "description": "The gstreamer h264 encoder to use. For now only 'x264' or 'nvh264' is supported. 'x264'\nis a pure software encoder. While 'nvh264' is a hw accelerated encoder for Nvidia based GPUs.\nsee https://gstreamer.freedesktop.org/documentation/x264/index.html\nsee https://gstreamer.freedesktop.org/documentation/nvenc/nvh264enc.html\n",
          "type": "string",
          "enum": [
            "x264",
            "nvh264",
            "vaapih264"
          ]
        },
        "maxPngBufferSize": {
          "description": "Maximum number of pixels an image can have before we switch to h264 encoding.\nIf lower, png encoding is used.\nPng encoding has perfect image quality but has a far larger image size than\nh264. Default is (256*256)-1 = 0xFFFF\n",
          "type": "integer",
          "minimum": 1
        }
      },
      "required": [
        "h264Encoder",
        "maxPngBufferSize"
      ]
    },
    "logging": {
      "description": "Logging options",
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "level": {
          "type": "string",
          "description": "Log Level",
          "enum": [
            "fatal",
            "error",
            "warn",
            "info",
            "debug",
            "trace"
          ]
        }
      },
      "required": [
        "level"
      ]
    }
  },
  "required": [
    "server",
    "public",
    "encoder",
    "logging"
  ]
}

[Zubnix]

I have no such file on this installation... My other installation runs fine with the same config....

You mean no configschema.json? It should be there as it's used at runtime to validate your config.

[rosensvv]

Okay, my docker image needed rebuilding. Now everything seems to work on this setup, no /dev/dri problems. Thank you yet again!