oele-isis-vanderbilt / livekit-gstreamer

Uses gstreamer to stream devices from Rust to Livekit rooms
Apache License 2.0
1 stars 0 forks source link

Missing devices #17

Open edavalosanaya opened 6 days ago

edavalosanaya commented 6 days ago

When I run the get_devices example, I get the following output:

$ cargo run --example get_devices
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s
     Running `target/debug/examples/get_devices`
============== Media Device Info ("Built-in Audio Analog Stereo"|"Audio/Source") ==============
Device path: front:0
Device name: Built-in Audio Analog Stereo
Device class: Audio/Source
Capabilities:
  Audio(AudioCapability { channels: 2, framerates: (44100, 192000), codec: "audio/x-raw" })
============== End Media Device Info ==============
============== Media Device Info ("Camera Mono"|"Audio/Source") ==============
Device path: hw:2
Device name: Camera Mono
Device class: Audio/Source
Capabilities:
  Audio(AudioCapability { channels: 1, framerates: (0, 0), codec: "audio/x-raw" })
============== End Media Device Info ==============
Video Devices:

------------------------------------------------------

Audio Devices:
============== Media Device Info ("Built-in Audio Analog Stereo"|"Audio/Source") ==============
Device path: front:0
Device name: Built-in Audio Analog Stereo
Device class: Audio/Source
Capabilities:
  Audio(AudioCapability { channels: 2, framerates: (44100, 192000), codec: "audio/x-raw" })
============== End Media Device Info ==============
============== Media Device Info ("Camera Mono"|"Audio/Source") ==============
Device path: hw:2
Device name: Camera Mono
Device class: Audio/Source
Capabilities:
  Audio(AudioCapability { channels: 1, framerates: (0, 0), codec: "audio/x-raw" })
============== End Media Device Info ==============

However, when I run the following command:

$ gst-device-monitor-1.0
Probing devices...

Device found:

        name  : Built-in Audio Analog Stereo
        class : Audio/Source
        caps  : audio/x-raw, layout=interleaved, format={ (string)S32LE, (string)S16LE }, rate=[ 44100, 192000 ], channels=2
        properties:
                alsa.card = 0
                alsa.card_name = HDA Intel PCH
                alsa.class = generic
                alsa.components = HDA:10ec0899,10280a63,00100003
                alsa.device = 0
                alsa.driver_name = snd_hda_intel
                alsa.id = PCH
                alsa.long_card_name = HDA Intel PCH at 0x4012110000 irq 162
                alsa.mixer_name = Realtek ALC3861
                alsa.name = ALC3861 Analog
                alsa.resolution_bits = 16
                alsa.subclass = generic-mix
                alsa.subdevice = 0
                alsa.subdevice_name = subdevice #0
                api.alsa.card.longname = HDA Intel PCH at 0x4012110000 irq 162
                api.alsa.card.name = HDA Intel PCH
                api.alsa.path = front:0
                api.alsa.pcm.card = 0
                api.alsa.pcm.stream = capture
                audio.channels = 2
                audio.position = FL,FR
                card.profile.device = 0
                device.api = alsa
                device.class = sound
                device.id = 48
                device.profile.description = Analog Stereo
                device.profile.name = analog-stereo
                device.routes = 3
                factory.name = api.alsa.pcm.source
                media.class = Audio/Source
                node.description = Built-in Audio Analog Stereo
                node.name = alsa_input.pci-0000_00_1f.3.analog-stereo
                node.nick = ALC3861 Analog
                node.pause-on-idle = false
                object.path = alsa:pcm:0:front:0:capture
                priority.driver = 2009
                priority.session = 2009
                factory.id = 18
                clock.quantum-limit = 8192
                client.id = 35
                node.driver = true
                factory.mode = split
                audio.adapt.follower = 
                library.name = audioconvert/libspa-audioconvert
                object.id = 42
                object.serial = 52
        gst-launch-1.0 pipewiresrc target-object=52 ! ...

Device found:

        name  : Camera Mono
        class : Audio/Source
        caps  : audio/x-raw, layout=interleaved, format=S16LE, rate=16000, channels=1
        properties:
                alsa.card = 2
                alsa.card_name = HD Web Camera
                alsa.class = generic
                alsa.components = USB05a3:9331
                alsa.device = 0
                alsa.driver_name = snd_usb_audio
                alsa.id = Camera
                alsa.long_card_name = HD Web Camera HD Web Camera at usb-0000:00:14.0-4.4, high speed
                alsa.mixer_name = USB Mixer
                alsa.name = USB Audio
                alsa.resolution_bits = 16
                alsa.subclass = generic-mix
                alsa.subdevice = 0
                alsa.subdevice_name = subdevice #0
                api.alsa.card.longname = HD Web Camera HD Web Camera at usb-0000:00:14.0-4.4, high speed
                api.alsa.card.name = HD Web Camera
                api.alsa.path = hw:2
                api.alsa.pcm.card = 2
                api.alsa.pcm.stream = capture
                audio.channels = 1
                audio.position = MONO
                card.profile.device = 1
                device.api = alsa
                device.class = sound
                device.id = 46
                device.profile.description = Mono
                device.profile.name = mono-fallback
                device.routes = 1
                factory.name = api.alsa.pcm.source
                media.class = Audio/Source
                node.description = Camera Mono
                node.name = alsa_input.usb-HD_Web_Camera_HD_Web_Camera_Ucamera001-02.mono-fallback
                node.nick = HD Web Camera
                node.pause-on-idle = false
                object.path = alsa:pcm:2:hw:2:capture
                priority.driver = 2000
                priority.session = 2000
                factory.id = 18
                clock.quantum-limit = 8192
                client.id = 35
                node.driver = true
                factory.mode = split
                audio.adapt.follower = 
                library.name = audioconvert/libspa-audioconvert
                object.id = 49
                object.serial = 49
        gst-launch-1.0 pipewiresrc target-object=49 ! ...

Device found:

        name  : Built-in Audio Analog Stereo
        class : Audio/Sink
        caps  : audio/x-raw, layout=interleaved, format={ (string)S32LE, (string)S16LE }, rate=[ 44100, 192000 ], channels=2
        properties:
                alsa.card = 0
                alsa.card_name = HDA Intel PCH
                alsa.class = generic
                alsa.components = HDA:10ec0899,10280a63,00100003
                alsa.device = 0
                alsa.driver_name = snd_hda_intel
                alsa.id = PCH
                alsa.long_card_name = HDA Intel PCH at 0x4012110000 irq 162
                alsa.mixer_name = Realtek ALC3861
                alsa.name = ALC3861 Analog
                alsa.resolution_bits = 16
                alsa.subclass = generic-mix
                alsa.subdevice = 0
                alsa.subdevice_name = subdevice #0
                api.alsa.card.longname = HDA Intel PCH at 0x4012110000 irq 162
                api.alsa.card.name = HDA Intel PCH
                api.alsa.path = front:0
                api.alsa.pcm.card = 0
                api.alsa.pcm.stream = playback
                audio.channels = 2
                audio.position = FL,FR
                card.profile.device = 5
                device.api = alsa
                device.class = sound
                device.id = 48
                device.profile.description = Analog Stereo
                device.profile.name = analog-stereo
                device.routes = 2
                factory.name = api.alsa.pcm.sink
                media.class = Audio/Sink
                node.description = Built-in Audio Analog Stereo
                node.name = alsa_output.pci-0000_00_1f.3.analog-stereo
                node.nick = ALC3861 Analog
                node.pause-on-idle = false
                object.path = alsa:pcm:0:front:0:playback
                priority.driver = 1009
                priority.session = 1009
                factory.id = 18
                clock.quantum-limit = 8192
                client.id = 35
                node.driver = true
                factory.mode = merge
                audio.adapt.follower = 
                library.name = audioconvert/libspa-audioconvert
                object.id = 33
                object.serial = 51
                node.max-latency = 16384/48000
                api.alsa.period-size = 1024
                api.alsa.period-num = 32
                api.alsa.headroom = 0
        gst-launch-1.0 ... ! pipewiresink target-object=51

Device found:

        name  : GA102 High Definition Audio Controller Digital Stereo (HDMI)
        class : Audio/Sink
        caps  : audio/x-raw, layout=interleaved, format={ (string)S32LE, (string)S16LE }, rate=[ 32000, 48000 ], channels=2
        properties:
                alsa.card = 1
                alsa.card_name = HDA NVidia
                alsa.class = generic
                alsa.components = HDA:10de009a,10283891,00100100
                alsa.device = 3
                alsa.driver_name = snd_hda_intel
                alsa.id = NVidia
                alsa.long_card_name = HDA NVidia at 0xa1080000 irq 17
                alsa.mixer_name = Nvidia GPU 9a HDMI/DP
                alsa.name = DELL S3422DW
                alsa.resolution_bits = 16
                alsa.subclass = generic-mix
                alsa.subdevice = 0
                alsa.subdevice_name = subdevice #0
                api.alsa.card.longname = HDA NVidia at 0xa1080000 irq 17
                api.alsa.card.name = HDA NVidia
                api.alsa.path = hdmi:1
                api.alsa.pcm.card = 1
                api.alsa.pcm.stream = playback
                audio.channels = 2
                audio.position = FL,FR
                card.profile.device = 4
                device.api = alsa
                device.class = sound
                device.id = 47
                device.profile.description = Digital Stereo (HDMI)
                device.profile.name = hdmi-stereo
                device.routes = 1
                factory.name = api.alsa.pcm.sink
                media.class = Audio/Sink
                node.description = GA102 High Definition Audio Controller Digital Stereo (HDMI)
                node.name = alsa_output.pci-0000_02_00.1.hdmi-stereo
                node.nick = DELL S3422DW
                node.pause-on-idle = false
                object.path = alsa:pcm:1:hdmi:1:playback
                priority.driver = 696
                priority.session = 696
                factory.id = 18
                clock.quantum-limit = 8192
                client.id = 35
                node.driver = true
                factory.mode = merge
                audio.adapt.follower = 
                library.name = audioconvert/libspa-audioconvert
                object.id = 50
                object.serial = 74
        gst-launch-1.0 ... ! pipewiresink target-object=74

Device found:

        name  : unknown
        class : Stream/Output/Audio
        caps  : audio/x-raw, layout=interleaved, format=F32LE, rate=48000, channels=2
        properties:
                client.api = pipewire-pulse
                pulse.server.type = unix
                application.name = Chromium
                application.process.id = 8319
                application.process.user = nicole
                application.process.host = pumpkinR12
                application.process.binary = vivaldi-bin
                application.language = en_US.UTF-8
                window.x11.display = :0
                application.process.machine-id = 27429a63fd3743b8a67efd824db3808d
                application.icon-name = chromium-browser
                media.name = Playback
                node.rate = 1/48000
                node.latency = 1024/48000
                stream.is-live = true
                node.name = Chromium
                node.want-driver = true
                node.autoconnect = true
                media.class = Stream/Output/Audio
                adapt.follower.spa-node = 
                object.register = false
                factory.id = 6
                clock.quantum-limit = 8192
                factory.mode = split
                audio.adapt.follower = 
                library.name = audioconvert/libspa-audioconvert
                client.id = 64
                object.id = 65
                object.serial = 85
                pulse.attr.maxlength = 4194304
                pulse.attr.tlength = 16384
                pulse.attr.prebuf = 12296
                pulse.attr.minreq = 4096

Device found:

        name  : HD Web Camera: HD Web Camera
        class : Video/Source
        caps  : video/x-raw, format=YUY2, width=1920, height=1080, pixel-aspect-ratio=1/1, framerate=5/1
                video/x-raw, format=YUY2, width=1280, height=960, pixel-aspect-ratio=1/1, framerate=5/1
                video/x-raw, format=YUY2, width=1280, height=720, pixel-aspect-ratio=1/1, framerate=5/1
                video/x-raw, format=YUY2, width=800, height=600, pixel-aspect-ratio=1/1, framerate=10/1
                video/x-raw, format=YUY2, width=640, height=480, pixel-aspect-ratio=1/1, framerate=30/1
                video/x-raw, format=YUY2, width=640, height=320, pixel-aspect-ratio=1/1, framerate=30/1
                image/jpeg, parsed=true, width=1920, height=1080, pixel-aspect-ratio=1/1, framerate=30/1
                image/jpeg, parsed=true, width=1280, height=960, pixel-aspect-ratio=1/1, framerate=30/1
                image/jpeg, parsed=true, width=1280, height=720, pixel-aspect-ratio=1/1, framerate=30/1
                image/jpeg, parsed=true, width=800, height=600, pixel-aspect-ratio=1/1, framerate=30/1
                image/jpeg, parsed=true, width=640, height=480, pixel-aspect-ratio=1/1, framerate=30/1
                image/jpeg, parsed=true, width=640, height=320, pixel-aspect-ratio=1/1, framerate=30/1
                video/x-h264, stream-format=byte-stream, alignment=au, width=1920, height=1080, pixel-aspect-ratio=1/1, framerate=30/1
                video/x-h264, stream-format=byte-stream, alignment=au, width=1280, height=960, pixel-aspect-ratio=1/1, framerate=30/1
                video/x-h264, stream-format=byte-stream, alignment=au, width=1280, height=720, pixel-aspect-ratio=1/1, framerate=30/1
                video/x-h264, stream-format=byte-stream, alignment=au, width=800, height=600, pixel-aspect-ratio=1/1, framerate=30/1
                video/x-h264, stream-format=byte-stream, alignment=au, width=640, height=480, pixel-aspect-ratio=1/1, framerate=30/1
                video/x-h264, stream-format=byte-stream, alignment=au, width=640, height=320, pixel-aspect-ratio=1/1, framerate=30/1
        properties:
                udev-probed = true
                device.bus_path = pci-0000:00:14.0-usb-0:4.4:1.0
                sysfs.path = /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.4/1-4.4:1.0/video4linux/video0
                device.bus = usb
                device.subsystem = video4linux
                device.vendor.id = 05a3
                device.vendor.name = HD\x20Web\x20Camera
                device.product.id = 9331
                device.product.name = HD Web Camera: HD Web Camera
                device.serial = HD_Web_Camera_HD_Web_Camera_Ucamera001
                device.capabilities = :capture:
                device.api = v4l2
                device.path = /dev/video0
                v4l2.device.driver = uvcvideo
                v4l2.device.card = HD Web Camera: HD Web Camera
                v4l2.device.bus_info = usb-0000:00:14.0-4.4
                v4l2.device.version = 394509 (0x0006050d)
                v4l2.device.capabilities = 2225078273 (0x84a00001)
                v4l2.device.device_caps = 69206017 (0x04200001)
        gst-launch-1.0 v4l2src ! ...

Device found:

        name  : llvmpipe (LLVM 17.0.6, 256 bits)
        class : Video/Sink
        caps  : video/x-raw(memory:VulkanImage)
        properties:
                vulkan.name = llvmpipe (LLVM 17.0.6, 256 bits)
                vulkan.type = CPU
                vulkan.api.version = 1.3.274
                vulkan.api.version.major = 1 (0x00000001)
                vulkan.api.version.minor = 3 (0x00000003)
                vulkan.api.version.patch = 274 (0x00000112)
                vulkan.driver.version = 0.0.1
                vulkan.driver.version.major = 0 (0x00000000)
                vulkan.driver.version.minor = 0 (0x00000000)
                vulkan.driver.version.patch = 1 (0x00000001)
                vulkan.vendor.id = 65541 (0x00010005)
                vulkan.device.id = 0 (0x00000000)
                vulkan.memory.n_heaps = 1 (0x00000001)
                vulkan.memory.heaps.0.size = 33306476544
                vulkan.memory.heaps.0.flags = 1 (0x00000001)
                vulkan.memory.heaps.0.flags.str = device-local
                vulkan.memory.n_types = 1 (0x00000001)
                vulkan.memory.types.0.heap = 0 (0x00000000)
                vulkan.memory.types.0.flags = 15 (0x0000000f)
                vulkan.memory.types.0.flags.str = device-local|host-visible|host-coherent|host-cached
                vulkan.n_queue_families = 1 (0x00000001)
                vulkan.queue_family.0.n_queues = 1 (0x00000001)
                vulkan.queue_family.0.flags = 7 (0x00000007)
                vulkan.queue_family.0.flags.str = graphics|compute|transfer
                vulkan.queue_family.0.timestamp_resolution = 64 (0x00000040)
                vulkan.queue_family.0.min_image_transfer_granuality.width = 1 (0x00000001)
                vulkan.queue_family.0.min_image_transfer_granuality.height = 1 (0x00000001)
                vulkan.queue_family.0.min_image_transfer_granuality.depth = 1 (0x00000001)
                is-default = false
        gst-launch-1.0 ... ! vulkansink

Device found:

        name  : NVIDIA GeForce RTX 3080
        class : Video/Sink
        caps  : video/x-raw(memory:VulkanImage)
        properties:
                vulkan.name = NVIDIA GeForce RTX 3080
                vulkan.type = discrete
                vulkan.api.version = 1.3.242
                vulkan.api.version.major = 1 (0x00000001)
                vulkan.api.version.minor = 3 (0x00000003)
                vulkan.api.version.patch = 242 (0x000000f2)
                vulkan.driver.version = 535.732.64
                vulkan.driver.version.major = 535 (0x00000217)
                vulkan.driver.version.minor = 732 (0x000002dc)
                vulkan.driver.version.patch = 64 (0x00000040)
                vulkan.vendor.id = 4318 (0x000010de)
                vulkan.device.id = 8710 (0x00002206)
                vulkan.memory.n_heaps = 3 (0x00000003)
                vulkan.memory.heaps.0.size = 10737418240
                vulkan.memory.heaps.0.flags = 1 (0x00000001)
                vulkan.memory.heaps.0.flags.str = device-local
                vulkan.memory.heaps.1.size = 24979857408
                vulkan.memory.heaps.1.flags = 0 (0x00000000)
                vulkan.memory.heaps.1.flags.str = 
                vulkan.memory.heaps.2.size = 257949696
                vulkan.memory.heaps.2.flags = 1 (0x00000001)
                vulkan.memory.heaps.2.flags.str = device-local
                vulkan.memory.n_types = 5 (0x00000005)
                vulkan.memory.types.0.heap = 1 (0x00000001)
                vulkan.memory.types.0.flags = 0 (0x00000000)
                vulkan.memory.types.0.flags.str = 
                vulkan.memory.types.1.heap = 0 (0x00000000)
                vulkan.memory.types.1.flags = 1 (0x00000001)
                vulkan.memory.types.1.flags.str = device-local
                vulkan.memory.types.2.heap = 1 (0x00000001)
                vulkan.memory.types.2.flags = 6 (0x00000006)
                vulkan.memory.types.2.flags.str = host-visible|host-coherent
                vulkan.memory.types.3.heap = 1 (0x00000001)
                vulkan.memory.types.3.flags = 14 (0x0000000e)
                vulkan.memory.types.3.flags.str = host-visible|host-coherent|host-cached
                vulkan.memory.types.4.heap = 2 (0x00000002)
                vulkan.memory.types.4.flags = 7 (0x00000007)
                vulkan.memory.types.4.flags.str = device-local|host-visible|host-coherent
                vulkan.n_queue_families = 5 (0x00000005)
                vulkan.queue_family.0.n_queues = 16 (0x00000010)
                vulkan.queue_family.0.flags = 15 (0x0000000f)
                vulkan.queue_family.0.flags.str = graphics|compute|transfer|sparse-binding
                vulkan.queue_family.0.timestamp_resolution = 64 (0x00000040)
                vulkan.queue_family.0.min_image_transfer_granuality.width = 1 (0x00000001)
                vulkan.queue_family.0.min_image_transfer_granuality.height = 1 (0x00000001)
                vulkan.queue_family.0.min_image_transfer_granuality.depth = 1 (0x00000001)
                vulkan.queue_family.1.n_queues = 2 (0x00000002)
                vulkan.queue_family.1.flags = 12 (0x0000000c)
                vulkan.queue_family.1.flags.str = transfer|sparse-binding
                vulkan.queue_family.1.timestamp_resolution = 64 (0x00000040)
                vulkan.queue_family.1.min_image_transfer_granuality.width = 1 (0x00000001)
                vulkan.queue_family.1.min_image_transfer_granuality.height = 1 (0x00000001)
                vulkan.queue_family.1.min_image_transfer_granuality.depth = 1 (0x00000001)
                vulkan.queue_family.2.n_queues = 8 (0x00000008)
                vulkan.queue_family.2.flags = 14 (0x0000000e)
                vulkan.queue_family.2.flags.str = compute|transfer|sparse-binding
                vulkan.queue_family.2.timestamp_resolution = 64 (0x00000040)
                vulkan.queue_family.2.min_image_transfer_granuality.width = 1 (0x00000001)
                vulkan.queue_family.2.min_image_transfer_granuality.height = 1 (0x00000001)
                vulkan.queue_family.2.min_image_transfer_granuality.depth = 1 (0x00000001)
                vulkan.queue_family.3.n_queues = 1 (0x00000001)
                vulkan.queue_family.3.flags = 44 (0x0000002c)
                vulkan.queue_family.3.flags.str = transfer|sparse-binding
                vulkan.queue_family.3.timestamp_resolution = 32 (0x00000020)
                vulkan.queue_family.3.min_image_transfer_granuality.width = 1 (0x00000001)
                vulkan.queue_family.3.min_image_transfer_granuality.height = 1 (0x00000001)
                vulkan.queue_family.3.min_image_transfer_granuality.depth = 1 (0x00000001)
                vulkan.queue_family.4.n_queues = 1 (0x00000001)
                vulkan.queue_family.4.flags = 268 (0x0000010c)
                vulkan.queue_family.4.flags.str = transfer|sparse-binding
                vulkan.queue_family.4.timestamp_resolution = 64 (0x00000040)
                vulkan.queue_family.4.min_image_transfer_granuality.width = 1 (0x00000001)
                vulkan.queue_family.4.min_image_transfer_granuality.height = 1 (0x00000001)
                vulkan.queue_family.4.min_image_transfer_granuality.depth = 1 (0x00000001)
                is-default = true
        gst-launch-1.0 ... ! vulkansink

My HD Web camera is recognized. However, livekit-gstreamer does not recognize this device.

umesh-timalsina commented 4 days ago

Hi Eduardo. Thanks for the issue. I think it stems from the following line in the code: https://github.com/oele-isis-vanderbilt/livekit-gstreamer/blob/44b5bef0ce1ada22a64a81169b7ebcbd6eb33316/src/media_device.rs#L35

umesh-timalsina commented 4 days ago

https://github.com/oele-isis-vanderbilt/livekit-gstreamer/pull/19. @edavalosanaya Please let me know if this PR fixes it.