search

pion / mediadevices

Go implementation of the MediaDevices API.
https://pion.ly/
MIT License
528 stars 121 forks source link

Raspberrypi unable to get audio #411

Closed xiongfeigood closed 2 years ago

xiongfeigood commented 2 years ago

Your environment.

What did you do?

Raspberry PI 32-bit, libopus-dev installed, but still can't get sound。 Hello, I use webrTC example, if open audio, still cannot establish connection. If I comment Audio out, I can make a connection like this:

s, err := mediadevices.GetUserMedia(mediadevices.MediaStreamConstraints{
 Video: func(c *mediadevices.MediaTrackConstraints) {
  c.FrameFormat = prop.FrameFormatOneOf{frame.FormatI420, frame.FormatYUY2}
  c.Width = prop.Int(640)
  c.Height = prop.Int(480)
 },
  //Audio: func(c *mediadevices.MediaTrackConstraints) {
 / /},
 Codec: codecSelector,
})

But if I comment out the Audio item, I can't get the Audio. Is there something wrong with me?

What did you expect?

I can get sound in chromium. I hope I can get sound through opus，please help me

What happened?

=============== Constraints ==============

DeviceID: any
VideoConstraints:
  Width: 640 (ideal)
  Height: 480 (ideal)
  FrameRate: any
  FrameFormat: I420,YUY2 (one of values)
AudioConstraints:
  ChannelCount: any
  Latency: any
  SampleRate: any
  SampleSize: any
  IsBigEndian: any
  IsFloat: any
  IsInterleaved: any

================ Best Fit ================

DeviceID: fd0552e9-7f4a-4fa4-9a19-2d7422362b9f
Video:
  Width: 640
  Height: 480
  FrameRate: 0
  FrameFormat: YUY2
Audio:
  ChannelCount: 0
  Latency: 0s
  SampleRate: 0
  SampleSize: 0
  IsBigEndian: false
  IsFloat: false
  IsInterleaved: false
mediadevices/driver/microphone DEBUG: 16:20:35.133973 microphone.go:165: Querying properties
mediadevices/driver/microphone DEBUG: 16:20:35.134122 microphone.go:165: Querying properties
mediadevices DEBUG: 16:20:35.134492 mediadevices.go:156: 
============ Found Properties ============

DeviceID: 39c76093-c9fb-4b21-a5c1-0463af37468a
Video:
  Width: 0
  Height: 0
  FrameRate: 0
  FrameFormat: 
Audio:
  ChannelCount: 2
  Latency: 20ms
  SampleRate: 48000
  SampleSize: 2
  IsBigEndian: false
  IsFloat: false
  IsInterleaved: true

DeviceID: 23915681-2bf8-4028-9b38-48452c7c12b0
Video:
  Width: 0
  Height: 0
  FrameRate: 0
  FrameFormat: 
Audio:
  ChannelCount: 1
  Latency: 20ms
  SampleRate: 48000
  SampleSize: 2
  IsBigEndian: false
  IsFloat: false
  IsInterleaved: true

=============== Constraints ==============

DeviceID: any
VideoConstraints:
  Width: any
  Height: any
  FrameRate: any
  FrameFormat: any
AudioConstraints:
  ChannelCount: any
  Latency: any
  SampleRate: any
  SampleSize: any
  IsBigEndian: any
  IsFloat: any
  IsInterleaved: any

================ Best Fit ================

DeviceID: 23915681-2bf8-4028-9b38-48452c7c12b0
Video:
  Width: 0
  Height: 0
  FrameRate: 0
  FrameFormat: 
Audio:
  ChannelCount: 1
  Latency: 20ms
  SampleRate: 48000
  SampleSize: 2
  IsBigEndian: false
  IsFloat: false
  IsInterleaved: true
pc INFO: 2022/07/10 16:20:35 signaling state changed to have-local-offer
ice WARNING: 2022/07/10 16:20:35 failed to resolve stun host: stun.l.google.com:19302: lookup stun.l.google.com on 183.221.253.100:53: no such host
pc INFO: 2022/07/10 16:20:35 signaling state changed to stable
mediadevices DEBUG: 16:20:35.217695 track.go:167: trying to build video/VP8 rtp reader
BraveSLC commented 2 years ago

hello, i have a similar problem. i can't get the video and audio stream. but my version is:

Linux raspberrypi 5.10.92-v8+ #1514 SMP PREEMPT Mon Jan 17 17:39:38 GMT 2022 aarch64 GNU/Linux

problem:

  1. cant get audio stream by libopus-dev, just like @xiongfeigood
  2. cant get video stream by mmal. i found that aarch64 userland does not provide 64bit mmal libs and headers. the error: 
    # github.com/pion/mediadevices/pkg/codec/mmal
In file included from ../../go/pkg/mod/github.com/pion/mediadevices@v0.3.7/pkg/codec/mmal/mmal.go:8:
./bridge.h:1:10: fatal error: interface/mmal/mmal.h: No such file or directory
1 | #include <interface/mmal/mmal.h>

    how to solve this problem. thanks very much. bro.

xiongfeigood commented 2 years ago

I upgraded to a 64-bit system and can get video and audio normally. However, I find that setting up ICE connections often fails. If you get the answer from the server immediately, the connection must have failed. If ICE negotiations are printed multiple times, then the connection will actually be established. I don't know why.

EmrysMyrddin commented 2 years ago

Possible duplicate of #407 ?

yaxiongwu commented 2 years ago

I also have the two problems as above, when I use fuction " mediadevices.EnumerateDevices() " to test, it seems some wrong devices list, maybe get some wrong audio devices to the track?

yaxiongwu commented 2 years ago

In the example of mediadevices/examples/webrtc, I comment out the video,

s, err := mediadevices.GetUserMedia(mediadevices.MediaStreamConstraints{ //Video: func(c mediadevices.MediaTrackConstraints) { // c.FrameFormat = prop.FrameFormat(frame.FormatI420) // c.Width = prop.Int(640) // c.Height = prop.Int(480) //}, Audio: func(c mediadevices.MediaTrackConstraints) { c.ChannelCount=prop.Int(1) }, Codec: codecSelector, })

the logs:

============ Found Properties ============

DeviceID: af7699b8-1bbc-4617-9468-64a4b0d8ba66 Video: Width: 0 Height: 0 FrameRate: 0 FrameFormat: Audio: ChannelCount: 1 Latency: 20ms SampleRate: 48000 SampleSize: 2 IsBigEndian: false IsFloat: false IsInterleaved: true

DeviceID: 8473793e-8437-4c05-9b36-7593f42031ef Video: Width: 0 Height: 0 FrameRate: 0 FrameFormat: Audio: ChannelCount: 2 Latency: 20ms SampleRate: 48000 SampleSize: 2 IsBigEndian: false IsFloat: false IsInterleaved: true

DeviceID: 4b85ac5a-d0aa-458f-82bd-5d1508cfed72 Video: Width: 0 Height: 0 FrameRate: 0 FrameFormat: Audio: ChannelCount: 2 Latency: 20ms SampleRate: 48000 SampleSize: 2 IsBigEndian: false IsFloat: false IsInterleaved: true

=============== Constraints ==============

DeviceID: any VideoConstraints: Width: any Height: any FrameRate: any FrameFormat: any AudioConstraints: ChannelCount: 1 (ideal) Latency: any SampleRate: any SampleSize: any IsBigEndian: any IsFloat: any IsInterleaved: any

================ Best Fit ================

DeviceID: af7699b8-1bbc-4617-9468-64a4b0d8ba66 Video: Width: 0 Height: 0 FrameRate: 0 FrameFormat: Audio: ChannelCount: 1 Latency: 20ms SampleRate: 48000 SampleSize: 2 IsBigEndian: false IsFloat: false IsInterleaved: true pc INFO: 2022/08/01 12:54:01 signaling state changed to have-remote-offer signal: floating point exception

yaxiongwu commented 2 years ago

I upgraded to a 64-bit system and can get video and audio normally. However, I find that setting up ICE connections often fails. If you get the answer from the server immediately, the connection must have failed. If ICE negotiations are printed multiple times, then the connection will actually be established. I don't know why.

Please tell me which 64-bit system you selected, thanks

xiongfeigood commented 2 years ago

I upgraded to a 64-bit system and can get video and audio normally. However, I find that setting up ICE connections often fails. If you get the answer from the server immediately, the connection must have failed. If ICE negotiations are printed multiple times, then the connection will actually be established. I don't know why.

Please tell me which 64-bit system you selected, thanks

2022-04-04-raspios-bullseye-arm64.img