search

matijagaspar / ws-avc-player

Video player for browser that is able to play realtime low latency h264 video stream from websocket.
BSD 3-Clause "New" or "Revised" License
156 stars 48 forks source link

h264 Mobile terminal #7

Open linggongzhi opened 4 years ago

linggongzhi commented 4 years ago

服务端视频流（h264）宽高变化，比如缩放或者倒置之后，客户端会重新绘制，导致移动端看到这个过程（低端手机）很卡，我想更流畅 一点。    最后总结一下：1、您的这个项目音频支持进展或者效果如何？2、移动端的效果您有测试过吗？ The H264 of the server is changed. After scaling or inversion, the client will redraw, which will lead the mobile terminal to see this process (low end cell phone) is very stuck, I want to be more fluent.     Finally: 1. How is the audio support progress or effect of your project? 2. Have you tested the effect of mobile terminal? 

matijagaspar commented 4 years ago

Not sure I follow. So the stream on the server changes height and width and this event has poor performance on client? I am not sure much can be done, specially on low-end mobile phones since h264 decoding is done with a wasm decoder (Brodway ). You could try using the webworker: var wsavc = new WSAvcPlayer.default({useWorker:true}); but make sure that lib/Decoder.js is on the http root (ie: http://example.com/Decoder.js).

It would help me if you could provide a sample stream (raw h264) in a file, so I can see and test it myself

linggongzhi commented 4 years ago

普通的h264即可，5G来临，网速好的情况下，我们通过websocket实时传输视频流到手机上显示。请问您这个音频部分有了吗？ 忘了说一个事，就是不光传输到手机上，还有浏览器上，344 720的宽高。 Ordinary h264, 5 g, speed good cases, we through the websocket real-time streaming mobile phone display. Do you have the audio portion? Forget to say one thing, it is not only transfer to mobile phones, and browser, 344, 720 wide.

matijagaspar commented 4 years ago

Do you have the audio portion?

Work in progress on: https://github.com/matijagaspar/ws-avc-player/tree/audio-support

Hopefully the release will be in around a week.

Regarding the original issue, best would be if you could capture a example stream and share it, so I can understand the problem better

hexstia commented 4 years ago

Is the audio format you send PCM or AAC，Whether it can be compressed to AAC format

matijagaspar commented 4 years ago

On the server the stream provided needs to be 16 LE PCM, that is than encoded into Opus. The decoding on client is done in a web worker using libopus compiled to wasm.

Check the draft PR #10

I opted for opus instead of ACC as it provides a more optimal compression for low-latency VOIP use: opus br