avplayer解码错误 - Githubissues

imzlh commented 3 months ago

有些视频音频格式是Opus（用ffmpeg默认参数压制的），视频会失效，显示：

[opus @ 0x517500] Could not update timestamps for skipped samples. avplayer.js:1 [src/avplayer/AVPlayer.ts][line 941][src/avplayer/AVPlayer.ts][line 941] [info] Input #0, mp4, from '001.mp4:' Duration: 00:23:52.106, start: 00:00:00.000, bitrate: 1301 kbps/s Stream #0:0 Video: hevc (Main10), yuv420p10le(tv, bt709), 1920x1080 [SAR: 1:1 DAR 16:9], 1254 kbps/s, 24.00 fps, 24.00 tbr, 16k tbn (default) Metadata: creationTime: 0 modificationTime: 0 language: 21956 languageString: und handlerName: VideoHandler vendorId: encoder: naluLengthSizeMinusOne: 3 Stream #0:1 Audio: opus, 48000 Hz, stereo, floatp, 46 kbps/s (default) Metadata: creationTime: 0 modificationTime: 0 language: 10766 languageString: jpn handlerName: SoundHandler vendorId: Stream #0:2 Subtitle: mov_text Metadata: creationTime: 0 modificationTime: 0 language: 0 [src/avplayer/AVPlayer.ts][line 1045] [info] call play, options: {"audio":true,"video":true,"subtitle":true}, status: 2 taskId: 133accc2-a050-4912-83a6-e11886711b96 avplayer.js:1 [src/avplayer/AVPlayer.ts][line 335] [warn] can not support mse for codec: video/mp4; codecs="hev1.2.20.H120.90", taskId: 133accc2-a050-4912-83a6-e11886711b96 98adfe8a-232e-45a7-a6b1-62b87e683d5e:173 [src/avpipeline/VideoDecodePipeline.ts][line 523] [error] cannot open hardware decoder, Error: not support 98adfe8a-232e-45a7-a6b1-62b87e683d5e:173 [src/avpipeline/VideoDecodePipeline.ts][line 419] [warn] webcodecs decoder open failed, Error: not support, try to fallback to wasm software decoder 17163d5c-8ba1-46a4-bb4d-24103bb5caea:173 [src/avpipeline/DemuxPipeline.ts][line 515] [debug] connect stream 0, taskId: 133accc2-a050-4912-83a6-e11886711b96 17163d5c-8ba1-46a4-bb4d-24103bb5caea:173 [src/avpipeline/DemuxPipeline.ts][line 515] [debug] connect stream 1, taskId: 133accc2-a050-4912-83a6-e11886711b96 avplayer.js:1 [src/avplayer/AVPlayer.ts][line 2007] [info] player call setVolume, set 1, used 1, taskId: 133accc2-a050-4912-83a6-e11886711b96 17163d5c-8ba1-46a4-bb4d-24103bb5caea:173 [src/avpipeline/DemuxPipeline.ts][line 515] [debug] connect stream 2, taskId: 133accc2-a050-4912-83a6-e11886711b96 17163d5c-8ba1-46a4-bb4d-24103bb5caea:173 [src/avpipeline/DemuxPipeline.ts][line 751] [debug] start demux loop, taskId: 133accc2-a050-4912-83a6-e11886711b96 98adfe8a-232e-45a7-a6b1-62b87e683d5e:173 [src/avpipeline/VideoDecodePipeline.ts][line 327] [error] video decode error, taskId: 133accc2-a050-4912-83a6-e11886711b96, ret: -1 d93c23b4-2c0d-46f2-9855-625f00c8e1fe:173 [opus @ 0x4d6210] Could not update timestamps for skipped samples. 9f071a0c-be8e-4585-b56b-88d408220062:173 [src/avpipeline/VideoRenderPipeline.ts][line 363] [info] video render ended, taskId: 133accc2-a050-4912-83a6-e11886711b96 3b7591af-add6-4061-a6eb-3bb5d91822ae:173 [src/avpipeline/AudioRenderPipeline.ts][line 229] [debug] got first audio frame, pts: 0(0ms), taskId: 133accc2-a050-4912-83a6-e11886711b96 avplayer.js:1 [src/avplayer/AVPlayer.ts][line 2450] [info] first audio frame rendered, taskId: 133accc2-a050-4912-83a6-e11886711b96 avplayer.js:1 [src/avplayer/AVPlayer.ts][line 1537] [info] call pause, taskId: 133accc2-a050-4912-83a6-e11886711b96 9f071a0c-be8e-4585-b56b-88d408220062:173 [src/avpipeline/VideoRenderPipeline.ts][line 693] [error] task has not played

官网的demo播放器也是这样，请修复

imzlh commented 3 months ago

我得出结论，似乎是WASM解码器造成的，去找了一个AV1+OPUS的，使用WebCodec播放没问题以下测试失败

HEVC+OPUS
HEVC+FLAC

PS: 我的GPU不支持HEVC，Chrome自带AV1软解支持

imzlh commented 3 months ago

另外，似乎添加的ASS字幕无法渲染，F12看了看xhr也没有，不清楚为什么

修改轨道后，似乎会有杂音出现，很吓人

https://github.com/user-attachments/assets/86bc11f6-824e-4e3a-bbbf-de2a4b8b0a10

imzlh commented 3 months ago

chapter似乎有错误，start比视频还晚

少了很多，这个是ffmpeg输出结果

Duration: 00:24:13.12, start: 0.000000, bitrate: 7144 kb/s Chapters: Chapter #0:0: start 0.000000, end 197.989000 Metadata: title : Chapter 01 Chapter #0:1: start 197.989000, end 695.028000 Metadata: title : Chapter 02 Chapter #0:2: start 695.028000, end 1303.010000 Metadata: title : Chapter 03 Chapter #0:3: start 1303.010000, end 1392.975000 Metadata: title : Chapter 04 Chapter #0:4: start 1392.975000, end 1435.976000 Metadata: title : Chapter 05 Chapter #0:5: start 1435.976000, end 1453.120000 Metadata: title : Chapter 06

zhaohappy commented 3 months ago

重新拉一下代码，修复了，chapter 里面是有时间单位的，可以自己换算到毫秒

imzlh commented 3 months ago

我有一些疑问，既然时间返回的是BigInt，为什么seek()用的是Number呢

await this.doSeek(BigInt(Math.floor(t)), e),

imzlh commented 3 months ago

seek()的时候继续报错，Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'sampleIndexesPosMap')

 if (1 & t.ioReader.flags && !i.sampleIndexesPosMap.has(s.f[17](e + 56)))

Worker线程报错，HEVC+FLAC，播放2s左右停止了这一条信息不知有没有用 [src/avformat/formats/IMatroskaFormat.ts][line 864] [error] not support compression stream, algo: undefined

imzlh commented 3 months ago

似乎 blob/master/src/avplayer/eventType.ts#L58 firstAudioRendered这个事件没有实现，没有回调

zhaohappy commented 3 months ago

seek()的时候继续报错，Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'sampleIndexesPosMap')
 if (1 & t.ioReader.flags && !i.sampleIndexesPosMap.has(s.f[17](e + 56)))
Worker线程报错，HEVC+FLAC，播放2s左右停止了这一条信息不知有没有用 [src/avformat/formats/IMatroskaFormat.ts][line 864] [error] not support compression stream, algo: undefined

这个视频可以发上来看看吗

imzlh commented 3 months ago

seek()的时候继续报错，Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'sampleIndexesPosMap')
 if (1 & t.ioReader.flags && !i.sampleIndexesPosMap.has(s.f[17](e + 56)))
Worker线程报错，HEVC+FLAC，播放2s左右停止了这一条信息不知有没有用 [src/avformat/formats/IMatroskaFormat.ts][line 864] [error] not support compression stream, algo: undefined
这个视频可以发上来看看吗

https://cloud.imzlh.top:8443/%E5%A4%A7%E5%AE%A4%E5%AE%B6/%5BAirota%26VCB-Studio%5D%20Ohmuro-ke%20-%20dear%20sisters%20%5BMa10p_1080p%5D%5Bx265_flac%5D.mkv

zhaohappy commented 3 months ago

seek()的时候继续报错，Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'sampleIndexesPosMap')
 if (1 & t.ioReader.flags && !i.sampleIndexesPosMap.has(s.f[17](e + 56)))
Worker线程报错，HEVC+FLAC，播放2s左右停止了这一条信息不知有没有用 [src/avformat/formats/IMatroskaFormat.ts][line 864] [error] not support compression stream, algo: undefined
这个视频可以发上来看看吗
https://cloud.imzlh.top:8443/%E5%A4%A7%E5%AE%A4%E5%AE%B6/%5BAirota%26VCB-Studio%5D%20Ohmuro-ke%20-%20dear%20sisters%20%5BMa10p_1080p%5D%5Bx265_flac%5D.mkv

访问不了

zhaohappy commented 3 months ago

我有一些疑问，既然时间返回的是BigInt，为什么seek()用的是Number呢
await this.doSeek(BigInt(Math.floor(t)), e),

有道理，我改成 int64 类型

zhaohappy commented 3 months ago

似乎 blob/master/src/avplayer/eventType.ts#L58 firstAudioRendered这个事件没有实现，没有回调

抛错了，抛成了 firstVideoRendered

imzlh commented 3 months ago

seek()的时候继续报错，Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'sampleIndexesPosMap')
 if (1 & t.ioReader.flags && !i.sampleIndexesPosMap.has(s.f[17](e + 56)))
Worker线程报错，HEVC+FLAC，播放2s左右停止了这一条信息不知有没有用 [src/avformat/formats/IMatroskaFormat.ts][line 864] [error] not support compression stream, algo: undefined
这个视频可以发上来看看吗
https://cloud.imzlh.top:8443/%E5%A4%A7%E5%AE%A4%E5%AE%B6/%5BAirota%26VCB-Studio%5D%20Ohmuro-ke%20-%20dear%20sisters%20%5BMa10p_1080p%5D%5Bx265_flac%5D.mkv
访问不了

没有IPV6吗，那这个 http://198.18.0.159:7297/%E5%A4%A7%E5%AE%A4%E5%AE%B6/%5BAirota%26VCB-Studio%5D%20Ohmuro-ke%20-%20dear%20sisters%20%5BMa10p_1080p%5D%5Bx265_flac%5D.mkv

imzlh commented 3 months ago

avPlayer销毁后，IO任务没有停止导致管道不可写入，报错：

Uncaught (in promise) TypeError: Cannot read properties of null (reading 'postMessage') at l.reply (08645876-3d1a-4f4b-ad22-572808a5f508:173:82359) at 08645876-3d1a-4f4b-ad22-572808a5f508:156:1026

imzlh commented 3 months ago

[src/avplayer/AVPlayer.ts][line 1706] [error]demuxer seek failed, code: -2097152, taskId: b136fb55-7cda-4fea-a28e-31ad09f8726d

seek失败，报错。经测试，（一周前的）旧版本没有问题传入的确实是BigInt，不清楚为什么

zhaohappy commented 3 months ago

[src/avplayer/AVPlayer.ts][line 1706] [error]demuxer seek failed, code: -2097152, taskId: b136fb55-7cda-4fea-a28e-31ad09f8726d

seek失败，报错。经测试，（一周前的）旧版本没有问题传入的确实是BigInt，不清楚为什么

说一下复现步骤，这周没有更改 seek 相关的逻辑，我这也没复现

imzlh commented 3 months ago

（总时长：1440032n）然后就报错[src/avplayer/AVPlayer.ts][line 1706] [error]demuxer seek failed, code: -1, taskId: c585bc36-e6d9-4fc4-860b-45eecffa708d

zhaohappy commented 3 months ago

-1 是格式不支持操作的错误码，什么格式的视频

imzlh commented 3 months ago

-1 是格式不支持操作的错误码，什么格式的视频

mkv, hevc+aac 示例也是可以播放的，但是我看了下源码似乎seek还在使用Number

slider.currentTime = Number(pts)

但是我无论seek参数是Number还是BigInt都是一样，很奇怪

zhaohappy commented 3 months ago

demo 那个是因为进度条那个组件只能支持 number，所以转了一下。你上面的意思是使用 demo 正常，你那不正常吗，如果是这样看看文件是不是没更新全

imzlh commented 3 months ago

好像是越界了我记得之前avPlayer返回的都是纳秒，现在变回毫秒了没改代码逻辑建议加一个判断逻辑，直接报错Out of range会更好理解

imzlh commented 3 months ago

添加ASS字幕似乎还是不生效，demo里也是这样

zhaohappy commented 3 months ago

添加ASS字幕似乎还是不生效，demo里也是这样

我试了可以，你啥步骤

imzlh commented 3 months ago

添加ASS字幕似乎还是不生效，demo里也是这样

我试了可以，你啥步骤

https://github.com/user-attachments/assets/e644f2d2-fc00-4363-abb3-18584c58373a

zhaohappy commented 3 months ago

把你那字幕发上来看看，同样的操作我这可以

imzlh commented 3 months ago

视频：http://175.42.241.73:7297/download/%E5%89%83%E9%A1%BB%E3%80%82%E7%84%B6%E5%90%8E%E6%8D%A1%E5%88%B0%E5%A5%B3%E9%AB%98%E4%B8%AD%E7%94%9F/001.mkv 字幕：http://175.42.241.73:7297/download/%E5%89%83%E9%A1%BB%E3%80%82%E7%84%B6%E5%90%8E%E6%8D%A1%E5%88%B0%E5%A5%B3%E9%AB%98%E4%B8%AD%E7%94%9F/001.ass

imzlh commented 2 months ago

能否添加字母偏移的功能，有的时候字幕与视频不对应就很难受当解码速度跟不上时可否修改为丢帧，有时候看着看着视频和音频就错位了

zhaohappy commented 2 months ago

能否添加字母偏移的功能，有的时候字幕与视频不对应就很难受当解码速度跟不上时可否修改为丢帧，有时候看着看着视频和音频就错位了

字幕偏移目前就有 API(setSubTitleDelay)；压力在解码上，解码是不能丢帧解的，必须要一帧帧解出来，机器性能实在不行用 mse 播放看有没有缓解吧。

imzlh commented 2 months ago

https://github.com/zhaohappy/libmedia/blob/master/src/avplayer/AVPlayer.ts#L2628 这里有个问题，那就是canvas如果与外部容器等大小而不等于视频比例，会有白边

imzlh commented 2 months ago

playNextFrame也有问题，会导致视频音频不同步建议playNextFrame后play同步一下轨道

imzlh commented 2 months ago

seek后报错 [src/avpipeline/VideoDecodePipeline.ts][line 330] [error] video decode error, taskId: c3c55806-7aaf-4adc-9a97-20b353988679, ret: -1，然后只有音频没有视频

zhaohappy commented 2 months ago

有问题新开一个 issues 吧，这样好管理一点，别人也会更好看一点

zhaohappy commented 2 months ago

https://github.com/zhaohappy/libmedia/blob/master/src/avplayer/AVPlayer.ts#L2628 这里有个问题，那就是canvas如果与外部容器等大小而不等于视频比例，会有白边

你的 canvas 背景色是白色吗，没有设置背景颜色的话应该是透明的，可以设置成黑色

zhaohappy commented 2 months ago

playNextFrame也有问题，会导致视频音频不同步建议playNextFrame后play同步一下轨道

有同步，之前是音视频差距 1 秒已上才同步，现在调整为 400ms

zhaohappy commented 2 months ago

seek后报错 [src/avpipeline/VideoDecodePipeline.ts][line 330] [error] video decode error, taskId: c3c55806-7aaf-4adc-9a97-20b353988679, ret: -1，然后只有音频没有视频

改了一版再观察一下

imzlh commented 2 months ago

好的，我本来是觉得如果BUG一个一个报得开很多issue，那close了

zhaohappy / libmedia

avplayer解码错误 #8