Question: VP8 or ffmpeg libraries

[Terricide]

Has anyone tried to get the wasm ports of this libraries? .net doesn't really have any encoding or decoding libraries that aren't just wrappers around c++ libraries that I have been able to find. This might be a good way to get them ported.

I've tried ffmpeg-webm.wasm/ffmpeg-mp4.wasm/webm-wasm.wasm and all of them get

System.ArgumentOutOfRangeException: 'WebAssemblyValueType 7 not recognized. Parameter name: valueType'

Any ideas?

[RyanLamansky]

Has anyone tried to get the wasm ports of this libraries? .net doesn't really have any encoding or decoding libraries that aren't just wrappers around c++ libraries that I have been able to find. This might be a good way to get them ported.

You're the first I've heard of. In principal, this library should enable that scenario, provided the WASM files don't require unsupported functionality...

I've tried ffmpeg-webm.wasm/ffmpeg-mp4.wasm/webm-wasm.wasm and all of them get

System.ArgumentOutOfRangeException: 'WebAssemblyValueType 7 not recognized. Parameter name: valueType'

Any ideas?

WebAssembly for .NET supports the 4 value types currently defined in the official spec ( https://webassembly.github.io/spec/core/binary/types.html#value-types ). I know they're working on adding more for features like SIMD, and I'm guessing those have been implemented in the V8 JavaScript engine already but the spec hasn't been updated.

So... to make those libraries work, I'd have to find out what exactly value type 7 (and likely others) is and add support for the instructions that use them.

[Terricide]

These are the ones I'm trying. They are 5 year old libraries. Maybe I should try and find an updated port.

https://github.com/BrianJFeldman/ffmpeg.js.wasm

[RyanLamansky]

Oh... that's before WASM 1.0 was finalized. So that might not be compatible with anything.

[Terricide]

I tried this one from 1.5 years ago and got the same error

https://github.com/GoogleChromeLabs/webm-wasm/blob/master/dist/webm-wasm.wasm

[Terricide]

Well I think it got a little farther with this one.

https://github.com/sonysuqin/WasmVideoPlayer/blob/master/libffmpeg.wasm

System.InvalidOperationException HResult=0x80131509 Message=System.Action is not a GenericTypeDefinition. MakeGenericType may only be called on a type for which Type.IsGenericTypeDefinition is true. Source=mscorlib StackTrace: at System.RuntimeType.MakeGenericType(Type[] instantiation) at WebAssembly.Runtime.Compile.FromBinary(Reader reader, CompilerConfiguration configuration, Type instanceContainer, Type exportContainer) in C:\Users\Mark\Source\Repos\dotnet-webassembly\WebAssembly\Runtime\Compile.cs:line 1031 at WebAssembly.Runtime.Compile.FromBinary[TExports](Stream input, CompilerConfiguration configuration) in C:\Users\Mark\Source\Repos\dotnet-webassembly\WebAssembly\Runtime\Compile.cs:line 109 at WebAssembly.Runtime.Compile.FromBinary[TExports](String path, CompilerConfiguration configuration) in C:\Users\Mark\Source\Repos\dotnet-webassembly\WebAssembly\Runtime\Compile.cs:line 73 at WebAssembly.Runtime.Compile.FromBinary[TExports](String path) in C:\Users\Mark\Source\Repos\dotnet-webassembly\WebAssembly\Runtime\Compile.cs:line 47 at Program.Main() in C:\Users\Mark\Source\Repos\sipsorcery\examples\WebRTCExamples\WebRTCTestPatternServer\MainClass.cs:line 19

This exception was originally thrown at this call stack: [External Code] WebAssembly.Runtime.Compile.FromBinary(WebAssembly.Reader, WebAssembly.Runtime.CompilerConfiguration, System.Type, System.Type) in Compile.cs WebAssembly.Runtime.Compile.FromBinary(System.IO.Stream, WebAssembly.Runtime.CompilerConfiguration) in Compile.cs WebAssembly.Runtime.Compile.FromBinary(string, WebAssembly.Runtime.CompilerConfiguration) in Compile.cs WebAssembly.Runtime.Compile.FromBinary(string) in Compile.cs Program.Main() in MainClass.cs

[RyanLamansky]

This looks like an internal error, I'll investigate in the next day or two.

[RyanLamansky]

Updates:

• [x] The "System.Action is not a GenericTypeDefinition" error is fixed in source and will be included in the next Nuget package, but this isn't enough to get the ffmpeg library working...
• [ ] There's an internal error where it's not tracking imported memory correctly during compilation, causing a failure to emit .NET instructions that use memory. This is weird cause I thought this worked. I'm investigating.
• [ ] This WASM requires table imports, which are not yet supported and is the last critical piece of WASM 1.0 functionality not yet implemented. This is complicated and will not happen fast.

Even after all of the above, you'll still be required to correctly supply a total of 40 imports to get this library running, which would require you to reverse-engineer https://github.com/sonysuqin/WasmVideoPlayer/blob/master/libffmpeg.js .

[Terricide]

Awesome thanks for the update. I'm going to try and get those 40 imports working using wasmer and I will post them here if I can get them working and then I can try and help and get it working correctly in dotnet-webassembly as that would be much better.

[RyanLamansky]

The memory error is fixed in code and will be covered by the next Nuget release.

As these things usually go, fixing that problem revealed yet another. WASM has different rules than CIL about how functions return and how things like "if" blocks work. I had assumed that an optimized WASM wouldn't depend on this difference, but either I was wrong or this WASM isn't optimized. Either way, I have to fix it... might involve building an AST and more intelligence around converting this pattern 🙄

[Terricide]

That's awesome, I'll continue to try and find some other ports, pretty much any video codec running in webassembly, then running on .net would be a really cool as there are pretty much zero codecs I can find that aren't just c/c++ libraries that need to be ported to every platform.

[RyanLamansky]

I retested libffmpeg.wasm with the current version of WebAssembly for .NET and the compiler was able to process it. So, maybe it works now? It has a lot of imports that need to be satisfied.

Let me know if you're still having issues.