Question: How to use IBasicVideoEffect?

[HermanEldering]

I've tried implementing IBasicVideoEffect with CsWinRT but I can't get it to activate with new VideoEffectDefinition("IBasicVideoEffectSample.MyVideoEffect");. Any help will be appreciated!

Just using the nuget reference I get System.Runtime.InteropServices.COMException: 'Class not registered (0x80040154 (REGDB_E_CLASSNOTREG))'.

I've tried adding a reference in the app manifest (in many variations). Pointing to my dll gives System.Runtime.InteropServices.COMException: 'Error in the DLL (0x800401F9 (CO_E_ERRORINDLL))' and pointing to WinRT.Host.dll like this

<file name="runtimes\win-x64\native\WinRT.Host.dll">
    <activatableClass
        name="IBasicVideoEffectSample.MyVideoEffect"
        threadingModel="both"
        xmlns="urn:schemas-microsoft-com:winrt.v1" />
</file>

gives: Exception thrown at 0x00007FF980664F69 (KernelBase.dll) in VideoEffectConsoleApp.exe: WinRT originate error - 0x80008093 : 'The .runtimeconfig.json file is invalid.'.

Below is my VideoEffectConsoleApp.runtimeconfig.json. Note that I have to change it manually each time after building, since the template content is added inside the runtimeOptions and activatableClasses should be in the root according to this. But the error is always there.

{
  "runtimeOptions": {
    "tfm": "net6.0",
    "framework": {
      "name": "Microsoft.NETCore.App",
      "version": "6.0.0"
    },
    "configProperties": {
      "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization": false
    }
  },
  "activatableClasses": {
    "IBasicVideoEffectSample.MyVideoEffect": "IBasicVideoEffectSample.dll"
  }
}

My testing code can be found here: https://github.com/HermanEldering/IBasicVideoEffectSample

[HermanEldering]

I have managed to fix the problem. Using procmon I found out it was looking for runtimes\win-x64\native\WinRT.Host.runtimeconfig.json.

To make it a bit easier for myself I've copied the WinRT.Host.dll from the subfolder to the app path. After that I've changed the app manifest to point to that file <file name="WinRT.Host.dll"> and added WinRT.Host.runtimeconfig.json with the same content as in my original message.

For me using VideoEffectConsoleApp.runtimeconfig.json to specify where to load the class didn't work.

[HermanEldering]

Just wanted to document my further experiences here. When using the PropertySet it appears as if two separate instances of .NET are loaded because I can't cast the same type after passing it from the ConsoleApp to the IBasicVideoEffectSample library.

System.InvalidCastException: '
[A]System.Action`2[System.IntPtr,IBasicVideoEffectSample.FrameInfo] cannot be cast to 
[B]System.Action`2[System.IntPtr,IBasicVideoEffectSample.FrameInfo]. 

Type A originates from 'System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e' in the context 'Default' at location 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\6.0.1\System.Private.CoreLib.dll'. 
Type B originates from 'System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e' in the context 'Default' at location 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\6.0.1\System.Private.CoreLib.dll'.'

Type A and B are exactly the same for so far the reflection shows. With an earlier attempt when I compared the reflection of the object instance with the reflection of the field I wanted to store it in and the equality operator returned true...

Edit: The problem is with my FrameInfo type, when I replaced that with primitive types it works.

[j0shuams]

Hi @HermanEldering , thanks for opening this issue and posting updates. I have a change in PR to make a note on not using a relative path for WinRt.Host in the manifest -- I know it was frustrating working through that. Hopefully the clarification can help prevent issues like this in the future.

Did you have a reason to not use the automatically made WinRt.Host.runtimeconfig.json? In most scenarios I think the one we generate is sufficient, so I'd like to hear more on your scenario.

I think we will close this issue soon, but please open another issue/discussion if you face any other complications with authoring/hosting.

[Scottj1s]

Hi @HermanEldering, some additional thoughts:

• You should be able to use Class Name probing in your situation, since the class name is a superset of the target managed assembly. That would simplify a lot with your runtimeconfig.json template. (see Hosting docs).
• Regarding the type casting - are you trying to access a hosted object in a another AssemblyLoadContext? That's not supported and would probably manifest as a typecast error from the same type to itself.

[HermanEldering]

@j0shuams Thanks, please see my response below.

Hi @HermanEldering , thanks for opening this issue and posting updates. I have a change in PR to make a note on not using a relative path for WinRt.Host in the manifest -- I know it was frustrating working through that. Hopefully the clarification can help prevent issues like this in the future.

I'm not sure I've had a problem with relative paths. For me xcopy install is important, so absolute paths are not feasible if that would be the solution.

Did you have a reason to not use the automatically made WinRt.Host.runtimeconfig.json? In most scenarios I think the one we generate is sufficient, so I'd like to hear more on your scenario.

I've checked the build output. There is a WinRt.Host.runtimeconfig.json in the nuget package, but that file is not in the output of the console app project (which references the winrt nuget). The console app does output VideoEffectConsoleApp.runtimeconfig.json, but it needs the WinRt.Host file (too). Neither of the files have the activatableClasses section by default, which seems required.

I think we will close this issue soon, but please open another issue/discussion if you face any other complications with authoring/hosting.

I think the issue might be that the Windows.Media library is loading my WinRt component instead of doing it directly from my own code. Perhaps you can confirm whether that is the case or not.

[HermanEldering]

@Scottj1s Thanks for the reply, please see my response below.

• You should be able to use Class Name probing in your situation, since the class name is a superset of the target managed assembly. That would simplify a lot with your runtimeconfig.json template. (see Hosting docs).

For me it is hard to understand that document. It feels like more knowledge is needed before it will make sense.

• Regarding the type casting - are you trying to access a hosted object in a another AssemblyLoadContext? That's not supported and would probably manifest as a typecast error from the same type to itself.

I think the problem is that my WinRt component is loaded by the Windows.Media library instead of my code loading it directly.

I've created a WinRt component project that has the IBasicVideoEffect implementation. Then there is another console project that loads the video file and adds the IBasicVideoEffect to the graph. But it passes just the classname, not an instance and the instance is created by the Windows library.

This is the way it worked in UWP so that is the way I've approached it with CsWinRT.

[lhak]

I have also tried to implement IBasicVideoEffect using cswinrt. I can get the component to load and can see that some of the implemented methods are called succesfully. However, I then get a System.MissingMethodException:

Method not found: 'System.Collections.Concurrent.ConcurrentDictionary`2<System.RuntimeTypeHandle,WinRT.IObjectReference> WinRT.IWinRTObject.get_QueryInterfaceCache()'.

[HermanEldering]

@lhak

I then get a System.MissingMethodException

My guess is that you're passing the dictionary across the winrt boundary. Probably only limited types can transfer that way.

I don't know what can be passed. In my code I think I limited myself to primitives for this reason. You can check my code if you want.

[lhak]

Comparing to the methods that are called when using c++ to implement the video effect, I guess it crashes when trying to call:

IReadOnlyList<VideoEncodingProperties> SupportedEncodingProperties 

Unfortunately, Visual Studio does not provide any stack trace for the exception.

[kaesardev]

I have also tried to implement IBasicVideoEffect using cswinrt. I can get the component to load and can see that some of the implemented methods are called succesfully. However, I then get a System.MissingMethodException:

Method not found: 'System.Collections.Concurrent.ConcurrentDictionary`2<System.RuntimeTypeHandle,WinRT.IObjectReference> WinRT.IWinRTObject.get_QueryInterfaceCache()'.

Has there been any progress in using IBasicVideoEffect?

[christosk92]

FYI, I am still experiencing the same issue:

MissingMethodException: Method not found: 'System.Collections.Concurrent.ConcurrentDictionary`2<System.RuntimeTypeHandle,WinRT.IObjectReference> WinRT.IWinRTObject.get_QueryInterfaceCache()'.