dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.07k stars 4.69k forks source link

Stateless vs Stateful LibraryImport native marshaller initialize backing variables differently #96886

Open ThadHouse opened 8 months ago

ThadHouse commented 8 months ago

Description

When using a stateful native marshaller, all created temporary variables are initialized to default(). However, when using stateless, all variables are left uninitialized. This can result in unexpected behavior changes by the caller, especially if they change from stateful to stateless.

Reproduction Steps

using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Runtime.InteropServices.Marshalling;

namespace CsCore.Natives;

[NativeMarshalling(typeof(StatefulMarshaller))]
public enum StatefulEnum : int
{
}

[NativeMarshalling(typeof(StatelessMarshaller))]
public enum StatelessEnum : int
{
}

[CustomMarshaller(typeof(StatefulEnum), MarshalMode.ManagedToUnmanagedOut, typeof(StatefulMarshaller))]
public ref struct StatefulMarshaller {
    public void FromUnmanaged(int unmanaged)
    {
        throw new System.NotImplementedException();
    }

    public StatefulEnum ToManaged()
    {
        throw new System.NotImplementedException();
    }

    public void Free()
    {
        throw new System.NotImplementedException();
    }
}

[CustomMarshaller(typeof(StatelessEnum), MarshalMode.ManagedToUnmanagedOut, typeof(StatelessMarshaller))]
public static class StatelessMarshaller {
    public static StatelessEnum ConvertToManaged(int unmanaged)
    {
        throw new System.NotImplementedException();
    }
}

public static unsafe partial class CsNative
{
    [LibraryImport("cscore", EntryPoint = "CS_GetPropertyKind")]
    [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])]
    public static partial void Stateful(out StatefulEnum status);

    [LibraryImport("cscore", EntryPoint = "CS_GetPropertyKind")]
    [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])]
    public static partial void Stateless(out StatelessEnum status);
}

Compile the above code, and review the generated output. __status_native, which is passed as a pointer into the P/Invoke is initialized to default for the stateful generator, and not initialized at all for the stateless generator.

Expected behavior

I would expect these 2 to use the same type of initialization for the same variable generated between them.

Actual behavior

One is generated with initialization, one is not.

Regression?

No response

Known Workarounds

No response

Configuration

.NET 8

Other information

I noticed this because I had a scenario where I needed to ensure an out parameter was initialized to 0 because the native function required that, and was trying to find a method that did so. Switching to stateful did work, but adds a try catch and a ton of overhead that might not be worth it for my use case.

ghost commented 8 months ago

Tagging subscribers to this area: @dotnet/interop-contrib See info in area-owners.md if you want to be subscribed.

Issue Details
### Description When using a stateful native marshaller, all created temporary variables are initialized to default(). However, when using stateless, all variables are left uninitialized. This can result in unexpected behavior changes by the caller, especially if they change from stateful to stateless. ### Reproduction Steps ``` using System.Runtime.CompilerServices; using System.Runtime.InteropServices; using System.Runtime.InteropServices.Marshalling; namespace CsCore.Natives; [NativeMarshalling(typeof(StatefulMarshaller))] public enum StatefulEnum : int { } [NativeMarshalling(typeof(StatelessMarshaller))] public enum StatelessEnum : int { } [CustomMarshaller(typeof(StatefulEnum), MarshalMode.ManagedToUnmanagedOut, typeof(StatefulMarshaller))] public ref struct StatefulMarshaller { public void FromUnmanaged(int unmanaged) { throw new System.NotImplementedException(); } public StatefulEnum ToManaged() { throw new System.NotImplementedException(); } public void Free() { throw new System.NotImplementedException(); } } [CustomMarshaller(typeof(StatelessEnum), MarshalMode.ManagedToUnmanagedOut, typeof(StatelessMarshaller))] public static class StatelessMarshaller { public static StatelessEnum ConvertToManaged(int unmanaged) { throw new System.NotImplementedException(); } } public static unsafe partial class CsNative { [LibraryImport("cscore", EntryPoint = "CS_GetPropertyKind")] [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] public static partial void Stateful(out StatefulEnum status); [LibraryImport("cscore", EntryPoint = "CS_GetPropertyKind")] [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] public static partial void Stateless(out StatelessEnum status); } ``` Compile the above code, and review the generated output. __status_native, which is passed as a pointer into the P/Invoke is initialized to default for the stateful generator, and not initialized at all for the stateless generator. ### Expected behavior I would expect these 2 to use the same type of initialization for the same variable generated between them. ### Actual behavior One is generated with initialization, one is not. ### Regression? _No response_ ### Known Workarounds _No response_ ### Configuration .NET 8 ### Other information I noticed this because I had a scenario where I needed to ensure an `out` parameter was initialized to 0 because the native function required that, and was trying to find a method that did so. Switching to stateful did work, but adds a try catch and a ton of overhead that might not be worth it for my use case.
Author: ThadHouse
Assignees: -
Labels: `area-System.Runtime.InteropServices`, `untriaged`
Milestone: -