Add a way to list native assets that a project will load from the app directory (list them in deps.json)

[safern]

In dotnet/runtime we're currently enabling a feature for applications to carry their own copy of ICU for globalization purposes instead of using system-wide installed ICU.

In order to make this feature work, we're using NativeLibrary.TryLoad API which uses some probing using NATIVE_DLL_SEARCH_DIRECTORIES property passed by the host to the runtime. However, this property is populated based of the deps.json file, so if we don't have any native assets coming from NuGet packages or we want to use a different folder for our custom built native assets, we wont be able to load from there as NATIVE_DLL_SEARCH_DIRECTORIES will not include this directory. This is of course blocking for framework dependent apps as for self contained apps NATIVE_DLL_SEARCH_DIRECTORIES includes the app directory always.

Currently the only way to have native assets included in deps.json file is via NuGet resolved assets, however, I did manage to do this by adding:

  <ItemGroup>
    <IcuAssemblies Include="icu\*.so*" />
    <RuntimeTargetsCopyLocalItems Include="@(IcuAssemblies)" AssetType="native" CopyLocal="true" DestinationSubDirectory="runtimes/linux-x64/native/" DestinationSubPath="%(FileName)%(Extension)" RuntimeIdentifier="linux-x64" NuGetPackageId="System.Private.Runtime.UnicodeData" />
  </ItemGroup>

This requires me to specify a NuGetPackageId and it has to be an actual package that the app references, so if I have a project that doesn't reference any NuGet packages, then I won't be able to do this.

Open questions

Should assets specify a RID or should the SDK assume it, based on the publish/build rid? How should they be listed in the deps.json file? Under the built app library section? Under a new loose native assets section? We need to make sure this works end-to-end on publish, packaging and as transitive dependencies as what if I have a reference to an app that depends on this loose native assets?

Notes

This will be a very important feature for ICU story since we're now providing a default behavior to use ICU on Windows and many customers will want to use the same version or a custom version with custom locales in their app across different OSs.

cc: @dsplaisted @ericstj @jkotas @tarekgh @eerhardt @jeffschwMSFT

[dsplaisted]

Some possible syntaxes (based on a group discussion):

<ItemGroup>
  <NativeLibrary Include="path/to/icu.so" RuntimeIdentifier="linux-x64" />

  <Content Include="path/to/icu.so" CopyToOutput="PreserveNewest"
     RuntimeIdentifier="linux-x64" />
</ItemGroup>

Ideally this would be transitive. That could require plumbing through NuGet.

Also look at how Xamarin represents native assets for iOS and Android.

[tarekgh]

Is it possible we can plumb NuGet from now?

[ericstj]

Might be the same issue as https://github.com/dotnet/sdk/issues/10575 Related https://github.com/dotnet/runtime/issues/11404 Seems like it's needed for https://github.com/dotnet/core/issues/5699 cc @terrajobst

I hacked this together a bit. The SDK does honor items that the project creates but they must be tied to a package. It won't honor any items that claim to come from the project.

Here's a very ugly hack: https://github.com/ericstj/sample-code/tree/nativeLibSample I wouldn't recommend shipping this deps file produced with this method, but it might do the job for local testing.

Not sure if NuGet would need to be involved for transitive references, what if these were just flowed via MSBuild items and you could add them to https://github.com/dotnet/sdk/blob/b91b88aec2684e3d2988df8d838d3aa3c6240a35/src/Tasks/Microsoft.NET.Build.Tasks/SingleProjectInfo.cs#L24?

[dotMorten]

<NativeLibrary Include="path/to/icu.so" RuntimeIdentifier="linux-x64" />

This is what I'd like to see. I'd also like for the nuget pack command to then correctly pack the library. That way when I do a nuget reference or a project reference, things works the exact same way

[robmen]

@ericstj and @terrajobst we also need a solution for this in the WiX Toolset to find things like mergemod.dll. For now, we're going to try to go with Eric's "very ugly hack"

[taoyouh]

It seems that the RuntimeTargetsCopyLocalItems solution is not transitive. Another project that references the project with RuntimeTargetsCopyLocalItems will not have the native assets included in the output.

[ericstj]

Correct, since it's not part of the assets file it's not transitive. There is no good way to flow additional files transitively through ProjectReferences by default. Your best bet is to handle it with some common targets that can be used in multiple projects in your repository.