How does --additional-deps work along with --additionalprobingpath

livarcocc commented 6 years ago

From @hvanbakel on January 24, 2018 20:46

First of all, what I'm trying to do here is to run an assembly as a container to be reused by multiple other assemblies. So I would like to run something like

dotnet MyHost.dll MyImplementation.dll

where MyHost.dll is reused and sets up the runtime for MyImplementation.dll (which contains webapi controllers). In order for that to work, the dotnet cli will need to understand what it needs to load. I've been reading this and I'm able to get it to work by manually crafting the deps.json file. However, I'm trying to use the existing tooling for this and I was under the impression that the --addtional-deps and --additionalprobingpath were to be used for this. However, I cannot get this to work. NetCoreLoaderTest.zip

Steps to reproduce

Use the added solution. What I'm trying to find out is what the difference is between running with --additional-deps and having a project reference.

Expected behavior

Running with additional-deps is equivalent to merging the deps.json files and/or running with a project reference. I would expect it to be the equivalent of 'merging' the deps.json files together.

Actual behavior

Having a project reference, building the solution and starting it (in the bin/debug/netcoreapp2.0 folder) using

dotnet NetCoreLoaderTest.dll

works

Removing the project reference and running it like this (again in the bin/debug/netcoreapp2.0 folder):

dotnet --additional-deps ........\Dep endentProject\bin\Debug\netcoreapp2.0\DependentProject.deps.json --additionalprobingpath ........\DependentProject\bi n\Debug\netcoreapp2.0\ .\NetCoreLoaderTest.dll

does not work. It crashes with:

Processing TPA for deps entry [DependentProject, 1.0.0, DependentProject.dll] Considering entry [DependentProject/1.0.0/DependentProject.dll] and probe dir [C:\Program Files\dotnet\shared\Microsof t.NETCore.App\2.0.5] Skipping... probe in deps json failed Skipping... not found in probe dir 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\2.0.5' Considering entry [DependentProject/1.0.0/DependentProject.dll] and probe dir [] Local path query did not exist D:\code\sandbox\NetCoreLoaderTest\NetCoreLoaderTest\bin\Debug\netcoreapp2.0\Dependent Project.dll Skipping... probe in deps dir 'D:\code\sandbox\NetCoreLoaderTest\NetCoreLoaderTest\bin\Debug\netcoreapp2.0\' failed Skipping... not found in probe dir '' Considering entry [DependentProject/1.0.0/DependentProject.dll] and probe dir [C:\Program Files\dotnet\store\x64\netco reapp2.0] Relative path query did not exist C:\Program Files\dotnet\store\x64\netcoreapp2.0\DependentProject\1.0.0\DependentPr oject.dll Skipping... not found in probe dir 'C:\Program Files\dotnet\store\x64\netcoreapp2.0' Considering entry [DependentProject/1.0.0/DependentProject.dll] and probe dir [D:\code\sandbox\NetCoreLoaderTest\Depen dentProject\bin\Debug\netcoreapp2.0] Relative path query did not exist D:\code\sandbox\NetCoreLoaderTest\DependentProject\bin\Debug\netcoreapp2.0\Depende ntProject\1.0.0\DependentProject.dll Skipping... not found in probe dir 'D:\code\sandbox\NetCoreLoaderTest\DependentProject\bin\Debug\netcoreapp2.0\' Considering entry [DependentProject/1.0.0/DependentProject.dll] and probe dir [C:\Users\jvanbakel.nuget\packages] Relative path query did not exist C:\Users\jvanbakel.nuget\packages\DependentProject\1.0.0\DependentProject.dll Skipping... not found in probe dir 'C:\Users\jvanbakel.nuget\packages' Considering entry [DependentProject/1.0.0/DependentProject.dll] and probe dir [C:\Program Files\dotnet\sdk\NuGetFallba ckFolder] Relative path query did not exist C:\Program Files\dotnet\sdk\NuGetFallbackFolder\DependentProject\1.0.0\DependentPr oject.dll Skipping... not found in probe dir 'C:\Program Files\dotnet\sdk\NuGetFallbackFolder' Error: An assembly specified in the application dependencies manifest (DependentProject.deps.json) was not found: package: 'DependentProject', version: '1.0.0' path: 'DependentProject.dll'

So from the bold lines, I'm taking that for some reason it thinks it is a package now?

My other solution was to copy all the files into one folder (so the output from both projects) and then run:

dotnet --additional-deps .\DependentProj ect.deps.json .\NetCoreLoaderTest.dll

But that crashes with

Unhandled Exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.DllNotFoundException: Unable to load DLL 'sni.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)

so that doesn't seem to load the runtime dependencies?

Environment data

dotnet --info output: .NET Command Line Tools (2.1.4)

Product Information: Version: 2.1.4 Commit SHA-1 hash: 5e8add2190

Runtime Environment: OS Name: Windows OS Version: 10.0.15063 OS Platform: Windows RID: win10-x64 Base Path: C:\Program Files\dotnet\sdk\2.1.4\

Copied from original issue: dotnet/cli#8467

hvanbakel commented 6 years ago

Additional information, if I drop all the assemblies into a single folder. The code fails when I use --additional-deps and specify one additional deps file. It fails because it doesn't seem to load the runtime values from the additional deps file (see the attached deps not working zip). So that is with

dotnet --additional-deps DependentProject.deps.json NetCoreLoaderTest.dll

If I however manually merge the 2 deps files into 1 and run

dotnet NetCoreLoaderTest.dll

then it works. I've attached the manual merged file. Again, my expected behavior is that the additional deps is merged with the NetCoreLoaderTest.deps.json.

deps not working.zip

deps working.zip

hvanbakel commented 6 years ago

Probably linked with https://github.com/dotnet/corefx/issues/11639

hvanbakel commented 6 years ago

After talking with @natemcmaster it seems the bug here is that the runtime dependencies are not loaded when --additional-deps is specified. The logic this was added for: https://github.com/dotnet/core-setup/issues/1597 and https://github.com/aspnet/Hosting/issues/951 which didn't need any runtime dependencies.

natemcmaster commented 6 years ago

For context, that conversation was here: http://www.natemcmaster.com/blog/2017/12/21/netcore-primitives/#comment-3724684249

steveharter commented 6 years ago

This is a bug. --additional-deps does not correctly process native dlls.

steveharter commented 6 years ago

I will fix this issue soon for 2.1, and provide a port to 2.0.x servicing.

steveharter commented 6 years ago

@hvanbakel does waiting for 2.1 work for you? Fixing this in 2.0.x is a higher bar.

@natemcmaster @davidfowl any thoughts on whether we should port this to 2.0.x for the original feature in https://github.com/dotnet/core-setup/issues/1597?

hvanbakel commented 6 years ago

@steveharter for me it would be fine to wait till 2.1 I've put in a temporary fix by inverting the hosting to not use additional deps but use the IHostingStartup and environment variables for now. I'll switch back once this is released.

just wondering, is there a way to test with a nightly build maybe so I can confirm that it's working without building the code myself?

natemcmaster commented 6 years ago

any thoughts on whether we should port this to 2.0.x for the original feature in dotnet/runtime#2620?

Given there are few workarounds available, this seems like a good patch candidate for 2.0.x. Impact should be low. Native assets is not something aspnetcore or AppInsights needed for dotnet/runtime#2620, but it would be still be a good fix for users like @hvanbakel who are trying to build their own extensions.

steveharter commented 6 years ago

@hvanbakel I did not add an automated test although I did confirm your scenario is fixed.

However, here's a way to verify the fix: 1) Open a command prompt in Windows 2) Set COREHOST_TRACE=1 3) Using a console app (e.g. console1) 4) Run -additional-deps with the current apps's own deps file: dotnet exec --additional-deps "bin\Debug\console1.deps.json" bin\Debug\console1.dll 2>out.txt 5) Check in out.txt to see if it thinks the console1 entry via the additional-deps's is a native\resource dlls. So you should not see this if the bug is fixed: Processing native/culture for deps entry [console1, 1.0.0, console1.dll]

Also for a simple console app, the end of out1.txt should look like this (if the bug is fixed) Property NATIVE_DLL_SEARCH_DIRECTORIES = C:\Program Files\dotnet\shared\Microsoft.NETCore.App\2.0.x\;

and not this (if the bug exists)

Property NATIVE_DLL_SEARCH_DIRECTORIES = D:\git\repros\console1\bin\Debug\netcoreapp2.0\;C:\Program Files\dotnet\shared\Microsoft.NETCore.App\2.0.x\;

hvanbakel commented 6 years ago

@steveharter what version of net core should I have to see this change? I'm running 2.1.4 and I don't think I see the issue as fixed.

I just tried with the latest build (from here), and I cannot confirm what you're saying as COREHOST_TRACE no longer seems to do anything (I just get the short output).

Here's my dotnet info

.NET Command Line Tools (2.1.4)

Product Information:
 Version:            2.1.4
 Commit SHA-1 hash:  5e8add2190

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.15063
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.1.4\
Performance Summary:
[97.74% 00.039s] Thread 1
  [93.72% 95.89% 00.037s] Program:Main

Microsoft .NET Core Shared Framework Host

  Version  : 2.1.0-preview2-26209-04
  Build    : 5df6e9b7ab674a461b2a7f01ac87fb6e0ca06666

hvanbakel commented 6 years ago

I just verified with the net core 2.1 preview release, this issue is resolved.

I haven't been able to get it to work on net core 2.0 though even though I this has been backported?

steveharter commented 6 years ago

Great. This will not be back ported to 2.0, unless there is additional pressure to do so.

I created https://github.com/dotnet/core-setup/issues/3584 for the port, but never got shiproom approval on that, mostly because 2.1 will be a long-term-support version.

hvanbakel commented 6 years ago

Interesting I thought dotnet/core-setup#3649 was the bug associated with that. So if I understand you correctly, 2.0 will read the additional-deps correctly but wouldn't be able to find the native images at that point because additionalprobingpath is not in 2.0?

steveharter commented 6 years ago

Sorry, your issue numbers are correct, I meant https://github.com/dotnet/core-setup/issues/3649

Yes 2.0 does not \ will not support additional-deps for native images (and resource files).

hvanbakel commented 6 years ago

I was able to get my solution to work on 2.0 after finding yet another way to do this: https://github.com/dotnet/coreclr/issues/13277

So assuming you have application A and library B and you where A is the host that needs to dynamically load B. If (and only if) A is not dependent upon any native images then running

dotnet exec --depsfile B.deps.json A.dll

will work but

dotnet --additional-deps B.deps.json A.dll

does not

Zetanova commented 6 years ago

I have the similar system in .net45 and i had now issues with Assembly.Load and Breakpoints.

@livarcocc How do u live debug?

liesauer commented 5 years ago

u can try this aewsome tool, NetCoreBeauty

dotnet / runtime