[Question] Packaging Platform specific from Vendor dll

tebeco commented 4 years ago

Details about Problem

NuGet product used => dotnet.exe:

I tried multiple version of dotnet:

3.1.201 (dotnet nuget 5.5.0.4)
5.0.100-preview.3.20216.6 (dotnet nuget 5.6.0.4)

VS version (if appropriate): VS 16.6 Preview 5

OS version (i.e. win10 v1607 (14393.321)): Microsoft Windows [Version 10.0.19619.1000]

Worked before? If so, with which NuGet version: Can't tell

Detailed repro steps so we can see the same problem

Clone the repo https://github.com/tebeco/NativeLibNugetStuff/
read https://github.com/tebeco/NativeLibNugetStuff/blob/master/README.md
i'm trying to get dotnet build works on the SLN (as opposed to the build.ps1 that have to split the build process in 2 and complexify stuff)

I'm looking for a nice way to be able to have ProjectReference working when ManagedWrapper are already "built in" I had to use PublishBuildOutput to false to tell the compiler to leave me alone in Corp.Common.ThirdPartPackaged as I got 0 sources (that may be a mistake on my side TBH)

With PackageReference :

With ProjectReference :

Sample Project

https://github.com/tebeco/NativeLibNugetStuff/ https://github.com/tebeco/NativeLibNugetStuff/blob/master/README.md

Workaround

The only workaround i see here would be to split in half the repo and the build to have a 2 step build process to be sure the nuget is produced As it would fix the "side effect" by them self that would not fix the issue and would make the build more complexe

kartheekp-ms commented 4 years ago

As per doc, ProjectReference project item has an item metadata named ReferenceOutputAssembly when set to false, does not include the output of the referenced project as a Reference of this project, but still ensures that the other project builds before this one.

I found another achieved doc which refers to the same ReferenceOutputAssembly MSBuild item with an example.

Please let us know If this comment doesn't fix your scenario.

tebeco commented 4 years ago

Did that work for you with the Repository I created just for this issue ? As I do not fully understand how nuget and dependency resolution I tested, that does not fix the issue

given the description :

when set to false, does not include the output of the referenced project as a Reference of this project, but still ensures that the other project builds before this one.

Is this related to the issue ? the project i'm using to pack our vendor dll is : Corp.Common.ThirdPartPackaged with PublishBuildOutput = false => this means that there is no Corp.Common.ThirdPartPackaged.dll produce => this means that "Consuming" it won't need to be forced to specify ReferenceOutputAssembly = false right ?

Also : PublishBuildOutput = false + PackageReference => does load transitive DLL PublishBuildOutput = false + ProjectReference => does not load transitive DLL ThirdPart.ManagedWrapper.dll <== this is the bug, it should be loaded from the Corp.Common.ThirdPartPackaged) PublishBuildOutput = false + ReferenceOutputAssembly = false ==> KO PublishBuildOutput = true + ReferenceOutputAssembly = false ==> KO

tebeco commented 4 years ago

@kartheekp-ms Can you try to see if that works on your side with the repository create for reproductibility purpose and tell me if you managed to fix it using your proposal ?

I changed :

  <ItemGroup>
    <PackageReference Include="Corp.Common.ThirdPartPackaged" Version="*" />
    <!-- <ProjectReference Include="../Corp.Common.ThirdPartPackaged/Corp.Common.ThirdPartPackaged.csproj" ReferenceOutputAssembly="false"/> -->
  </ItemGroup>

to

  <ItemGroup>
    <!--<PackageReference Include="Corp.Common.ThirdPartPackaged" Version="*" />-->
    <ProjectReference Include="../Corp.Common.ThirdPartPackaged/Corp.Common.ThirdPartPackaged.csproj" ReferenceOutputAssembly="false"/>
  </ItemGroup>

but as describe my issue is not about loading Corp.Common.ThirdPartPackaged but the transitive DLL ThirdPart.ManagedWrapper.dll that our vendor sends us by mail

donnie-msft commented 4 years ago

You could use nuget pack with -IncludeReferencedProjects. Have you tried this approach?

Secondly, you may try dotnet add command for this reference. If you already did, please share the --verbosity detailed output? dotnet add app/app.csproj reference lib/lib.csproj https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-add-reference#examples

Here's a StackOverflow describing adding the DLLs with the .nuspec approach https://stackoverflow.com/questions/43277715/create-nuget-package-from-dlls

tebeco commented 4 years ago

the repository I created especially for that and linked with all the repro step did not work for you either ?

will try that tomorrow

tebeco commented 4 years ago

TLDR

I think that there's something I probably did not understand correctly in the readme I created for the repro Can you tell me if there's some confusion somehow ?

Do you prefer that I split the repo in two like:

NO GOAL (I have a "thing" working but it's not what i'm trying to acheive as it force me do build an "inner nuget" during a build of another nuget and also unit testing
GOAL ... I could add Scripts or whatever, Just tell what went wrong when you tried to build it The goal here is just to clone the repo and change :

https://github.com/tebeco/NativeLibNugetStuff/blob/master/Corp.Common/src/Corp.Common.Lib/Corp.Common.Lib.csproj from

  <ItemGroup>
    <PackageReference Include="Corp.Common.ThirdPartPackaged" Version="*" />
    <!--<ProjectReference Include="../Corp.Common.ThirdPartPackaged/Corp.Common.ThirdPartPackaged.csproj" />-->
  </ItemGroup>

to

  <ItemGroup>
    <!--<PackageReference Include="Corp.Common.ThirdPartPackaged" Version="*" />-->
    <ProjectReference Include="../Corp.Common.ThirdPartPackaged/Corp.Common.ThirdPartPackaged.csproj" />
  </ItemGroup>

You could use nuget pack with -IncludeReferencedProjects. Have you tried this approach?

https://github.com/tebeco/NativeLibNugetStuff/blob/master/Corp.Common/src/Corp.Common.ThirdPartPackaged/Corp.Common.ThirdPartPackaged.csproj As you can see there's 0 ProjectReference or PackageReference so i'm not sure which project you you want me to run that ? (also I don't use nuget here, only dotnet and dotnet pack)

Secondly, you may try dotnet add command for this reference. If you already did, please share the --verbosity detailed output? dotnet add app/app.csproj reference lib/lib.csproj

From which project to which project ?

Here's a StackOverflow describing adding the DLLs with the .nuspec approach https://stackoverflow.com/questions/43277715/create-nuget-package-from-dlls

The nugets seems works perfectly, my issue is that a DON'T want a <PackageReference but a <ProjectReference

tebeco commented 4 years ago

What's your personal syntax for verbosity on add reference ?

tebeco commented 4 years ago

did that answered your questions @donnie-msft @kartheekp-ms ?

donnie-msft commented 4 years ago

@tebeco sorry and nevermind about detailed verbosity, as apparently that's not supported yet.

Have you tried starting from scratch, and using the dotnet add command I referenced earlier? The simpler the repro, the more clarity on the problem.

I cloned your repo and even if I change nothing at all, there are errors about references which don't exist. See Output Window with "Show output from Solution" selected:

C:\apps\NativeLibNugetStuff\NativeLibNugetStuff.Consumer\NativeLibNugetStuff.Consumer.csproj : error : The project file could not be loaded. Could not find a part of the path 'C:\apps\NativeLibNugetStuff\NativeLibNugetStuff.Consumer\NativeLibNugetStuff.Consumer.csproj'. C:\apps\NativeLibNugetStuff\NativeLibNugetStuff.Consumer\NativeLibNugetStuff.Consumer.csproj

C:\apps\NativeLibNugetStuff\NativeLibNugetStuff.Lib\NativeLibNugetStuff.Lib.csproj : error : The project file could not be loaded. Could not find a part of the path 'C:\apps\NativeLibNugetStuff\NativeLibNugetStuff.Lib\NativeLibNugetStuff.Lib.csproj'. C:\apps\NativeLibNugetStuff\NativeLibNugetStuff.Lib\NativeLibNugetStuff.Lib.csproj

C:\apps\NativeLibNugetStuff\NativeLibNugetStuff.Wrapper\NativeLibNugetStuff.Wrapper.csproj : error : The project file could not be loaded. Could not find a part of the path 'C:\apps\NativeLibNugetStuff\NativeLibNugetStuff.Wrapper\NativeLibNugetStuff.Wrapper.csproj'. C:\apps\NativeLibNugetStuff\NativeLibNugetStuff.Wrapper\NativeLibNugetStuff.Wrapper.csproj

tebeco commented 4 years ago

oO i'll clone it elsewhere to be sure what's going on and I'll fix that this weekend ;) Sorry about that

This repo is like the "minimal" stuff we do at work and as you suggested I started from scratch

Warning : native build

The only thing is that, in order for you to "trust me" i did not added any native dll so that's why it contains Rust code + a script to build it down to native So you'll need to have rust https://www.rust-lang.org/tools/install I could probably to the same in C / gcc or go, the rust one was already done by @poke, and it felt very very easy to build a native lib

Warning : native pack

The path in the csproj are hardcoded for win-x64 If you use it over MacOs or Linux or wsl tell me so that I'll update the repo

tebeco commented 4 years ago

@donnie-msft sorry about the repo, that sln should have never existed in my fork. It's an artefact from the Original repo before the fork I re-cloned the repo else where and just ran build.ps1 can you let me know if that build.ps1 works for you ?

 C:\dev\github\temp\NativeLibNugetStuff   master ≣ 
[19:14]❯ cat .\build.ps1

Write-Host 'Running ThirdPart build script' -ForegroundColor Blue
./ThirdPart/build.ps1

Write-Host 'Running Common lib build script' -ForegroundColor Blue
./Corp.Common/build.ps1

Write-Host 'Running Consumer build script' -ForegroundColor Blue
./Corp.FeatureTeam/build.ps1

Once it's done, the idea is that you CANNOT chance third-part folder (except the samples) The sln you will want to open is : ./Corp.Common/Corp.Common.sln within this solution this issue is in the ./Corp.Common/src/Corp.Common.Lib/Corp.Common.Lib.csproj Where I'm trying to replace the PackageReference to a ProjectReference toward Corp.Common.ThirdPartPackaged

here is how the "reclone / build" went here :

[18:40]❯ cd ..\temp\
 C:\dev\github\temp 
[18:40]❯ git clone https://github.com/tebeco/NativeLibNugetStuff/
Cloning into 'NativeLibNugetStuff'...
remote: Enumerating objects: 106, done.
remote: Counting objects: 100% (106/106), done.
remote: Compressing objects: 100% (79/79), done.
Receiving objects:  42% (45/106)used 96 (delta 20), pack-reused 0
Receiving objects: 100% (106/106), 15.89 KiB | 856.00 KiB/s, done.
Resolving deltas: 100% (30/30), done.

 C:\dev\github\temp\NativeLibNugetStuff   master ≣ 
[18:58]❯ .\build.ps1
Running ThirdPart build script
ThirdPart Stuff
Cleaning pre-existing artifacts
Building thirdparth native library (Rust)
   Compiling third-part-native v0.1.0 (C:\dev\github\temp\NativeLibNugetStuff\ThirdPart\src\third-part-native)
    Finished release [optimized] target(s) in 3.71s
Building thirdparth managed wrapper(CSharp)
Microsoft (R) Build Engine version 16.6.0-preview-20181-02+9f3e4e725 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored C:\dev\github\temp\NativeLibNugetStuff\ThirdPart\src\ThirdPart.ManagedWrapper\ThirdPart.ManagedWrapper.csproj (in 264 ms).
  You are using a preview version of .NET Core. See: https://aka.ms/dotnet-core-preview
  You are using a preview version of .NET Core. See: https://aka.ms/dotnet-core-preview
  ThirdPart.ManagedWrapper -> C:\dev\github\temp\NativeLibNugetStuff\ThirdPart\src\ThirdPart.ManagedWrapper\bin\Release\net5.0\ThirdPart.ManagedWrapper.dll
  ThirdPart.ManagedWrapper -> C:\dev\github\temp\NativeLibNugetStuff\ThirdPart\src\ThirdPart.ManagedWrapper\bin\Release\netstandard2.0\ThirdPart.ManagedWrapper.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:03.39
Copying ThirdPart build output to C:\dev\github\temp\NativeLibNugetStuff\ThirdPart/lib
Running Common lib build script
Building Corp.Common
Copying third part binary to Corp.Common.ThirdPartPackaged
Packaging Version 2020.0516.1858.18
Building Corp.Common.ThirdPartPackaged
Microsoft (R) Build Engine version 16.6.0-preview-20181-02+9f3e4e725 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored C:\dev\github\temp\NativeLibNugetStuff\Corp.Common\src\Corp.Common.ThirdPartPackaged\Corp.Common.ThirdPartPackaged.csproj (in 105 ms).
  You are using a preview version of .NET Core. See: https://aka.ms/dotnet-core-preview
  Corp.Common.ThirdPartPackaged -> C:\dev\github\temp\NativeLibNugetStuff\Corp.Common\src\Corp.Common.ThirdPartPackaged\bin\Debug\netstandard2.0\Corp.Common.ThirdPartPackaged.dll
  Successfully created package 'C:\dev\github\temp\NativeLibNugetStuff\nugets\Corp.Common.ThirdPartPackaged.2020.516.1858.18.nupkg'.
Building Corp.Common.Lib
Microsoft (R) Build Engine version 16.6.0-preview-20181-02+9f3e4e725 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored C:\dev\github\temp\NativeLibNugetStuff\Corp.Common\src\Corp.Common.Lib\Corp.Common.Lib.csproj (in 196 ms).
  You are using a preview version of .NET Core. See: https://aka.ms/dotnet-core-preview
  Corp.Common.Lib -> C:\dev\github\temp\NativeLibNugetStuff\Corp.Common\src\Corp.Common.Lib\bin\Debug\netstandard2.0\Corp.Common.Lib.dll
  Successfully created package 'C:\dev\github\temp\NativeLibNugetStuff\nugets\Corp.Common.Lib.2020.516.1858.18.nupkg'.
Running Consumer build script
Publishing Corp.FeatureTeam.Host
Microsoft (R) Build Engine version 16.6.0-preview-20181-02+9f3e4e725 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored C:\dev\github\temp\NativeLibNugetStuff\Corp.FeatureTeam\Corp.FeatureTeam.Host\Corp.FeatureTeam.Host.csproj (in 1.34 sec).
  You are using a preview version of .NET Core. See: https://aka.ms/dotnet-core-preview
  Corp.FeatureTeam.Host -> C:\dev\github\temp\NativeLibNugetStuff\Corp.FeatureTeam\Corp.FeatureTeam.Host\bin\Release\net5.0\win10-x64\Corp.FeatureTeam.Host.dll
  Corp.FeatureTeam.Host -> C:\dev\github\temp\NativeLibNugetStuff\Corp.FeatureTeam\Corp.FeatureTeam.Host\bin\Release\net5.0\win10-x64\publish\
Running published Corp.FeatureTeam.Host
Native function do_stuff called with argument 12
12: 144
Native function do_stuff called with argument 23
23: 529

tebeco commented 4 years ago

@donnie-msft was that a bit better ?

NuGet / Home