Created project.assets.json does not contain all assets

Stefan75 commented 1 year ago

NuGet Product Used

MSBuild.exe

Product Version

MSBuild version 17.4.1+9a89d02ff for .NET Framework 17.4.1.60106

Worked before?

no

Impact

It's more difficult to complete my work

Repro Steps & Context

Hello,

I created a C# project with the package reference NUnit3TestAdapter.

After restore and build, I find following files in bin\debug\net4.8:

nunit.engine.api.dll
nunit.engine.core.dll
nunit.engine.dll
NUnit3.TestAdapter.dll
NUnit3.TestAdapter.pdb
Project.dll
Project.pdb
testcentric.engine.metadata.dll

I would have expected that in the first section these files are listed:

  "targets": {
    ".NETFramework,Version=v4.8": {
      "NUnit3TestAdapter/4.3.1": {
        "type": "package",
        "build": {
          "build/net35/NUnit3TestAdapter.props": {}
        }
      }
    }
  },

Is this a bug?

If this is not a bug, how can I determine the actual copied files for each NuGet package? (Important for use-cases like SBOM, mapping Files to the source NuGet package)

Regards, stefan

Verbose Logs

I created the following project file and performed a restore and build:
 xml
<?xml version="1.0" encoding="utf-8"?>
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Library</OutputType>
    <TargetFramework>net48</TargetFramework>
    <LangVersion>latest</LangVersion>
  </PropertyGroup>
  <PropertyGroup>
    <DisableImplicitNuGetFallbackFolder>true</DisableImplicitNuGetFallbackFolder>
    <AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
    <GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>
    <ManagePackageVersionsCentrally>False</ManagePackageVersionsCentrally>
  </PropertyGroup>
  <ItemGroup>
    <Reference Include="System" />
    <Reference Include="System.Core" />
    <Reference Include="System.Xaml" />
    <Reference Include="System.XML" />
    <Reference Include="System.Xml.Linq" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="NUnit3TestAdapter" Version="4.3.1" />
  </ItemGroup>
</Project>

After restore and build, I find following files in bin\debug\net4.8:

nunit.engine.api.dll
nunit.engine.core.dll
nunit.engine.dll
NUnit3.TestAdapter.dll
NUnit3.TestAdapter.pdb
Project.dll
Project.pdb
testcentric.engine.metadata.dll

But If I look into the created project.assets.json file:

{
  "version": 3,
  "targets": {
    ".NETFramework,Version=v4.8": {
      "NUnit3TestAdapter/4.3.1": {
        "type": "package",
        "build": {
          "build/net35/NUnit3TestAdapter.props": {}
        }
      }
    }
  },
  "libraries": {
    "NUnit3TestAdapter/4.3.1": {
      "sha512": "R+bGFtsUpLWywjT1nb3xMmoVa2AIw6ClIGC+XjW9lYE8hwJeos+NdR/mtg4RXbBphmC9epALrnUc6MM7mUG8+Q==",
      "type": "package",
      "path": "nunit3testadapter/4.3.1",
      "files": [
        ".nupkg.metadata",
        ".signature.p7s",
        "build/net35/NUnit3.TestAdapter.dll",
        "build/net35/NUnit3.TestAdapter.pdb",
        "build/net35/NUnit3TestAdapter.props",
        "build/net35/nunit.engine.api.dll",
        "build/net35/nunit.engine.core.dll",
        "build/net35/nunit.engine.dll",
        "build/net35/testcentric.engine.metadata.dll",
        "build/netcoreapp3.1/NUnit3.TestAdapter.dll",
        "build/netcoreapp3.1/NUnit3.TestAdapter.pdb",
        "build/netcoreapp3.1/NUnit3TestAdapter.props",
        "build/netcoreapp3.1/nunit.engine.api.dll",
        "build/netcoreapp3.1/nunit.engine.core.dll",
        "build/netcoreapp3.1/nunit.engine.dll",
        "build/netcoreapp3.1/testcentric.engine.metadata.dll",
        "nunit3testadapter.4.3.1.nupkg.sha512",
        "nunit3testadapter.nuspec"
      ]
    }
  },
  "projectFileDependencyGroups": {
    ".NETFramework,Version=v4.8": [
      "NUnit3TestAdapter >= 4.3.1"
    ]
  },
  "packageFolders": {
    "C:\\Users\\schostzf\\AppData\\Roaming\\NuGetPackageResolver\\Temp\\2023-02-03_143058.6\\Packages": {}
  },
  "project": {
    "version": "1.0.0",
    "restore": {
      "projectUniqueName": "C:\\Users\\schostzf\\AppData\\Roaming\\NuGetPackageResolver\\Temp\\2023-02-03_143058.6\\Project.csproj",
      "projectName": "Project",
      "projectPath": "C:\\Users\\schostzf\\AppData\\Roaming\\NuGetPackageResolver\\Temp\\2023-02-03_143058.6\\Project.csproj",
      "packagesPath": "C:\\Users\\schostzf\\AppData\\Roaming\\NuGetPackageResolver\\Temp\\2023-02-03_143058.6\\Packages",
      "outputPath": "C:\\Users\\schostzf\\AppData\\Roaming\\NuGetPackageResolver\\Temp\\2023-02-03_143058.6\\obj\\",
      "projectStyle": "PackageReference",
      "configFilePaths": [
        "C:\\Users\\schostzf\\AppData\\Roaming\\NuGetPackageResolver\\Temp\\2023-02-03_143058.6\\NuGet.Config",
        "C:\\Users\\schostzf\\AppData\\Roaming\\NuGet\\NuGet.Config",
        "C:\\Program Files (x86)\\NuGet\\Config\\Microsoft.VisualStudio.FallbackLocation.config",
        "C:\\Program Files (x86)\\NuGet\\Config\\Microsoft.VisualStudio.Offline.config"
      ],
      "originalTargetFrameworks": [
        "net48"
      ],
      "sources": {
        "C:\\Program Files\\dotnet\\library-packs": {},
        "https://api.nuget.org/v3/index.json": {}
      },
      "frameworks": {
        "net48": {
          "targetAlias": "net48",
          "projectReferences": {}
        }
      },
      "warningProperties": {
        "warnAsError": [
          "NU1605"
        ]
      }
    },
    "frameworks": {
      "net48": {
        "targetAlias": "net48",
        "dependencies": {
          "NUnit3TestAdapter": {
            "target": "Package",
            "version": "[4.3.1, )"
          }
        },
        "runtimeIdentifierGraphPath": "C:\\Program Files\\dotnet\\sdk\\7.0.102\\RuntimeIdentifierGraph.json"
      }
    }
  }
}

I would have expected that in the first section these files are listed:

  "targets": {
    ".NETFramework,Version=v4.8": {
      "NUnit3TestAdapter/4.3.1": {
        "type": "package",
        "build": {
          "build/net35/NUnit3TestAdapter.props": {}
        }
      }
    }
  },

If I add as an additional component NUnit to, then the first section includes these assemblies:

  "targets": {
    ".NETFramework,Version=v4.8": {
      "NUnit/3.13.3": {
        "type": "package",
        "compile": {
          "lib/net45/nunit.framework.dll": {
            "related": ".xml"
          }
        },
        "runtime": {
          "lib/net45/nunit.framework.dll": {
            "related": ".xml"
          }
        },
        "build": {
          "build/NUnit.props": {}
        }
      },
      "NUnit3TestAdapter/4.3.1": {
        "type": "package",
        "build": {
          "build/net35/NUnit3TestAdapter.props": {}
        }
      }
    }

jeffkl commented 1 year ago

@Stefan75 at the moment this is by design. This particular package only contains MSBuild logic which extends your project's build by adding items to be copied to the output directory.

https://nuget.info/packages/NUnit3TestAdapter/4.3.1

build/net35/NUnit3TestAdapter.props

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <ItemGroup>
    <None Include="$(MSBuildThisFileDirectory)NUnit3.TestAdapter.dll">
      <Link>NUnit3.TestAdapter.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </None>
    <None Include="$(MSBuildThisFileDirectory)NUnit3.TestAdapter.pdb" Condition="Exists('$(MSBuildThisFileDirectory)NUnit3.TestAdapter.pdb')">
      <Link>NUnit3.TestAdapter.pdb</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </None>
    <None Include="$(MSBuildThisFileDirectory)nunit.engine.dll">
      <Link>nunit.engine.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </None>
    <None Include="$(MSBuildThisFileDirectory)nunit.engine.api.dll">
      <Link>nunit.engine.api.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </None>
    <None Include="$(MSBuildThisFileDirectory)nunit.engine.core.dll">
      <Link>nunit.engine.core.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </None>
    <None Include="$(MSBuildThisFileDirectory)testcentric.engine.metadata.dll">
      <Link>testcentric.engine.metadata.dll</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Visible>False</Visible>
    </None>
  </ItemGroup>
</Project>

The only asset consumed by NuGet is build/net35/NUnit3TestAdapter.props. If the package author used contentFiles instead, they would be listed in the assets file. I suggest you file an issue against the package author to make this package more compatible with SBOM.

Stefan75 commented 1 year ago

Hello @jeffkl ,

I created an issue at the nunit repo: https://github.com/nunit/nunit3-vs-adapter/issues/1052

They are not sure what would be the proposed solution.

I checked other existing solutions:

xunit https://nuget.info/packages/xunit.runner.visualstudio/2.4.5

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<None Include="$(MSBuildThisFileDirectory)xunit.runner.visualstudio.testadapter.dll">
  <Link>xunit.runner.visualstudio.testadapter.dll</Link>
  <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  <Visible>False</Visible>
</None>
<None Include="$(MSBuildThisFileDirectory)xunit.runner.reporters.net452.dll">
  <Link>xunit.runner.reporters.net452.dll</Link>
  <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  <Visible>False</Visible>
</None>
<None Include="$(MSBuildThisFileDirectory)xunit.runner.utility.net452.dll">
  <Link>xunit.runner.utility.net452.dll</Link>
  <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  <Visible>False</Visible>
</None>
<None Include="$(MSBuildThisFileDirectory)xunit.abstractions.dll">
  <Link>xunit.abstractions.dll</Link>
  <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  <Visible>False</Visible>
</None>
<ProjectCapability Include="TestContainer" />
</ItemGroup>
</Project>

and

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
<metadata>
<id>xunit.runner.visualstudio</id>
<version>2.4.5</version>
<title>xUnit.net [Runner: Visual Studio]</title>
<authors>.NET Foundation and Contributors</authors>
<owners></owners>
<developmentDependency>true</developmentDependency>
<requireLicenseAcceptance>false</requireLicenseAcceptance>
<license type="expression">MIT</license>
<icon>logo-512-transparent.png</icon>
<projectUrl>https://github.com/xunit/visualstudio.xunit</projectUrl>
<description>Visual Studio 2019 16.8+ Test Explorer runner for the xUnit.net framework. Capable of running xUnit.net v1.9.2 and v2.0+ tests. Supports .NET 4.7.2 or later, .NET Core 3.1 or later, and Universal Windows 10.0.16299 or later.</description>
<copyright>Copyright (c) .NET Foundation and Contributors.</copyright>
<repository type="git" url="https://github.com/xunit/visualstudio.xunit" commit="1caef2f33e9cda2f6bf93557aedbea4d297ac3b8" />
<dependencies>
  <group targetFramework=".NETFramework4.6.2" />
  <group targetFramework=".NETCoreApp3.1" />
  <group targetFramework="UAP10.0.16299" />
</dependencies>
</metadata>
</package>

mstest https://nuget.info/packages/MSTest.TestAdapter/3.0.2

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<TestAdapterContent Include="$(MSBuildThisFileDirectory)Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.dll">
  <Link>Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.dll</Link>
  <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  <Visible>False</Visible>
</TestAdapterContent>
<TestAdapterContent Include="$(MSBuildThisFileDirectory)Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll">
  <Link>Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll</Link>
  <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  <Visible>False</Visible>
</TestAdapterContent>
<TestAdapterContent Include="$(MSBuildThisFileDirectory)\Microsoft.TestPlatform.AdapterUtilities.dll">
  <Link>Microsoft.TestPlatform.AdapterUtilities.dll</Link>
  <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  <Visible>False</Visible>
</TestAdapterContent>
</ItemGroup>

<ItemGroup>
<!--
  Including `@(TestAdapterContent)` in the `None` ItemGroup to get the `CopyToOutputDirectory` behavior be default,
  package consumers can opt-out of this behavior by removing `@(TestAdapterContent)` from the `None` ItemGroup
  (i.e. `<None Remove="@(TestAdapterContent)" />`)
  -->
<None Include="@(TestAdapterContent)" />
</ItemGroup>
</Project>

and

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
<metadata>
<id>MSTest.TestAdapter</id>
<version>3.0.2</version>
<title>MSTest.TestAdapter</title>
<authors>Microsoft</authors>
<owners>Microsoft</owners>
<requireLicenseAcceptance>true</requireLicenseAcceptance>
<license type="expression">MIT</license>
<icon>Icon.png</icon>
<projectUrl>https://github.com/microsoft/testfx</projectUrl>
<iconUrl>http://go.microsoft.com/fwlink/?LinkID=288859</iconUrl>
<description>The adapter to discover and execute MSTest Framework based tests.

  Supported platforms:
  - .NET 4.6.2+
  - .NET Core 3.1+ (Universal Windows Apps 10+)
  - .NET 6.0
  - .NET 6.0 Windows.18362+
  - ASP.NET Core 1.0+</description>
<copyright>© Microsoft Corporation. All rights reserved.</copyright>
<tags>MSTest TestFramework TestAdapter VisualStudio Unittest MSTestV2 Microsoft</tags>
<repository type="git" url="https://github.com/microsoft/testfx" branch="rel/3.0.2" commit="9c34076eaf2fb204da9cc56d5b9c10d189314f69" />
<dependencies>
  <group targetFramework=".NETStandard2.0" />
  <group targetFramework=".NETFramework4.6.2" />
  <group targetFramework="UAP10.0" />
  <group targetFramework=".NETCoreApp3.1" />
  <group targetFramework="net6.0" />
</dependencies>
</metadata>
</package>

It seems that all test adapter packages are using a different approach than the suggested one from you. (xunit seems to be similar to nunit, msbuild uses a different tag with the name TestAdapterContent - but this seems not to help with the created assets file)

Would you please so kind to elobrate your suggestion a little bit more?

See comment from @OsirisTerje

Not suite sure what @jeffkl is pointing to. The nuspec file contains all the files needed. The props file are just there to cover the different targets for MSBuild, so not sure why they would include that file, and not the Files listed in the nuspec. Content files are normally other non-executable files, (ref https://devblogs.microsoft.com/nuget/nuget-contentfiles-demystified/) so don't think it is wise to include the executables there. Suggest @jeffkl elaborates on this.

Regards, Stefan

jeffkl commented 1 year ago

The short summary is that this package has implemented the copying of files to the output directory in a custom way so NuGet does not report their existence in the assets file since they are not considered content files, reference assemblies, etc.

NuGet / Home