Nuget restore on .NET Core project does not publish all the dependencies to build info

[jigar61]

I am trying to use the jfrog cli to collect Nuget dependencies for a .NET core project using a virtual repository in Artifactory. During the restore, I see that some of the packages are skipped with the error "[Warn] The file C:\Users***.nuget\packages\microsoft.aspnetcore.http.connections.common\1.0.4\microsoft.aspnetcore.http.connections.common.1.0.4.nupkg doesn't exist in the NuGet cache directory but it does exist as a target in the assets files. Skipping adding this file to the build info.". Because of this, the number of dependencies that I see on the build in artifactory does not seem to match the actual number of dependencies on the project. Any way to get around this? I tried running the nuget restore with "-NoCache" and "-Force" option and tried clearing all the local caches before running the restore command but to no avail.

[eyalbe4]

Thanks for raising this question @jigar61.

When the jfrog rt nuget restore command is executed, JFrog CLI first executes the restore command using the nuget client, and then starts collecting the list of packages (build dependencies), which were resolved (downloaded from Artifactory or cached locally) by nuget. This list of dependencies is then added to the build-info. JFrog CLI supports both .net projects with use the project.assets.json file (like in this case) or packages.config. The project.assets.json includes the details of the dependencies used by the project. Some dependencies however may be "core packages" which were not downloaded from Artifactory by the nuget client, but are included as part of .net framework or operating system. In this case, the package is microsoft.aspnetcore.http.connections.common. This package file therefore is located in a different location on the file system, and therefore, is not picked up by JFrog CLI, as the message in logs states. This behaviour is expected, since this package comes with the framework, and is not an external dependency that the build uses. We may need to change this warning log message into an info message.

Please let me know what you think.

[liron-shalom]

@jigar61, Are you still facing the issue of the missing dependencies? Which NuGet resolving method are you using?

We tried to reproduce this issue with several .NET projects, but unfortunately we didn't receive these warning messages. Is it possible for you to provide us some minimal example project (or even just packages.config or .csproj file) to reproduce the issue?

[jigar61]

Yup we still are seeing those missing dependencies. As @eyalbe4 said above, we are accepting those missed dependencies as "core packages" that don't need to be included in the build info.

[liron-shalom]

@jigar61 Can you please list some of the full names and versions of these dependencies? Are you using packages.config or the reference method?

[jigar61]

We are using the jfrog cli nuget restore command on both .NET Framework and .NET Core projects. I am attaching a sample csproj file that produces the missing dependencies. Some examples of those are below: microsoft.aspnetcore.razor.design\2.1.2\microsoft.aspnetcore.razor.design.2.1.2.nupkg microsoft.aspnetcore.session\2.1.1\microsoft.aspnetcore.session.2.1.1.nupkg microsoft.identitymodel.clients.activedirectory\3.14.2\microsoft.identitymodel.clients.activedirectory.3.14.2.nupkg runtime.win-arm64.runtime.native.system.data.sqlclient.sni\4.4.0\runtime.win-arm64.runtime.native.system.data.sqlclient.sni.4.4.0.nupkg system.diagnostics.tools\4.3.0\system.diagnostics.tools.4.3.0.nupkg system.reflection.extensions\4.3.0\system.reflection.extensions.4.3.0.nupkg microsoft.aspnetcore.http\2.1.1\microsoft.aspnetcore.http.2.1.1.nupkg runtime.fedora.24-x64.runtime.native.system.security.cryptography.openssl\4.3.0\runtime.fedora.24-x64.runtime.native.system.security.cryptography.openssl.4.3.0.nupkg runtime.native.system.security.cryptography.openssl\4.3.0\runtime.native.system.security.cryptography.openssl.4.3.0.nupkg system.composition.runtime\1.0.31\system.composition.runtime.1.0.31.nupkg system.diagnostics.process\4.3.0\system.diagnostics.process.4.3.0.nupkg system.security.cryptography.pkcs\4.5.0\system.security.cryptography.pkcs.4.5.0.nupkg messagepack\1.7.3.4\messagepack.1.7.3.4.nupkg microsoft.extensions.hosting\2.1.1\microsoft.extensions.hosting.2.1.1.nupkg microsoft.extensions.logging.abstractions\2.1.1\microsoft.extensions.logging.abstractions.2.1.1.nupkg system.diagnostics.fileversioninfo\4.3.0\system.diagnostics.fileversioninfo.4.3.0.nupkg system.reflection.emit.lightweight\4.3.0\system.reflection.emit.lightweight.4.3.0.nupkg microsoft.extensions.fileproviders.embedded\2.1.1\microsoft.extensions.fileproviders.embedded.2.1.1.nupkg microsoft.extensions.localization.abstractions\2.1.1\microsoft.extensions.localization.abstractions.2.1.1.nupkg microsoft.extensions.logging.eventsource\2.1.1\microsoft.extensions.logging.eventsource.2.1.1.nupkg system.io\4.3.0\system.io.4.3.0.nupkg microsoft.aspnetcore.applicationinsights.hostingstartup\2.1.1\microsoft.aspnetcore.applicationinsights.hostingstartup.2.1.1.nupkg stackexchange.redis.strongname\1.2.6\stackexchange.redis.strongname.1.2.6.nupkg system.net.sockets\4.3.0\system.net.sockets.4.3.0.nupkg microsoft.aspnetcore.diagnostics.entityframeworkcore\2.1.1\microsoft.aspnetcore.diagnostics.entityframeworkcore.2.1.1.nupkg microsoft.aspnetcore.staticfiles\2.1.1\microsoft.aspnetcore.staticfiles.2.1.1.nupkg microsoft.extensions.diagnosticadapter\2.1.0\microsoft.extensions.diagnosticadapter.2.1.0.nupkg system.collections.specialized\4.3.0\system.collections.specialized.4.3.0.nupkg system.reflection.primitives\4.3.0\system.reflection.primitives.4.3.0.nupkg microsoft.aspnetcore.dataprotection.azurestorage\2.1.1\microsoft.aspnetcore.dataprotection.azurestorage.2.1.1.nupkg system.collections.immutable\1.5.0\system.collections.immutable.1.5.0.nupkg microsoft.extensions.configuration\2.1.1\microsoft.extensions.configuration.2.1.1.nupkg sqlitepclraw.provider.e_sqlite3.netstandard11\1.1.11\sqlitepclraw.provider.e_sqlite3.netstandard11.1.1.11.nupkg system.identitymodel.tokens.jwt\5.2.0\system.identitymodel.tokens.jwt.5.2.0.nupkg microsoft.aspnetcore.authentication.core\2.1.1\microsoft.aspnetcore.authentication.core.2.1.1.nupkg microsoft.extensions.configuration.commandline\2.1.1\microsoft.extensions.configuration.commandline.2.1.1.nupkg system.reflection\4.3.0\system.reflection.4.3.0.nupkg microsoft.aspnetcore.diagnostics\2.1.1\microsoft.aspnetcore.diagnostics.2.1.1.nupkg runtime.native.system.data.sqlclient.sni\4.4.0\runtime.native.system.data.sqlclient.sni.4.4.0.nupkg system.collections.concurrent\4.3.0\system.collections.concurrent.4.3.0.nupkg system.composition.attributedmodel\1.0.31\system.composition.attributedmodel.1.0.31.nupkg microsoft.aspnetcore.localization.routing\2.1.1\microsoft.aspnetcore.localization.routing.2.1.1.nupkg microsoft.codeanalysis.csharp.workspaces\2.8.0\microsoft.codeanalysis.csharp.workspaces.2.8.0.nupkg microsoft.extensions.configuration.ini\2.1.1\microsoft.extensions.configuration.ini.2.1.1.nupkg microsoft.extensions.fileproviders.composite\2.1.1\microsoft.extensions.fileproviders.composite.2.1.1.nupkg runtime.ubuntu.16.10-x64.runtime.native.system.security.cryptography.openssl\4.3.0\runtime.ubuntu.16.10-x64.runtime.native.system.security.cryptography.openssl.4.3.0.nupkg microsoft.aspnetcore.authentication.abstractions\2.1.1\microsoft.aspnetcore.authentication.abstractions.2.1.1.nupkg microsoft.aspnetcore.localization\2.1.1\microsoft.aspnetcore.localization.2.1.1.nupkg sqlitepclraw.lib.e_sqlite3.linux\1.1.11\sqlitepclraw.lib.e_sqlite3.linux.1.1.11.nupkg microsoft.netcore.dotnethostresolver\2.1.0\microsoft.netcore.dotnethostresolver.2.1.0.nupkg system.io.filesystem\4.3.0\system.io.filesystem.4.3.0.nupkg microsoft.extensions.configuration.xml\2.1.1\microsoft.extensions.configuration.xml.2.1.1.nupkg microsoft.identitymodel.tokens.saml\5.2.0\microsoft.identitymodel.tokens.saml.5.2.0.nupkg remotion.linq\2.2.0\remotion.linq.2.2.0.nupkg system.diagnostics.stacktrace\4.3.0\system.diagnostics.stacktrace.4.3.0.nupkg system.threading.timer\4.3.0\system.threading.timer.4.3.0.nupkg microsoft.aspnetcore.websockets\2.1.1\microsoft.aspnetcore.websockets.2.1.1.nupkg microsoft.extensions.configuration.json\2.1.1\microsoft.extensions.configuration.json.2.1.1.nupkg runtime.ubuntu.14.04-x64.runtime.native.system.security.cryptography.openssl\4.3.0\runtime.ubuntu.14.04-x64.runtime.native.system.security.cryptography.openssl.4.3.0.nupkg microsoft.aspnetcore.responsecaching\2.1.1\microsoft.aspnetcore.responsecaching.2.1.1.nupkg microsoft.extensions.logging\2.1.1\microsoft.extensions.logging.2.1.1.nupkg microsoft.rest.clientruntime\2.3.8\microsoft.rest.clientruntime.2.3.8.nupkg runtime.debian.8-x64.runtime.native.system.security.cryptography.openssl\4.3.0\runtime.debian.8-x64.runtime.native.system.security.cryptography.openssl.4.3.0.nupkg system.componentmodel.annotations\4.5.0\system.componentmodel.annotations.4.5.0.nupkg system.reflection.emit.ilgeneration\4.3.0\system.reflection.emit.ilgeneration.4.3.0.nupkg microsoft.aspnetcore.antiforgery\2.1.1\microsoft.aspnetcore.antiforgery.2.1.1.nupkg sqlitepclraw.lib.e_sqlite3.osx\1.1.11\sqlitepclraw.lib.e_sqlite3.osx.1.1.11.nupkg system.text.encodings.web\4.5.0\system.text.encodings.web.4.5.0.nupkg microsoft.aspnetcore.diagnostics.abstractions\2.1.1\microsoft.aspnetcore.diagnostics.abstractions.2.1.1.nupkg microsoft.visualstudio.web.browserlink\2.1.1\microsoft.visualstudio.web.browserlink.2.1.1.nupkg newtonsoft.json.bson\1.0.1\newtonsoft.json.bson.1.0.1.nupkg system.composition.typedparts\1.0.31\system.composition.typedparts.1.0.31.nupkg system.diagnostics.contracts\4.3.0\system.diagnostics.contracts.4.3.0.nupkg system.runtime\4.3.0\system.runtime.4.3.0.nupkg system.threading\4.3.0\system.threading.4.3.0.nupkg system.threading.tasks\4.3.0\system.threading.tasks.4.3.0.nupkg system.security.accesscontrol\4.5.0\system.security.accesscontrol.4.5.0.nupkg microsoft.aspnetcore.responsecaching.abstractions\2.1.1\microsoft.aspnetcore.responsecaching.abstractions.2.1.1.nupkg system.io.pipelines\4.5.2\system.io.pipelines.4.5.2.nupkg system.linq.parallel\4.3.0\system.linq.parallel.4.3.0.nupkg system.text.regularexpressions\4.3.0\system.text.regularexpressions.4.3.0.nupkg microsoft.aspnetcore.hosting\2.1.1\microsoft.aspnetcore.hosting.2.1.1.nupkg microsoft.aspnetcore.http.extensions\2.1.1\microsoft.aspnetcore.http.extensions.2.1.1.nupkg microsoft.extensions.fileproviders.abstractions\2.1.1\microsoft.extensions.fileproviders.abstractions.2.1.1.nupkg system.security.cryptography.primitives\4.3.0\system.security.cryptography.primitives.4.3.0.nupkg system.security.cryptography.xml\4.5.0\system.security.cryptography.xml.4.5.0.nupkg My.csproj.txt

[liron-shalom]

@jigar61, I was able to restore the the project defined in the attached .csproj file, and my build info includes all the dependencies listed in your comment above.

It is documented in our source code that those dependencies might be located in NuGetFallbackFolder. As described here, NuGetFallbackFolder folder "contains a cache of NuGet packages used by an SDK during the restore operation...This folder is only used prior to .NET Core 3.0."

I checked .NET Core SDK version: $ dotnet --version 3.1.201

After downgrading $ dotnet --version 2.2.207 I started to see the warning messages, and my build info is now missing 142 dependencies.

Can you please tell us the .NET Core SDK version in use?

[jigar61]

The .NET Core SDK version in use is 3.1.201 for us as well. However we are using jfrog rt nuget restore to restore the project which I assume uses msbuild and not the dotnet cli.

[voneinem]

We have the same issue here during JFrog/XRay evaluation. I tried 'jf dotnet restore' and 'jf nuget restore' - tried in various combinations including nuget locals all -clear. Any plan to resolve this issue from JFrog?

[eyalbe4]

@voneinem - Will you be able to share more information about the issue? What exactly is the challenge you're facing?