JFrog Nuget Task Fails To Restore Package Due to perceived UTF-8 parsing error

[fourpastmidnight]

Describe the bug

Our existing Azure DevOps Server build pipelines use the built-in Nuget task to restore NuGet packages from our internally hosted Baget server. We are implementing JFrog Artifactory in a test environment.

To this end, I have updated one of our build pipeline to use the JFrog Nuget task to replace the built-in Nuget get task (to restore solution package dependencies).

When using the built-in task, all packages are restored with no issues. However, using the JFrog Nuget task, a warning is encountered: <Package> couldn't be parsed due to: XML syntax error on line 1: invalid UTF-8. Skipping the package dependency. Furthermore, the task completes successfully, which is another problem with this task since, the build is downloading dependencies which, obviously without all dependencies, the build could not possibly be successful. So this type of issue, even if deemed sever enough to continue to emit a warning, should in fact be an error and fail the build.

Current behavior

After replacing the built-in Nuget task in order to perform package restores in preparation for a build, the JFrog Nuget task produces the following warning and completes successfully:

Executing JFrog CLI Command:
D:\Agent01\_work\_tool\_jf\current\jf.exe nuget restore -PackagesDirectory "D:\\Agent01\\_work\\3\\packages\\" -Verbosity "Normal" --build-name="My Application - Test Artifactory Integration" --build-number="My-Application_$_My-Application_Main_App_2023-12-14T12.50.24"
13:02:55 [Info] Running nuget...
All packages listed in packages.config are already installed.
13:03:00 [Warn] Package: microsoft.web.infrastructure:1.0.0 couldn't be parsed due to: XML syntax error on line 1: invalid UTF-8 . Skipping the package dependency.
13:03:01 [Info] nuget finished successfully.

It should be noted that no such warning or error is ever emitted by the built-in NuGet task and my build completes successfully. However, with the JFrog Nuget task, this warning is emitted, skipping the package dependency; and because this step does not outright fail, an attempt to build proceeds to occur which ultimately ends in failure due to the missing dependency.

Reproduction steps

This is probably the result of the fact that the *.nuspec file embedded in the NuGet package does not have a UTF-8 BOM (or, perhaps the other way around--or perhaps an invalid one?).

1. Attempt to produce a NuGet package whose *.nuspec file contains invalid UTF-8 on line 1 (again, probably something related to the BOM)
2. Source the package from Artifactory. In our case, the package lives in a Baget Nuget server which is configured as a remote repository in Artifactory, surfaced via a virtual repository.
3. Consume the dependency via a .NET Framework project
4. Attempt to restore packages using the JFrog Nuget task
5. The task will emit the warning, but otherwise complete "successfully". The Build of the project will fail due to missing dependencies.

Expected behavior

The real world is messy. Bad data. I expect that the JFrog Nuget task would function exactly like the built-in Nuget task and that all project dependencies would be restored.

I further expect that any issue with restoring a dependency (especially one more severe than a UTF-8 issue, such as physically unable to write package files to some location) would result in not just a warning, but an error such that the JFrog Nuget step would fail, resulting in a failed build even before attempting an actual build which would never succeed due to missing dependencies!

Azure DevOps extension name and version

JFrog Azure DevOps Extension

JFrog CLI version

2.50.4

Operating system type and version

Windows Server 2019

JFrog Artifactory version (if relevant)

7.63.7

JFrog Xray version (if relevant)

N/A

JFrog Distribution version (if relevant)

N/A

[fourpastmidnight]

For completeness, I re-downloaded the Microsoft.Web.Infrastructure-1.0.0.nupkg nuget package from Nuget.org and re-uploaded it into our Baget instance from where it was being sourced. Our Artifactory server does not yet have firewall rules in place to allow it to source these packages directly from Nuget.org. That being said however, since I just re-uploaded the package from Nuget.org, in effect, the package sourced from our internal server is identical to the one at Nuget.org, and the JFrog Nuget task is refusing to restore it due to the UTF-8 error originally reported. So this is a fundamental problem with this task.

I guess I'll go back to using the bulit-in Nuget task instead, pointing it to the Artifactory server.

[fourpastmidnight]

From the referenced issue jfrog/jfrog-cli#840, apparently Microsoft messed up the package in some way, most likely the NUSPEC file, such that the NUSPEC file is XML and, usually, the prolog of an XML file specifies enocding="utf-8", but in this case, the actual file encoding is UTF-16 (or, probably more accurately, UCS-2), the default of programs like Windows Notepad. Anyway, JFrog really seems to not like that. Nuget.exe, OTOH, simply ignores this mis-matched encoding.

On the one hand, this is completely invalid. But on the other, at the end of the day, I am not in control of these dependencies, and my project relies on this dependency. So, I expect JFrog tooling to be a "drop-in" replacement for NuGet. So, when things "unexpectedly" don't work as they do for NuGet, I have a problem.

[sverdlov93]

Hi @fourpastmidnight Thanks for reporting this issue. This PR - https://github.com/jfrog/build-info-go/pull/226 should fix it. I will update this thread once the next release will be out.

[sverdlov93]

@fourpastmidnight https://github.com/jfrog/jfrog-azure-devops-extension/releases/tag/2.9.2 is Released! Looking forward to your feedback on it.

[fourpastmidnight]

We updated our extension in Azure DevOps Server, and I'm still receiving this error.