Libraries could not be resolved by the "cdnjs" provider

[duykhiem]

Describe the bug

We are having an issue with the "cdnjs" provider when building our project in both Hanoi, Vietnam, and Grimstad, Norway. It is happening with both local build and Azure DevOps Pipeline build. Failed on some random libraries like jquery@3.4.1, tabulator@4.4.1, tempusdominus-bootstrap-4@5.1.2...

Error:

Restore operation started
##[error]libman.json(0,0): Error LIB002: The "jquery@3.4.1" library could not be resolved by the "cdnjs" provider
     1>libman.json : error LIB002: The "jquery@3.4.1" library could not be resolved by the "cdnjs" provider 
         One or more libraries failed to restore

Build FAILED.

       (LibraryManagerRestore target) -> 
         **libman.json : error LIB002: The "jquery@3.4.1" library could not be resolved by the "cdnjs" provider** 

To Reproduce

Steps to reproduce the behavior:

1. Using this version of Visual Studio 2022
2. Build the solution
3. See error:

Expected behavior

Build and restore client libraries successfully

Screenshots

Additional context

• We are using cdnjs as the default provider in LibMan
• It was working perfectly before but suddenly failed today without any changes related to Libman or package references

LibMan CLI version: 2.1.161+abc97ecc7d.RR Visual Studio 2022 Microsoft.Web.LibraryManager.Build" version="2.1.161" targetFramework="net472"

[duykhiem]

The workaround is to clear Libman cache https://github.com/cdnjs/cdnjs/issues/14141#issuecomment-1160220546

[stebueh]

possibly related to cdnjs API change, see https://github.com/cdnjs/cdnjs/issues/14140; it's being rolled out partially, which would explain the intermittent failure

[duykhiem]

The workaround is to clear Libman cache cdnjs/cdnjs#14141 (comment)

Ah. Cleaning cache is just a lucky shot

I think the main issue is related to https://github.com/cdnjs/cdnjs/issues/14140

Libman still uses the old endpoint: https://github.com/aspnet/LibraryManager/blob/8dd0ac5fdfe76c2d37ded284834c67633cc18d7f/src/LibraryManager/Providers/Cdnjs/CdnjsCatalog.cs#L23

Failed restore/install fall into their 10% production request https://github.com/cdnjs/cdnjs/issues/14140#issuecomment-1159470071

I think clean cache somehow and sometime helps our request to fall outside of above 10% :D

[duykhiem]

possibly related to cdnjs API change, see cdnjs/cdnjs#14140; it's being rolled out partially, which would explain the intermittent failure

yes I think so

[MattIPv4]

Hey folks, just wanted to note that after a quick read through the code here, it does seem like this is caused by the deprecation of the assets property returning every version on cdnjs.

The fix for libman here would be to use the versions property to determine the library versions, not assets. And then to use the api.cdnjs.com/libraries/:library/:version endpoint when it needs to actually get the assets for a specific version.

[jthomas81]

Does someone have an example of syntax to do this? This is what I have been doing: { "version": "1.0", "defaultProvider": "cdnjs", "libraries": [ { "library": "jquery-validate@1.17.0", "destination": "wwwroot/lib/jquery-validate/" } ] }

[the-doctor1994]

Does someone have an example of syntax to do this? This is what I have been doing:

it's not a change for the end user, MattIPv4 is suggesting LibMan issue an update to their code, something that could take days, to conform to the altered API behavior.... instead of just reverting the cdnjs api change that is causing this issue

[MattIPv4]

Unfortunately, LibMan is not the only consumer of the cdnjs API, and the deprecation that caused this issue is part of larger ongoing work across the API to improve stability (the cdnjs API often runs 5xx errors for assorted consumers, not just LibMan users). We're not really in a position where we can revert this API change as doing so will only increase the failure rate for the API generally.

[the-doctor1994]

I understand the change is necessary and inevitable, but time could and should be given to allow for proper deprecation updates by all the users of the API who may be negatively affected (or outright broken) by this change. By your own proposal the deprecation was fast-tracked under the incorrect assumption nobody would be affected...

[MattIPv4]

I appreciate what you're saying, and this is why we're approaching this as a slow rollout with only 10% of traffic reaching endpoints that implement this deprecation, so that folks are made aware of if they're impacted by the deprecation (turns out it's pretty impossible to determine who consumes what properties of a public API just looking at unauthenticated GET requests), but can also still easily get around the deprecation by just re-requesting (falling into the 90% of traffic that is unaffected at present).

Until we hear more here in terms of what a fix would look like on LibMan's end, we don't intend to further rollout the change, so it will remain at 10% for now. That being said, ultimately we will have to continue with this deprecation as until we complete this rollout a large number of requests to the API from all consumers, not just LibMan users, will continue to fail unexpectedly.

[jimmylewis]

Just quickly jumping in here to say: I'm working on the code change for LibMan, but the rollout will require servicing each version of Visual Studio that includes it (2017, 2019, 2022). There will be some delays to prepare the LibMan builds and VS servicing changes in each product, and then ultimately it will require customers to install the updates as well.

I don't have timelines yet, I'm focusing first on making the code change for VS2022 and our CLI tool, then I will start investigating the servicing requirements for each VS release.

[duykhiem]

I think the safest and fastest way to workaround this issue is to include the successfully generated static files by LibMan to our project (when our restore client-side libraries command falls into the 90% of traffic that is unaffected at present) and then disable LibMan. We can re-enable it when the issue is fixed.

Another workaround is try to use other CDN providers like filesystem, jsdelivr, unpkg instead of cdnjs for LibMan

Cleaning LibMan cache is not really a workaround for this issue. Our build will still have a failure probability of 10% even though we tried to clear the LibMan cache

[gavin-theta]

I'm not sure if this will help but in our case we had a "dotnet restore" followed by a "dotnet publish" the publish was restoring the packages again. So I disable that by "dotnet publish --no-restore" and the build is now happy.

[ennerperez]

Also, Azure Pipelines are failing too.

[gavin-theta]

Also, Azure Pipelines are failing too.

Try my fix above. The azure restore seems to work as a different step. Then disable the restore in the publish. Worked for me.

[jimmylewis]

Alright, for folks using the CLI tool or MSBuild task could you please update to the latest version (2.1.174) and see if you have any issues with that? From a few projects I've tried, it's working fine. This will be the fastest way to workaround the issue (at least for unblocking CI or local builds) as the VS servicing cycle will take a longer to sort out.

Timeline on VS updates still TBD...

[calvaradocl]

On our azure pipelines we are getting this error after updating the package to the 2.1.174 version, this persists even if we have Newtonsoft.Json installed.

/root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: The "Microsoft.Web.LibraryManager.Build.RestoreTask" task failed unexpectedly. [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: System.TypeInitializationException: The type initializer for 'Microsoft.Web.LibraryManager.Cache.WebRequestHandler' threw an exception. ---> System.TypeInitializationException: The type initializer for 'Microsoft.Web.LibraryManager.Configuration.Settings' threw an exception. ---> System.IO.FileLoadException: Could not load file or assembly 'Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'. Could not find or load a specific file. (Exception from HRESULT: 0x80131621) [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Web.LibraryManager.Configuration.Settings..ctor() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Web.LibraryManager.Configuration.Settings..cctor() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: --- End of inner exception stack trace --- [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Web.LibraryManager.Configuration.ProxySettings.get_Default() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Web.LibraryManager.Cache.WebRequestHandler..cctor() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: --- End of inner exception stack trace --- [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Web.LibraryManager.Build.Contracts.Dependencies.Initialize() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Web.LibraryManager.Build.RestoreTask.Execute() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [/src/Frontend/Frontend.csproj] /root/.nuget/packages/microsoft.web.librarymanager.build/2.1.174/build/Microsoft.Web.LibraryManager.Build.targets(35,9): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [/src/Frontend/Frontend.csproj] 2 Warning(s) 1 Error(s)

[stebueh]

updating the MSBuild task fixed the issue for me building locally, thank you @jimmylewis for the timly fix; our azure devops pipeline is using the CLI tool and was running flawlessly all the time, so I guess it is/was still on the 90% of traffic unaffected by the API change.

[ardalis]

I appreciate what you're saying, and this is why we're approaching this as a slow rollout with only 10% of traffic reaching endpoints that implement this deprecation, so that folks are made aware of if they're impacted by the deprecation (turns out it's pretty impossible to determine who consumes what properties of a public API just looking at unauthenticated GET requests), but can also still easily get around the deprecation by just re-requesting (falling into the 90% of traffic that is unaffected at present).

Running into Hyrum's Law:

With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody.

[farhadmammadli]

cdnjs issue is resolved in the new update of the LibraryManager package. the updated version is 2.1.174

[thiago-bessa]

I just tried the new version, and I'm still having problems:

C:\Users\thiago.bessa>dotnet tool install -g Microsoft.Web.LibraryManager.Cli
You can invoke the tool using the following command: libman
Tool 'microsoft.web.librarymanager.cli' (version '2.1.174') was successfully installed.

libman restore
[LIB002]: The "jquery@2.2.4" library could not be resolved by the "cdnjs" provider
[LIB002]: The "jquery-validate@1.19.1" library could not be resolved by the "cdnjs" provider

[jimmylewis]

@thiago-bessa try again after running libman cache clean? We use the cache library metadata to disk for 1 day (IIRC of the top of my head) to avoid hitting the service repeatedly on queries, so if you have recently cached metadata from the older build, it might be conflicting with the changes.

I was able to restore fine with this libman.json file

{
  "version": "1.0",
  "defaultProvider": "cdnjs",
  "libraries": [
    {
      "library": "jquery@2.2.4",
      "destination": "lib\\jquery"
    },
    {
      "library": "jquery-validate@1.19.1",
      "destination": "lib\\jquery-validate"
    }
  ]
}

[farhadmammadli]

I just tried the new version, and I'm still having problems:

C:\Users\thiago.bessa>dotnet tool install -g Microsoft.Web.LibraryManager.Cli
You can invoke the tool using the following command: libman
Tool 'microsoft.web.librarymanager.cli' (version '2.1.174') was successfully installed.

libman restore
[LIB002]: The "jquery@2.2.4" library could not be resolved by the "cdnjs" provider
[LIB002]: The "jquery-validate@1.19.1" library could not be resolved by the "cdnjs" provider

have you tried to clean cache using this? libman cache clean

[thiago-bessa]

Sorry @farhadmammadli and @jimmylewis, I didn't copy/paste everything.

I did run libman cache clean before running libman restore, and it still failed.

Now I ran it an hour later (same cmd prompt, with the same project), and it could restore everything.

Then I ran it in other project of ours (after the first one restored correctly), and it again failed to resolve:

Then I just ran restore for 3 times in a row, it finally worked. It seems kind of intermittent.

[alex-jitbit]

I have the same issue, and clearing cache helps. Which probably means the issue is inside libman, not cdnjs?

[Cloudfare123]

Our Web solutions all fixed now locally and CI by updating to latest package. 👍

[easlavov]

On our azure pipelines we are getting this error after updating the package to the 2.1.174 version, this persists even if we have Newtonsoft.Json installed.

@calvaradocl, we're having the same issue after upgrading to 2.1.174. Did you manage to fix it?

[calvaradocl]

@easlavov, no, we still have the same issue even after multiple runs.

We had to copy our dependencies manually into a folder and use gulp, for now, to move them to the www-root folder.

[Smaiil]

Our Web solutions all fixed now locally and CI by updating to latest package. 👍

thank you!

[nimor109]

We have the same issues in production in our ci/cd pipelines. We also have to manually manage a cdnjs "feed" (shared folder) which is really bad. According to their thread this was done on purpose and won't be fixed. We'll see if our teams can use the NPM feed instead for their needs

[MattIPv4]

Fwiw, for folks still running into cdnjs issues after updating, you're likely running into the general API stability issues that this change was part of the fix for. Until we can roll out this change to 100%, folks on any libman version will be hitting random API issues irrespective of the version (though those not on the fixed release will also be guaranteed 10% failure rates at the moment due to the initial rollout of the API fix)

[WyrmUK]

The 2.1.175 release fixes a build (when you build the solution or rebuild all) however in Visual Studio 2022 Professional if you right click on the libman.json file and select Restore Client-Side Libraries it can't find any libraries:

The "jquery@3.6.0" library could not be resolved by the "cdnjs" provider

[jimmylewis]

Unfortunately, the process for shipping updates in VS is slower than publishing the NuGet packages. The same fixes will be included in the next servicing releases for VS2019 and VS2022, as well as the next VS2022 preview.

[WyrmUK]

However I have another project that is a NetCoreApp3.1 based web app where even 2.1.175 doesn't fix the issue. It won't list anything other than 6.1.1 for font-awesome and all libraries state something similar to:

The "jquery@3.6.0" library could not be resolved by the "cdnjs" provider

This is basically killing the use of this. I know several large users who have decided to cut their losses and stop using Libman and CD systems.

[jimmylewis]

It won't list anything other than 6.1.1 for font-awesome

@WyrmUK can you provide the steps you're using to list the versions or share your libman.json file? Installing a newer NuGet package won't impact the experience in VS (Add New Client-side Library wizard or install on save in libman.json), those will need to wait for the VS update. However, the CLI and build tasks should be agnostic to the cdnjs API change (i.e. should work with responses with or without the change), as we stopped relying on the assets property entirely.

[WyrmUK]

It appears that was a regional API thing presumably due to CDNJS. When on a VPN: editing the libman.json file wouldn't retrieve anything other than 6.1.1 for font-awesome and all libraries were unable to be resolved when building. When not on the VPN all was fine. This wasn't a 10% thing as the build was attempted many times. CDNJS presumably have fixed things in their API now as on the VPN it is all fine today. Whoever thought it was a good idea to a) remove functionality at a moments notice and b) to implement changes to only some servers needs to be removed from the project - who knows what bad decisions they will make in the future.

[xtuc]

@WyrmUK b) we did not implement changes only on some servers.

[WyrmUK]

Rather than spend time issuing denials, perhaps you should spend more time making good decisions in the first place? The confidence in CDNJS has been significantly dented because of all this.

[MattIPv4]

As I've explained further up in this thread, this API change was made as part of work to stabilise the API as many consumers (LibMan users, as well as others) had been running into random errors and failures with the API. This change was part of work to solve for that, and so had to be made.

Given the scale of the cdnjs API and it being unauthenticated, there is no easy way for us to know who relies on what properties of the API responses, nor an easy touch-point to communicate changes with consumers, hence we chose the slow rollout approach to allow folks time to notice that they may have been relying on the deprecated property and get things updated as such.

As soon as we found out that LibMan was reliant on this property we stopped any further rollout and left the change at 10% to prompt folks to update as soon as possible. I believe the final LibMan patch should be going out today, so once that it out we will continue the rollout once again to finally stabilise the issues with the cdnjs API :)

[MattIPv4]

(Also as Sven noted, this change wasn't regional nor have we changed anything since the initial 10% rollout -- it is a globally distributed Worker that is serving 10% of requests randomly.)

[WyrmUK]

However this is about libman and not CDNJS so whilst that change could have been handled so much better - I'll leave things there and report back on libman when the fix has rolled out.

[jimmylewis]

The changes to Visual Studio are now available in the 16.9.23, 16.11.17, 17.0.12, 17.2.6, and 17.3 Preview 3 releases. Please let me know if you are still having issues after updating to one of these versions. In my testing with each of them, I was able to install the latest and non-latest versions of multiple packages and did not encounter any issues.

[MattIPv4]

Thank you for all the help in getting this resolved @jimmylewis 🧡

We'll leave things as-is over the weekend, and then I suspect we'll start increasing the rollout percentage for the API changes moving into next week, playing it by ear to ensure these updates are working for folks.

[PromoFaux]

LGTM, thanks for the fix!

[WyrmUK]

The update seems to have addressed the remaining issues.

I would advise CDNJS to leave things a little longer than a few days before increasing the rollout to give people time to update both Visual Studio and their various projects. Commercial developments on a release phase often take a couple of weeks to complete during which time they will have frozen the codebase and won't look kindly on anything that prevents them building the final release at the end of the phase.

[MattIPv4]

won't look kindly on anything that prevents them building the final release at the end of the phase

I hear your point, but the API is currently rather unstable as-is with random failures happening, and the 10% rollout remains in place, so really the API is already broken from their perspective. Rolling out to 100% feels like the best bet here so that folks are no longer tracking down an inconsistent, but rather a consistent one that's easier to spot and then update to fix.

[JosephAmalfitanoSSA]

any ETA?

[MattIPv4]

@JosephAmalfitanoSSA The releases with the fix were made available mid July: https://github.com/aspnet/LibraryManager/issues/685#issuecomment-1185272933

[JosephAmalfitanoSSA]

@MattIPv4 I noticed that, but i see the ticket is still marked open. I assumed because it wasn't working again. I'm still running into.

I've tried:

• updating VS to latest stable
• updating VS to latest preview build (17.4.0 preview 1.0)
• manually installing: dotnet tool install -g Microsoft.Web.LibraryManager.Cli and running libman cache clean & libman install
• maybe something else i can't remember

All results are the same X library could not be resolved by the "cdnjs" provider.

[WyrmUK]

Update VS to latest stable AND update the Library Manager (Microsoft.Web.LibraryManager.Build) to the latest in the project.