CycloneDX / cyclonedx-dotnet

Creates CycloneDX Software Bill of Materials (SBOM) from .NET Projects
https://cyclonedx.org/
Apache License 2.0
167 stars 78 forks source link

CycloneDx 2.5.1 throwing notFound Error #606

Open gauti11 opened 1 year ago

gauti11 commented 1 year ago

Since the Cyclonedx has been updated to version 2.5.1. I have started to get this error. Its still working fine if I go with 2.3.0 version. Do you know what can cause this error. I am doing the BOM creation on azure pipelines

Retrieving GitHub license for repository StackExchange/StackExchange.Redis and ref master GitHub API failed with status code NotFound and message Not Found. No license found on GitHub for repository StackExchange/StackExchange.Redis using ref master Retrieving GitHub license for repository dotnet/corefx and ref master GitHub API failed with status code NotFound and message Not Found. No license found on GitHub for repository dotnet/corefx using ref master Retrieving GitHub license for repository dotnet/standard and ref master GitHub API failed with status code NotFound and message Not Found. No license found on GitHub for repository dotnet/standard using ref master Retrieving GitHub license for repository dotnet/corefx and ref master GitHub API failed with status code NotFound and message Not Found. No license found on GitHub for repository dotnet/corefx using ref master Retrieving GitHub license for repository dotnet/corefx and ref master GitHub API failed with status code NotFound and message Not Found. No license found on GitHub for repository dotnet/corefx using ref master Retrieving GitHub license for repository dotnet/corefx and ref master GitHub API failed with status code NotFound and message Not Found. No license found on GitHub for repository dotnet/corefx using ref master Retrieving GitHub license for repository dotnet/corefx and ref master GitHub API failed with status code NotFound and message Not Found. No license found on GitHub for repository dotnet/corefx using ref master Retrieving GitHub license for repository dotnet/corefx and ref master GitHub API failed with status code NotFound and message Not Found. No license found on GitHub for repository dotnet/corefx using ref master Retrieving GitHub license for repository dotnet/corefx and ref master GitHub API failed with status code NotFound and message Not Found. No license found on GitHub for repository dotnet/corefx using ref master Retrieving GitHub license for repository dotnet/corefx and ref master GitHub API failed with status code NotFound and message Not Found. No license found on GitHub for repository dotnet/corefx using ref master Retrieving GitHub license for repository xunit/xunit and ref master Retrieving GitHub license for repository xunit/xunit and ref master Retrieving GitHub license for repository xunit/xunit.analyzers and ref master Retrieving GitHub license for repository xunit/xunit and ref master Retrieving GitHub license for repository xunit/xunit and ref master

mthomas-vix commented 1 year ago

We have also started experiencing this error. I was previously running version 2.3.0, which would successfully generate a BOM. However after upgrading to v2.5.1 and running the tool on the exact same solution, we get the following errors logged out:

Retrieving GitHub license for repository confluentinc/confluent-kafka-dotnet and ref master  
Retrieving GitHub license for repository moq/moq4 and ref main  
Retrieving GitHub license for repository dotnet/corefx and ref master  
GitHub API failed with status code NotFound and message Not Found.  
No license found on GitHub for repository dotnet/corefx using ref master  
Retrieving GitHub license for repository dotnet/corefx and ref master  
GitHub API failed with status code NotFound and message Not Found.  
No license found on GitHub for repository dotnet/corefx using ref master  
Retrieving GitHub license for repository AutoMapper/AutoMapper and ref master  
Retrieving GitHub license for repository danm-de/Fractions and ref master  
Retrieving GitHub license for repository Microsoft/dotnet and ref master  
GitHub API failed with status code NotFound and message Not Found.  
No license found on GitHub for repository Microsoft/dotnet using ref master  
Retrieving GitHub license for repository dotnet/corefx and ref master  
GitHub API failed with status code NotFound and message Not Found.  
No license found on GitHub for repository dotnet/corefx using ref master  
Retrieving GitHub license for repository dotnet/corefx and ref master  
GitHub API failed with status code NotFound and message Not Found.  
No license found on GitHub for repository dotnet/corefx using ref master  
Retrieving GitHub license for repository dotnet/corefx and ref master  
GitHub API failed with status code NotFound and message Not Found.  
No license found on GitHub for repository dotnet/corefx using ref master  
Retrieving GitHub license for repository xunit/xunit and ref master  
Retrieving GitHub license for repository gasparnagy/BoDi and ref master  
Retrieving GitHub license for repository dotnet/corefx and ref master  
GitHub API failed with status code NotFound and message Not Found.  
No license found on GitHub for repository dotnet/corefx using ref master  
Retrieving GitHub license for repository dotnet/corefx and ref master  
GitHub API failed with status code NotFound and message Not Found.  
No license found on GitHub for repository dotnet/corefx using ref master  
Retrieving GitHub license for repository dotnet/corefx and ref master  
GitHub API failed with status code NotFound and message Not Found.  
No license found on GitHub for repository dotnet/corefx using ref master  
Retrieving GitHub license for repository dotnet/corefx and ref master  
GitHub API failed with status code NotFound and message Not Found.  
No license found on GitHub for repository dotnet/corefx using ref master  
Retrieving GitHub license for repository dotnet/corefx and ref master  
GitHub API failed with status code NotFound and message Not Found.  
No license found on GitHub for repository dotnet/corefx using ref master  
Retrieving GitHub license for repository dotnet/corefx and ref master  
GitHub API failed with status code NotFound and message Not Found.  
No license found on GitHub for repository dotnet/corefx using ref master  
Unable to locate valid bom ref for Microsoft.EntityFrameworkCore.Design [6.0.7, )  

UPDATE: I am also getting this issue when running 2.4.1.0, so that must have been the version to introduce the issue.

gauti11 commented 1 year ago

I also tried with version 2.4.1.0, its giving out the same issue, were you able to find what's the issue?

rkg-mm commented 1 year ago

Running into the same issue: Unable to locate valid bom ref for Microsoft.Extensions.Configuration.Json [2.1.0, )

Anyone found a solution?

edit: downgrading to 2.3.0 works btw

Apholisha commented 1 year ago

I'm having the same issue.

SchneiderMikeTriplett commented 1 year ago

We are having the same issue with v2.7.0.0. In our case, it is a project ref to another project in the same solution, so does not include a version number. The other project is at 'ver#-beta04' and the error msg indicates it is looking for 'ver#-beta01'

Unable to locate valid bom ref for (redacted project name) [1.4.0-beta01, )

shawner18 commented 1 year ago

We are having the same issue when referencing another project within the same solution.

Debugged my issue and was able to resolve it. I added a PR with more context that might help you uncover your issue https://github.com/CycloneDX/cyclonedx-dotnet/pull/662

adcorduneanu commented 1 year ago

Any fix for this one?

rkg-mm commented 9 months ago

Also still looking for a fix. New version contains more information in the output: Unable to locate valid bom ref for NETStandard.Library [2.0.0, ) referenced by (Name: MQTTnet Version: 3.0.8)

Does that somehow help? Any idea what causes this?

raphaelquati commented 7 months ago

I've the same problem here using the latest version (2.9.0):

Unable to locate valid bom ref for rebus [6.0.0, 7.0.0) referenced by (Name: Rebus.RabbitMq Version: 7.4.3)

Checking Rebus.RabbitMq on nuget site (https://www.nuget.org/packages/Rebus.RabbitMq/7.4.3#dependencies-body-tab), the package supports more than one version:

image

raphaelquati commented 7 months ago

Checking my project, dependencies:

Solution
-> MyProjectA
    -> MyInternalLib (nuget)
        -> Rebus.Microsoft.Extensions.Logging (3.0.0)
              -> Rebus (=> 6.0.0)
        -> Rebus.OpenTelemetry (0.0.4)
              -> Rebus (=> 6.4.1)
         -> Rebus.Diagnostics (0.0.4)
              -> Rebus (>= 6.4.1)
         -> Rebus.RabbitMq (7.4.3)
              -> Rebus (>= 6.0.0 && < 7.0.0)
         -> Rebus (6.6.5)
     -> SubProject
               -> Rebus (7.0.0)

In this situation, nuget warnings about inconsistencies. Looking at project.assets.json (CycloneDX parse these files), there's a log message:

 "logs": [
    {
      "code": "NU1608",
      "level": "Warning",
      "warningLevel": 1,
      "message": "Detected package version outside of dependency constraint: Rebus.RabbitMq 7.4.3 requires rebus (>= 6.0.0 && < 7.0.0) but version Rebus 7.0.0 was resolved.",
      "libraryId": "rebus",
      "targetGraphs": [
        "net7.0"
      ]
    }
  ]

But dotnet build the solution with no error (and the project is running at my production environment), using Rebus 7.0.0.

raphaelquati commented 7 months ago

Running CycloneDX locally, the error occurs because this line: https://github.com/CycloneDX/cyclonedx-dotnet/blob/8a6ad3dcc5a8b63dc51d8f13eec89c27e661f823/CycloneDX/Program.cs#L319-L320

In my case, packageNameMatch finds 2 packages: Rebus 6.6.5 and Rebus 7.0.0. Error occurs.

We know this is an error on our code. But what is correct to report? 6.6.5 or 7.0.0? Or both?

Maybe this is related to https://github.com/CycloneDX/cyclonedx-dotnet/issues/581

mtsfoni commented 7 months ago

There are two different problems as far as I captured it. One is a general problem with .NET standard being listed as dependency but never added to the list of known dependencies. That should be #739 fixed by #771.

Another error seems to happen, if a package is a dependency (also transitive) with two not overlapping version ranges. The problem is, CycloneDX runs at design time, and I'm not a fan of predicting which package might end up being used and which is being thrown out.

raphaelquati commented 7 months ago

.... Another error seems to happen, if a package is a dependency (also transitive) with two not overlapping version ranges. The problem is, CycloneDX runs at design time, and I'm not a fan of predicting which package might end up being used and which is being thrown out.

I agree with you. But in project.assets.json (nuget), the warning NU1608 indicates which version was taken to be used (thru the field "targetGraphs").

raphaelquati commented 7 months ago

I agree with you. But in project.assets.json (nuget), the warning NU1608 indicates which version was taken to be used (thru the field "targetGraphs").

Ops.... targetGraphs shows net version, not lib being used...... sorry

mtsfoni commented 7 months ago

Too bad, after reading your post from last week, I kinda hoped there was an easy, overlooked solution to the problem.

I will get to look into it in about 2 weeks.

mtsfoni commented 6 months ago

This gives an actual Error NU1107.

Solution
-> ConsoleApp
  -> Rebus.RabbitMq (7.4.6)    
    -> Rebus (>= 6.0.0 && < 7.0.0)
  -> SubProject (Project Reference)
    -> Rebus (7.0.0)

This gives a Warning NU1608.

Solution
-> ConsoleApp
  -> Rebus.RabbitMq (7.4.6)    
    -> Rebus (>= 6.0.0 && < 7.0.0)
  -> Rebus (7.0.0)

CyconeDX prints a warning Dependency (rebus) with version range ([6.0.0, 7.0.0)) referenced by (Name:Rebus.RabbitMq Version:7.4.6) did not resolve to a specific version. but the output seems correct:

<dependency ref="pkg:nuget/Rebus.RabbitMq@7.4.6">
      <dependency ref="pkg:nuget/RabbitMQ.Client@6.4.0" />
      <dependency ref="pkg:nuget/Rebus@7.0.0" />
    </dependency>

This Visual Studio didn't allow me to build (Rollback).

Solution
-> ConsoleApp
  -> Rebus.RabbitMq (7.4.6)    
    -> Rebus (>= 6.0.0 && < 7.0.0)
  -> MyPackage (Nuget)
    -> Rebus (7.0.0)

This looks like your example, but it also give me an error NU1107:

Solution
-> ConsoleApp
  -> MyPackage (Nuget)    
    -> Rebus.RabbitMq (7.4.6)    
      -> Rebus (>= 6.0.0 && < 7.0.0)
  -> Subproject (project reference)
    -> Rebus (7.0.0)

If you can still reproduce the problem, can you maybe send me a sanitized solution example? Maybe also the project.assets.json could be enough already.

Tried on Master (3.0.2)

raphaelquati commented 6 months ago

I've cloned this repo and build the master branch. Same error.

I've created a test solution to reproduce the behavior

TestCyclone.zip

mtsfoni commented 5 months ago

I could only reproduce this, calling CycloneDX directly on the .sln-file. That creates the situation that dependencies of each project are gathered separately and then aggregated. This way CycloneDX cannot know, which is the actually used version as depending on which project you build either is true.

I am not 100% sure if this should be fixed, as the result might not reflect the actual truth.

However, who generates an SBOM from a .sln-file that includes multiple packages or other output projects (like executables or services, I like to call those root-projects) cannot expect the SBOM to include sensible information.

So in that case, CycloneDX will show this warning and resolve to the first found reference:

Warning: Multiple BOM references were found for rebus [6.0.0, 7.0.0). It appears that the component has been located multiple times with different versions. To resolve this issue, consider targeting a project file directly that's using a project.assets.json to resolve dependencies and avoid using the --recursive argument.
Choosing first found reference rebus 7.1.0 to resolve.
Bertk commented 5 months ago

@mtsfoni The current design does not use any build process data and therefore ends with ambiguous version resolution. Sorry for this but I again need to mention Buildalyzer (#736) which provides this details and could be used to resolve the version resolution.

image
mtsfoni commented 5 months ago

Buildanalyzer is no. 1 on my list now. But i doubt this is not a problem it could solve.

The Problem here is that Raphael has two separate programmes in one solution. Through this whole "one is a package, that being a package referenced by the other" the actual problem is made to look more complicated.

But this is the problematic structure:

someSolution.sln
├ SomeCLITool.csproj
│ └ DependencyA@1.0.0
└ CompleteDifferentCLITool.csproj
  └ DependencyA@2.0.0

Building this solution will usually have two outputs:

./SomeCLITool/bin/
./CompleteDifferentCLITool/bin/

One of those outputs has DependencyA in version 1.0.0 and the other in version 2.0.0. This would also be in the respective BOM of each project, no problems with resolving here. However, if you make the SBOM of the whole solution, which is nonsensible here but possible, Meta-Component "someSolution" is dependent on both versions simultaneously.

Long story short: This actually is a problem about aggregating/merging, rather than generating/resolving.

Bertk commented 5 months ago

This is actually a supported scenario. Buildanlyzer uses a project array and the references, properties, … are project specific. There is a performance cost and I think the design build should only be done once for the solution. The recursive option might be a challenge but could be solved with a pseudo solution for all csproj.

The result with 2 versions for a package in the solution SBOM is correct and something which should be expected.

mtsfoni commented 5 months ago

The result with 2 versions for a package in the solution SBOM is correct and something which should be expected.

You are right, this is a case for being very stupid. Just finding and listing all dependencies, if there is a multiple version we just show every single one. There is no higher level of accuracy possible with a .sln-file and we shouldn't claim to have one. Buildalzyer should be able to provide us all project files to look at.

The recursive option might be a challenge but could be solved with a pseudo solution for all csproj.

I think we still have to iterate ourselves, but should use buildalyzer to get the project-references instead of the current implementation reading the xml. That might help with cases #826 where CycloneDX ignores the condition in the csproj.