CycloneDX / cyclonedx-maven-plugin

Creates CycloneDX Software Bill of Materials (SBOM) from Maven projects
https://cyclonedx.org/
Apache License 2.0
295 stars 84 forks source link

Missing dependency in 2.0.0 #54

Open mvanini opened 4 years ago

mvanini commented 4 years ago

There seems to be a missing dependency in latest release. Since few hours ago we seen a lot of failure on our CIs about invocation of cyclonedx-maven-plugin via cli, like that:

[ERROR] Failed to execute goal org.cyclonedx:cyclonedx-maven-plugin:2.0.0:makeAggregateBom (default-cli) on project MyProject: Execution default-cli of goal org.cyclonedx:cyclonedx-maven-plugin:2.0.0:makeAggregateBom failed: Plugin org.cyclonedx:cyclonedx-maven-plugin:2.0.0 or one of its dependencies could not be resolved: Could not find artifact com.github.everit-org.json-schema:org.everit.json.schema:jar:1.12.1 in my-mirror (https://nexus.example.com/nexus/repository/my-mirror) -> [Help 1]

I searched com.github.everit-org.json-schema:org.everit.json.schema on search.maven.org and i didn't found it, even for past versions... is groupdId and artifactId correct?

coderpatros commented 4 years ago

Hi @mvanini

That dependency is pulled in via jitpack.io

I couldn't reproduce it locally, but I don't use maven or java normally. So maybe I'm doing it wrong. But I'm sure @stevespringett will be all over this when he comes online. Especially seeing as a few people are hitting it from the thumbs ups :(

coderpatros commented 4 years ago

Is there any interesting information about your CI environment that you can share? And do you also hit it running locally?

mvanini commented 4 years ago

Hi @coderpatros , maven by default resolve dependency via https://repo1.maven.org (which is the backend of search.maven.org webapp which I mentioned before). In our CI environment we have a corporate maven repository (Sonatype Nexus) which is also used as a proxy of all major public repository, repo1.maven.org included. For our scenario we can add a mirror for jitpack in our Nexus but is a workaround not really a solution: not anyone that use this plugin has a local maven repository to store that dependency or possibility to mirror jitpack.

I think that all dependencies needed by a maven plugin must be publicy available on the same repository where maven look for it.

Is there any interesting information about your CI environment that you can share? And do you also hit it running locally?

Yes, I have the same result if I run it locally

coderpatros commented 4 years ago

Yeah, I probably don't know enough about maven to help on this one. Maven picked it up fine for me on a test project. But that was without a corporate maven repository. I know that jitpack.io is added as a repository in the pom of the core java package, which the maven plugin uses.

stevespringett commented 4 years ago

I think that all dependencies needed by a maven plugin must be publicly available on the same repository where maven look for it.

Not every dependency resides in Maven Central. The JSON schema validator is a dependency of CycloneDX Java Core which has the necessary repositories entry in the POM so that resolution can occur from non-default repositories.

https://github.com/CycloneDX/cyclonedx-core-java/blob/master/pom.xml#L103

It appears your environment however, is ignoring this. This might be a Nexus proxy configuration issue, but I'm not entirely sure. I personally do not use Nexus proxy since provenance information becomes opaque.

The GitHub action that builds the Maven plugin would have failed if the dependency could not be resolved. Additionally, some of my projects, also built with GH actions or Travis, resolve the component properly.

mvanini commented 4 years ago

I think that the mistake resides in the mirror use in our settings: if I declare a mirror in my settings.xml then all requests to resolve a dependency will be redirected through that mirror which, in our case, doesn't contains a proxy for the gitpack repository.

It's strange that it faults only on version 2.0.0, our config it's the same since years. Is this a new declared dependency in latest release?

I personally do not use Nexus proxy since provenance information becomes opaque.

In an enterprise scenario with many projects, each one with his dependencies, having a local maven repository for public dependencies saves us a huge outbound network traffic to internet.

stevespringett commented 4 years ago

Is this a new declared dependency in latest release?

Yes. It's necessary for JSON validation. Out of only a handful of libraries available for Java, everit-org is the most widely used and the only one that relies solely on the org.json library. The others use gson or jackson databind which I try to avoid, especially for a Mavan plugin. The only other alternative was AGPL licensed, which is toxic in most enterprises.

So the everit-org library is ideal for use with the Maven plugin both from an architecture and license perspective, but just so happens to not be hosted on Maven Central.

mvanini commented 4 years ago

I discovered in everit-org/json-schema#360 that it seems the dependency is published even on maven-central but with a different groupId/artifactId (why????):

        <dependency>
            <groupId>com.github.erosb</groupId>
            <artifactId>everit-json-schema</artifactId>
            <version>1.12.1</version>
        </dependency>

You can verify on https://search.maven.org/artifact/com.github.erosb/everit-json-schema

Could you consider to switch to that GAV for the next releases?

stevespringett commented 4 years ago

That artifact is a fork of the main project. Forks can introduce unnecessary security risk since they need to be consistently updated whenever the upstream component releases a new version. Forks are also less likely to be reviewed by security researchers and are a target for potential malware.

rajesh2887 commented 4 years ago

can anyone help on this below error.while trying to build from Jenkins pipeline getting below error.

2020-07-16 13:38:53 [ERROR] Failed to execute goal org.cyclonedx:cyclonedx-maven-plugin:2.0.0:makeAggregateBom (default-cli) on project "myproject": Execution default-cli of goal org.cyclonedx:cyclonedx-maven-plugin:2.0.0:makeAggregateBom failed: java.net.ConnectException: Connection timed out: connect -> [Help 1]

msymons commented 4 years ago

I have seen the missing org.everit.json.schema problem with cyclonedx-maven-plugin 2.0.3

I am using Nexus Repository Manager.

The interesting thing is that I have 5 separate Jenkins servers and all seem to have identical maven configuration... and yet only one of them had a problem, even though inspection of Nexus showed that org.everit.json.schema was not available.

I did then set up a proxied repository in NXRM for jitpack.io (taking but 1 minute to do) and suddenly all came right... the builds on the problem Jenkins server are now fine (and org.everit.json.schema is now in NXRM, obviously).

Wrt " provenance information becomes opaque" I had a look at what NXRM was telling me. It (v3.25.1) reports:

Containing repo jitpack.io

There is also an "Attributes" section for each file, and this section contains "Provenance". Trouble is... it reports "hashes_not_verified" for everything that I bothered to inspect.

pachulo commented 4 years ago

We were having the same problem.

I think that the mistake resides in the mirror use in our settings: if I declare a mirror in my settings.xml then all requests to resolve a dependency will be redirected through that mirror which, in our case, doesn't contains a proxy for the gitpack repository.

@mvanini in our case, changing the config from this:

....
    <mirror>
       <id>mirror</id>
      <mirrorOf>*</mirrorOf>
      <name>MyCompany Nexus Mirror</name>
      <url>https://nexus.mycompany.net/content/groups/public</url>
    </mirror>
....

To this:

....
    <mirror> 
      <id>mirror</id> 
      <mirrorOf>central</mirrorOf> 
      <name>MyCompany Nexus Mirror</name> 
      <url>https://nexus.mycompany.net/content/groups/public</url> 
    </mirror>
....

(Mind the mirrorOf change) solved the issue.

More info: https://maven.apache.org/guides/mini/guide-mirror-settings.html

bigboban commented 3 years ago

Same error for me in gradle build:

08:30:37  > Could not resolve all artifacts for configuration ':classpath'.
08:30:37     > Could not find com.github.everit-org.json-schema:org.everit.json.schema:1.12.1.
08:30:37       Searched in the following locations:
08:30:37         - https://plugins.gradle.org/m2/com/github/everit-org/json-schema/org.everit.json.schema/1.12.1/org.everit.json.schema-1.12.1.pom
08:30:37       If the artifact you are trying to retrieve can be found in the repository but without metadata in 'Maven POM' format, you need to adjust the 'metadataSources { ... }' of the repository declaration.
08:30:37       Required by:
08:30:37           project : > org.cyclonedx.bom:org.cyclonedx.bom.gradle.plugin:1.2.1 > com.cyclonedx:cyclonedx-gradle-plugin:1.2.1 > org.cyclonedx:cyclonedx-core-java:3.0.5

I don't understand - issue exists more than half of year and still not fixed?

patrickp-at-work commented 3 years ago

We ran into the same issue with our CI setup that uses a mirror in front of the public repositories. It is unfortunate because we noticed at the very last step, when the changes were tested and pushed, and then the build suddenly failed. It would be really convenient not to have to research what causes the failure an then look for a workaround.

stevespringett commented 3 years ago

The most common cause is a mis-configured Nexus or Artifactory repo. The component is question does not reside in Maven Central. It resided in Jitpack.

prabhu commented 3 years ago

This particular issue is causing an headache for many users. For nexus, the settings suggest here works https://github.com/jitpack/jitpack.io/issues/506#issuecomment-267338823

stevespringett commented 3 years ago

With version 2.3.0 of the Maven plugin, this issue no longer exists.