[FEATURE]: Adding an XML Catalog file

Nicolas-Peiffer commented 3 months ago

Describe the feature

Adding an XML catalog schema/xmlcatalog.xml file under the schema/ folder. This addition is transparent, but will allow other projects to use this XML catalog in their XML parser.

<?xml version="1.0"?>
<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">
  <uri name="http://cyclonedx.org/schema/spdx" uri="spdx.xsd"/>
  <uri name="http://cyclonedx.org/schema/bom/1.0" uri="bom-1.0.xsd"/>
  <uri name="http://cyclonedx.org/schema/bom/1.1" uri="bom-1.1.xsd"/>
  <uri name="http://cyclonedx.org/schema/bom/1.2" uri="bom-1.2.xsd"/>
  <uri name="http://cyclonedx.org/schema/bom/1.3" uri="bom-1.3.xsd"/>
  <uri name="http://cyclonedx.org/schema/bom/1.4" uri="bom-1.4.xsd"/>
  <uri name="http://cyclonedx.org/schema/bom/1.5" uri="bom-1.5.xsd"/>
  <uri name="http://cyclonedx.org/schema/bom/1.6" uri="bom-1.6.xsd"/>
</catalog>

Possible solutions

While working with cyclonedx-go, I noticed 2 variables xmlNamespaces and xmlSchemaFiles:

xmlNamespaces: which is used to bind XML namespaces URLs with CycloneDX spec versions;
xmlSchemaFiles: which is used to bind local XSD schema files location path with CycloneDX spec versions;

var xmlSchemaFiles = map[SpecVersion]string{
    SpecVersion1_0: "./schema/bom-1.0.xsd",
    SpecVersion1_1: "./schema/bom-1.1.xsd",
    SpecVersion1_2: "./schema/bom-1.2.xsd",
    SpecVersion1_3: "./schema/bom-1.3.xsd",
    SpecVersion1_4: "./schema/bom-1.4.xsd",
    SpecVersion1_5: "./schema/bom-1.5.xsd",
    SpecVersion1_6: "./schema/bom-1.6.xsd",
}

var xmlNamespaces = map[SpecVersion]string{
    SpecVersion1_0: "http://cyclonedx.org/schema/bom/1.0",
    SpecVersion1_1: "http://cyclonedx.org/schema/bom/1.1",
    SpecVersion1_2: "http://cyclonedx.org/schema/bom/1.2",
    SpecVersion1_3: "http://cyclonedx.org/schema/bom/1.3",
    SpecVersion1_4: "http://cyclonedx.org/schema/bom/1.4",
    SpecVersion1_5: "http://cyclonedx.org/schema/bom/1.5",
    SpecVersion1_6: "http://cyclonedx.org/schema/bom/1.6",
}

They were lacking information about <uri name="http://cyclonedx.org/schema/spdx" uri="spdx.xsd"/>.

My goal is to write an XML validation method for XML formated BOMs. I am discussing this possibility on the CycloneDX Go Slack channel. Especially, we are discussing an issue about the need to enable CGO when using github.com/lestrrat-go/libxml2.

XML Catalogs are very handy when it comes to solving the issue of binding XML Namespace URLs with locally stored XSD files.

See my example here: https://github.com/Nicolas-Peiffer/cyclonedx-go/blob/427e8adad7e05f57008c78a76075ddc6e38fb007/validator_xml.go#L47-L48

import (
    "github.com/lestrrat-go/libxml2"
    "github.com/lestrrat-go/libxml2/xsd"
)

[...]

    // Load the XSD schema and the XML catalog
    schema, err := xsd.Parse(schemaData.Bytes(), xsd.WithPath("schema/xmlcatalog.xml"))

First tests of using an XML catalog seems promising: the tests go through with success. https://github.com/Nicolas-Peiffer/cyclonedx-go/commit/a4f347065cba19e4e41ec16cc652913e23e221f5

Copie d'écran_20240607_190900

Alternatives

I do not know alternatives to XML catalogs, except implementing custom variables like cyclonedx-go's xmlNamespaces and xmlSchemaFiles variables.

Additional context

See the Slack Go channel.

jkowalleck commented 3 months ago

maybe add some sanity/plausibility/regression test that checks whether the namespace in the catalog matches the referenced file, and assert that the referenced file exists.

Nicolas-Peiffer commented 3 months ago

maybe add some sanity/plausibility/regression test that checks whether the namespace in the catalog matches the referenced file, and assert that the referenced file exists.

Good idea @jkowalleck . But first, I updated the version of cyclonedx-core-java from cyclonedx-core-java-8.0.3 to cyclonedx-core-java-9.0.2, because 9.0.2 is the latest available: see my new pom.xml file.

And I added the tests for CDX v1.6: indeed you can see below that 1.6 was not tested:

JsonSchemaVerificationTest.java only goes to 1.5
BaseSchemaVerificationTest.java only goes to 1.5
XmlSchemaVerificationTest.java only goes to 1.5

So far, the tests seems to go through for 1.6, which is a good news :smiley: .

Now I need to implement tests for the xmlcatalog.xml file like you suggested. This is a work in progress. I have some trouble locating the file under ./schema/ when the test source code is here ./tools/src/test/java/org/cyclonedx/schema/. I need to figure this out.

NB : if I were doing my tool in Java, it seems the CycloneDX Java implementation is more mature than the cyclonedx-go.

Nicolas-Peiffer commented 3 months ago

@jkowalleck My latest commit https://github.com/ThalesGroup/cyclonedx-specification/commit/c1fe8a9781ee08d766292bdf269391e909712707 seems very promising:

I added your comments on the XML catalog, plus a couple of other comments;
I managed to write a Java test for testing the content of the XML Catalog and verify if the content of the XSD files match
I updated the JSON and XML Java test to CycloneDX 1.6
Java unit tests go trough and validate !

And on my machine environment (VSCode, openjdk 22), tests seems to run correctly. But I have no way of testing on another environment. Unless I activate github actions...

java  --version

openjdk 22 2024-03-19
OpenJDK Runtime Environment (build 22)
OpenJDK 64-Bit Server VM (build 22, mixed mode, sharing)

Below a screen shot from the VSCode tests window:

Copie d'écran_20240611_191514

jkowalleck commented 3 months ago

The modernizations in the tests make this a mix of topics. @Nicolas-Peiffer, could you split the PR in multiple parts? One is the actual feature, and then additional pull requests for the modernizations?

Nicolas-Peiffer commented 3 months ago

On the principle, I agree that should be 2 PR :

one for the XML catalog,
one for the update of Java unit tests.

I have to figure the best way to do that. What I suggest :

I close and cancel current PR: https://github.com/CycloneDX/specification/pull/477
I create a branch pr-xml-catalog dedicated for the XML catalog. I do a new PR with this branch. This PR should be merged after the first, but not mandatory.
I create a branch pr-update-java-utests dedicated for the update of Java-cycloneDX. I do a new PR with this branch. This PR should be merged second, but not mandatory.

I close this PR to open new ones.

CycloneDX / specification