FEAT: Option to add license text to SBOM result

rkg-mm commented 2 years ago

Is your feature request related to a problem? Please describe.

For legal documentation, we need the original text of the licenses of components.

Describe the solution you'd like

An option to enable integration of the license-text in the BOM file, like the old @cyclonedx/bom package had, would be great to have again here.

read https://cyclonedx.org/news/cyclonedx-v1.3-released/#copyright-and-license-evidence

Acceptance criteria

the feature to add license texts should be enabled by a CLI switch called --gather-license-evidence (name to be discussed)
the feature is disabled per default
only if the feature is enabled:
- for all components, meta-components, root-components and nested components: regardless of SPDX license ID, SPDX license expression or named license, the deteced license texts should be added, each as an evidence Examples:
```
{
//...
"evidence": { 
  "licenses": [
    {"name":"file: LICENSE", "text": {
      "contentType": "text/plain",
      "encoding": "base64",
      // base64 of content of file `LICENSE`
      "content": "bG9yZW0gaXBzdW0="
    }}
    {"name":"file: NOTICE", "text": {
      "contentType": "text/plain",
      "encoding": "base64",
      // base46 of content of file `NOTICE`
      "content": "bG9yZW0gaXBzdW0="
    }}
  ]
},
// ...
}
```
- if a license text is detected with the package, it would be added to Component's @.evicence.licenses
- @.name would be 'License of : '
- @.text would hold the test
  - the content type is to be derived from file extension
  - the content SHOULD be base64 encoded
- license files patterns are:
- LICEN[CS]E*
- NOTICE* -- addendum for Apache-2.0 and others
- if no license text is shipped with a package, no license test is added as a evidence. Nope, no license template is derived from package's declared SPDX license id.
  Reason: license templates (like BSD clause 3) are designed to be modified (unlike others, like Apache2, which is not a template but a complete text)

jkowalleck commented 2 years ago

this very feature request is similar to https://github.com/CycloneDX/cyclonedx-node-module/issues/343#issue-1418891898

jkowalleck commented 2 years ago

previous license text gathering implementation: https://github.com/CycloneDX/cyclonedx-node-module/blob/3.x/model/LicenseChoice.js#L72

jkowalleck commented 2 years ago

reminder: npm6 had the readme as part of the result of npm ls this was dropped on npm7 ...

unfortunately there is no such feature for the license. it needs to be gathered manually.

jkowalleck commented 2 years ago

feel free to open pull-requests to bring this feature to the tool.

jkowalleck commented 1 year ago

this very feature request is similar to https://github.com/CycloneDX/cyclonedx-javascript-library/issues/404

jkowalleck commented 1 year ago

I am investigating which library could be used for license text extraction. Do not want to implement nor maintain a solution, if there is one already.

:mag: result of my research:

we could utilize license-scanner after https://github.com/CycloneDX/license-scanner/issues/9 was done.

PS: if no lib nor scanner can be found, lets simply pull the license texts from files that seam appropriate.

jkowalleck commented 1 year ago

@stevespringett mentioned:

Keep in mind that compositions are the only way of saying that something is complete. Our build implementations do not produce compositions nor should they. Its really up to the end user org to attest if the BOM is complete or not and add the corresponding composition if it is. Also note, that licenses go in two places. 1) the declared license for the component goes into component/licenses and all the evidence of copyrights and licenses for the entirety of the component goes into component/evidence

read: https://cyclonedx.org/news/cyclonedx-v1.3-released/#copyright-and-license-evidence

jkowalleck commented 1 year ago

after #726 is done, the underlying data types allow the use of any stringable, so Buffers will be possible to use.

PS: or helper classes that return the proper base64-sting when toString() is called on them.

ghost commented 1 year ago

I would love to help implement the feature, but I am not sure where this code would reside. For SPDX identifiers, we could parse the package.json of the dependencies or rather execute npm for this, right? For specific license text we could use the old logic to identify custom text, if the SPDX identifier is not present.

Could you give me some hints on where to start and what you expect as a maintainer?

hacst commented 1 year ago

Is the spdx id based license template text really the expected thing to add as license text in the sbom? The actual project license contains things like copyright holders, date ranges etc that will not be part of the license template but are quite critical. E.g. one use case that comes to mind is reproducing all OSS licenses of dependencies that require you to do so for the user.

jkowalleck commented 1 year ago

re https://github.com/CycloneDX/cyclonedx-node-npm/issues/256#issuecomment-1671001917

I would love to help implement the feature, but I am not sure where this code would reside. when the code exists, we will find a place. it might start here: src/licenseText.ts

For SPDX identifiers, we could parse the package.json of the dependencies or rather execute npm for this, right?

SPDX licenses ID are already detected. a subroutine called by BomBuilder.makeComponent() handles it properly. see the internals of https://github.com/CycloneDX/cyclonedx-node-npm/blob/42208c67c907b671caa01b82f4930a3d9c4203f7/src/builders.ts#L414 here https://github.com/CycloneDX/cyclonedx-javascript-library/blob/main/src/builders/fromNodePackageJson.node.ts

For specific license text we could use the old logic to identify custom text, if the SPDX identifier is not present.

yes, basically we could use it to detect all kinds of license files and addendum and notice and others.

when it comes to the question "where do i put the detected license texts in the BOM", please read https://github.com/CycloneDX/cyclonedx-node-npm/issues/256#issuecomment-1448734037 License texts and such are evidence and should be placed in the respective section of the BOM.

I updated the acceptance criteria on the issue, so it is much clearer what to expect. Did this help?

implementation-wise an attachment can be anything that has a toString() method that returns base64. see https://github.com/CycloneDX/cyclonedx-javascript-library/blob/main/src/models/attachment.ts So instead of pulling all the files' contents from disc and putting them into the RAM at the same time, it would be great to use a wrapper class, that stores the file name, and when it comes to calling the toString() method of an instance, it returns the base64 of the file content. Let me know if you have trouble creating such a Class.

jkowalleck commented 1 year ago

re https://github.com/CycloneDX/cyclonedx-node-npm/issues/256#issuecomment-1671072574

Is the spdx id based license template text really the expected thing to add as license text in the sbom?

nope. goal is to collect evidence, not the template texts. see https://cyclonedx.org/news/cyclonedx-v1.3-released/#copyright-and-license-evidence The original request for this issue came from a law-perspective. I know layers love texts, but they are not stupid, they can gather the template texts themselves.

I updated the acceptance criteria on the issue, so it is much clearer what to expect. Did this help?

jkowalleck commented 1 month ago

A similar feature was added to the webpack plugin see https://github.com/CycloneDX/cyclonedx-webpack-plugin/pull/1309 see https://github.com/CycloneDX/cyclonedx-webpack-plugin/pull/1312

CycloneDX / cyclonedx-node-npm