search

oss-review-toolkit / ort

A suite of tools to automate software compliance checks.
https://oss-review-toolkit.org
Apache License 2.0
1.6k stars 309 forks source link

Empty Reports e.g. NOTICE file after successful scan and evaluation for Yarn #6380

Closed schvvarzekatze closed 1 year ago

schvvarzekatze commented 1 year ago

The scan now seems to be successful with a complete set of result and finds all packages (>2k). But the reports as NOTICE file and bill of materials are still empty.

If ORT is run on gradle packages there exists a report filled with content including license texts and copyrights.

Do you have an idea what could have caused the empty reports? Is there an additional config necessary that the report templates also work for npm?

The artifacts with package examples and the logs are shown below:

analyzer.yml

repository:
  vcs:
    type: "Git"
    url: "git@git.intra.xyz.xyz/project.git"
    revision: "e5a6f897d7bf0849a3e48f6ea282b941404b5709"
    path: ""
  vcs_processed:
    type: "Git"
    url: "ssh://git@git.intra.xyz.com/xyz/project.git"
    revision: "e5a6f897d7bf0849a3e48f6ea282b941404b5709"
    path: ""
  config:
    excludes:
      scopes:
      - pattern: "compileOnly"
        reason: "BUILD_DEPENDENCY_OF"
        comment: "Gradle: Only needed during build"
      - pattern: "devDependencies"
        reason: "DEV_DEPENDENCY_OF"
        comment: "NPM: Packages for development only."
analyzer:
  start_time: "2023-01-22T18:35:17.772162657Z"
  end_time: "2023-01-22T18:37:36.271636374Z"
  environment:
    ort_version: "DOCKER-SNAPSHOT"
    java_version: "17.0.5"
    os: "Linux"
    processors: 32
    max_memory: 34359738368
    variables:
      JAVA_HOME: "/opt/java/openjdk"
      ANDROID_HOME: "/opt/android-sdk"
    tool_versions:
      Yarn: "1.22.17"
  config:
    allow_dynamic_versions: true
    enabled_package_managers:
    - "yarn"
    package_managers:
      Yarn:
        options:
          directDependenciesOnly: "true"
      Gradle:
        options:
          directDependenciesOnly: "true"
      NPM:
        options:
          directDependenciesOnly: "true"
  result:
    projects:
    - id: "Yarn::package.json:"
      definition_file_path: "package.json"
      declared_licenses: []
      declared_licenses_processed: {}
      vcs:
        type: ""
        url: ""
        revision: ""
        path: ""
      vcs_processed:
        type: "Git"
        url: "ssh://git@git.intra.xyz.com/xyz/project.git"
        revision: "e5a6f897d7bf0849a3e48f6ea282b941404b5709"
        path: ""
      homepage_url: ""
      scope_names: []
    packages:
    - metadata:
        id: "NPM::abab:1.0.4"
        purl: "pkg:npm/abab@1.0.4"
        authors:
        - "Jeff Carpenter"
        declared_licenses:
        - "ISC"
        declared_licenses_processed:
          spdx_expression: "ISC"
        description: "WHATWG spec-compliant implementations of window.atob and window.btoa."
        homepage_url: "https://github.com/jsdom/abab#readme"
        binary_artifact:
          url: ""
          hash:
            value: ""
            algorithm: ""
        source_artifact:
          url: "https://registry.npmjs.org/abab/-/abab-1.0.4.tgz"
          hash:
            value: "5faad9c2c07f60dd76770f71cf025b62a63cfd4e"
            algorithm: "SHA-1"
        vcs:
          type: "Git"
          url: "git+https://github.com/jsdom/abab.git"
          revision: "c98068b06b4321949a8195408360ca84140d795d"
          path: ""
        vcs_processed:
          type: "Git"
          url: "https://github.com/jsdom/abab.git"
          revision: "c98068b06b4321949a8195408360ca84140d795d"
          path: ""
      curations: []

scan-result.yml

packages:
    - metadata:
        id: "NPM::abab:1.0.4"
        purl: "pkg:npm/abab@1.0.4"
        authors:
        - "Jeff Carpenter"
        declared_licenses:
        - "ISC"
        declared_licenses_processed:
          spdx_expression: "ISC"
        description: "WHATWG spec-compliant implementations of window.atob and window.btoa."
        homepage_url: "https://github.com/jsdom/abab#readme"
        binary_artifact:
          url: ""
          hash:
            value: ""
            algorithm: ""
        source_artifact:
          url: "https://registry.npmjs.org/abab/-/abab-1.0.4.tgz"
          hash:
            value: "5faad9c2c07f60dd76770f71cf025b62a63cfd4e"
            algorithm: "SHA-1"
        vcs:
          type: "Git"
          url: "git+https://github.com/jsdom/abab.git"
          revision: "c98068b06b4321949a8195408360ca84140d795d"
          path: ""
        vcs_processed:
          type: "Git"
          url: "https://github.com/jsdom/abab.git"
          revision: "c98068b06b4321949a8195408360ca84140d795d"
          path: ""
      curations: []

evaluation-result.yml

packages:
    - metadata:
        id: "NPM::abab:1.0.4"
        purl: "pkg:npm/abab@1.0.4"
        authors:
        - "Jeff Carpenter"
        declared_licenses:
        - "ISC"
        declared_licenses_processed:
          spdx_expression: "ISC"
        description: "WHATWG spec-compliant implementations of window.atob and window.btoa."
        homepage_url: "https://github.com/jsdom/abab#readme"
        binary_artifact:
          url: ""
          hash:
            value: ""
            algorithm: ""
        source_artifact:
          url: "https://registry.npmjs.org/abab/-/abab-1.0.4.tgz"
          hash:
            value: "5faad9c2c07f60dd76770f71cf025b62a63cfd4e"
            algorithm: "SHA-1"
        vcs:
          type: "Git"
          url: "git+https://github.com/jsdom/abab.git"
          revision: "c98068b06b4321949a8195408360ca84140d795d"
          path: ""
        vcs_processed:
          type: "Git"
          url: "https://github.com/jsdom/abab.git"
          revision: "c98068b06b4321949a8195408360ca84140d795d"
          path: ""
      curations: []

Template for notice file:

[#assign noticeCategoryName = "include-in-notice-file"]
[#assign sourceCodeCategoryName = "include-source-code-offer-in-notice-file"]

[#-- Add the licenses of all packages. --]
[#list packages?filter(p -> !p.excluded) as package]
    [#assign hasNoticePackageLicenses = false]

[#assign licensesFilteredIncludeInNoticeFile = LicenseView.ONLY_DETECTED.filter(package.license.licenses)]
[#assign resolvedLicenses = helper.filterForCategory(
package.licensesNotInLicenseFiles(licensesFilteredIncludeInNoticeFile), noticeCategoryName
)]
[#assign licensesFilteredIncludeInSourceCodeOffer = LicenseView.ONLY_DETECTED.filter(package.license.licenses)]
[#assign resolvedLicensesSourceCodeOffer = helper.filterForCategory(
package.licensesNotInLicenseFiles(licensesFilteredIncludeInSourceCodeOffer), sourceCodeCategoryName
    )]
    [#list resolvedLicenses as resolvedLicense]
        [#assign declaredLicenses = resolvedLicense.originalExpressions?filter(expression -> expression.source?has_content && expression.source="DECLARED")]
        [#list declaredLicenses as declaredLicense]
        [#assign licenseName = resolvedLicense.license.simpleLicense()]
        [#assign licenseLocations = resolvedLicense.locations]
        [#assign licenseText = licenseTextProvider.getLicenseText(licenseName)!]
        [#if !licenseText?has_content][#continue][/#if]
            [#if !hasNoticePackageLicenses]
########################################################################################################################

Package: [#if package.id.namespace?has_content]             ${package.id.namespace}:[/#if]${package.id.name}:${package.id.version}
                [#assign hasNoticePackageLicenses = true]
                [#if resolvedLicensesSourceCodeOffer?seq_contains(resolvedLicense)]
*******************************

Written Offer for Source Code

                [#if licenseLocations?has_content && licenseLocations[0].provenance?has_content]
                    [#assign artifactProvenance = licenseLocations[0].provenance]
                        [#if artifactProvenance?has_content && artifactProvenance.sourceArtifact?has_content]
        ${artifactProvenance.sourceArtifact.url}
                        [#elseif artifactProvenance?has_content && artifactProvenance.binaryArtifact?has_content]
        ${artifactProvenance.binaryArtifact.url}
                        [#elseif artifactProvenance?has_content && artifactProvenance.vcsInfo?has_content]
        ${artifactProvenance.vcsInfo.url}
                        [#else]
        @TODO: Please Check for other Source Artifact Category
                        [/#if]
                     [#else]
        No Source artifact
                [/#if]

    If the source code for the technology was not provided to you with the binary, you can also receive a copy of the
    source code on physical media by submitting a written request to:

                [/#if]

[#if licenseName?has_content]
[/#if]
*******************************

License name: ${licenseName}

    [/#if]
    [#assign copyrights = resolvedLicense.getCopyrights()]
    [#if copyrights?has_content]
*******************************

The license contains the following copyright(s):

${copyrights?join("\n")}

    [/#if]
    [#if licenseText?has_content]
*******************************

License text:

${licenseText}

    [/#if]
    [#assign exceptionName = resolvedLicense.license.exception()!]
    [#assign exceptionText = licenseTextProvider.getLicenseText(exceptionName)!]
    [#if exceptionText?has_content]
*******************************

License Exceptions:

        ${exceptionText}
    [/#if]
        [/#list]
    [/#list]
[/#list]

The run of scanner, evaluator and reporter lead to the following logs which suggest that the run for all steps was successful:

SUCCESS: Created file:///builds/xyz/project/build/oss-review-toolkit/yarn/analyzer/analyzer-result.yml
INFO: Scan analyzer results with OSS review toolkit
18:37:43,162 |-INFO in ch.qos.logback.classic.LoggerContext[default] - This is logback-classic version 1.4.5
18:37:43,176 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
18:37:43,178 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [jar:file:/opt/ort/lib/helper-cli-DOCKER-SNAPSHOT.jar!/logback.xml]
18:37:43,179 |-WARN in ch.qos.logback.classic.util.DefaultJoranConfigurator@7a7d1b47 - Resource [logback.xml] occurs multiple times on the classpath.
18:37:43,179 |-WARN in ch.qos.logback.classic.util.DefaultJoranConfigurator@7a7d1b47 - Resource [logback.xml] occurs at [jar:file:/opt/ort/lib/helper-cli-DOCKER-SNAPSHOT.jar!/logback.xml]
18:37:43,179 |-WARN in ch.qos.logback.classic.util.DefaultJoranConfigurator@7a7d1b47 - Resource [logback.xml] occurs at [jar:file:/opt/ort/lib/cli-DOCKER-SNAPSHOT.jar!/logback.xml]
18:37:43,184 |-INFO in ch.qos.logback.core.joran.spi.ConfigurationWatchList@6eb82908 - URL [jar:file:/opt/ort/lib/helper-cli-DOCKER-SNAPSHOT.jar!/logback.xml] is not of type file
18:37:43,253 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - Processing appender named [STDOUT]
18:37:43,253 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
18:37:43,259 |-INFO in ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
18:37:43,278 |-INFO in ch.qos.logback.classic.model.processor.RootLoggerModelHandler - Setting level of ROOT logger to WARN
18:37:43,278 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [STDOUT] to Logger[ROOT]
18:37:43,279 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.http.headers] to ERROR
18:37:43,279 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.http.wire] to ERROR
18:37:43,279 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.eclipse.jgit.internal.storage.file.FileSnapshot] to ERROR
18:37:43,279 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.analyzer.managers.Yarn2] to INFO
18:37:43,279 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.clients.fossid.FossIdRestService] to INFO
18:37:43,279 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.reporter.reporters.fossid.FossIdReporter] to INFO
18:37:43,279 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.scanner.scanners.fossid.FossId] to INFO
18:37:43,279 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.scanner.scanners.fossid.FossIdConfig] to INFO
18:37:43,279 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.scanner.scanners.fossid.FossIdUrlProvider] to INFO
18:37:43,279 |-INFO in ch.qos.logback.core.model.processor.DefaultProcessor@4a8df3e2 - End of configuration.
18:37:43,280 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@3d98d138 - Registering current configuration as safe fallback point
18:37:43.411 [main] INFO  org.ossreviewtoolkit.model.config.OrtConfiguration - Using ORT configuration file '/project-project/ort/config.yml'.
18:37:43.751 [main] INFO  org.ossreviewtoolkit.utils.common.EnvironmentVariableFilter - EnvironmentVariableFilter initialized with denySubstrings = [key, pass, pwd, token, user] and allowNames = [CARGO_HTTP_USER_AGENT, COMPOSER_ALLOW_SUPERUSER, CONAN_LOGIN_ENCRYPTION_KEY, CONAN_LOGIN_USERNAME, CONAN_PASSWORD, CONAN_USERNAME, CONAN_USER_HOME, CONAN_USER_HOME_SHORT, DOTNET_CLI_CONTEXT_ANSI_PASS_THRU, GIT_ASKPASS, GIT_HTTP_USER_AGENT, GRADLE_USER_HOME, HACKAGE_USERNAME, HACKAGE_PASSWORD, HACKAGE_KEY, PWD, USER, USERPROFILE].
 ______________________________
/        \_______   \__    ___/ The OSS Review Toolkit, version DOCKER-SNAPSHOT.
|    |   | |       _/ |    |
|    |   | |    |   \ |    |    Running 'scan' as 'root' under Java 17.0.5 on Linux
\________/ |____|___/ |____|    with 32 CPUs and a maximum of 32768 MiB of memory.
Environment variables:
ORT_CONFIG_DIR = /project/ort
ORT_DATA_DIR = /root/.ort
JAVA_HOME = /opt/java/openjdk
ANDROID_HOME = /opt/android-sdk
Looking for ORT configuration in the following file:
    /project/ort/config.yml
Scanning projects with:
18:37:43.793 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'scancode --version' in '/home/ort'...
    ScanCode (version 31.2.1)
Scanning packages with:
18:37:44.818 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'scancode --version' in '/home/ort'...
    ScanCode (version 31.2.1)
18:37:47.080 [main] INFO  org.ossreviewtoolkit.cli.commands.ScannerCommand - Read ORT result from 'analyzer-result.yml' (2.90 MiB) in 1.191266213s.
18:37:47.118 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Scanning 1 project(s) with 1 scanner(s).
18:37:47.119 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Resolving provenance for 1 package(s).
18:37:47.141 [DefaultDispatcher-worker-3] INFO  com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting...
18:37:47.246 [DefaultDispatcher-worker-3] INFO  com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@50d76461
18:37:47.248 [DefaultDispatcher-worker-3] INFO  com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Start completed.
18:37:47.384 [DefaultDispatcher-worker-3] INFO  org.ossreviewtoolkit.scanner.provenance.DefaultPackageProvenanceResolver - Found a stored repository resolution for package 'Yarn::package.json:' with the fixed revision e5a6f897d7bf0849a3e48f6ea282b941404b5709 which was resolved to e5a6f897d7bf0849a3e48f6ea282b941404b5709.
18:37:47.389 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Resolved provenance for 1 package(s) in 266.609144ms.
18:37:47.390 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Resolving nested provenances for 1 package(s).
18:37:47.416 [DefaultDispatcher-worker-3] INFO  org.ossreviewtoolkit.scanner.provenance.DefaultNestedProvenanceResolver - Found a stored nested provenance for RepositoryProvenance(vcsInfo=VcsInfo(type=Git, url=ssh://git@git.intra.xyz.com/xyz/project.git, revision=e5a6f897d7bf0849a3e48f6ea282b941404b5709, path=), resolvedRevision=e5a6f897d7bf0849a3e48f6ea282b941404b5709) with only fixed revisions, skipping resolution.
18:37:47.417 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Resolved nested provenance for 1 package(s) in 26.286550ms.
18:37:47.418 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Reading stored scan results for 1 package(s) with 1 provenance(s).
18:37:47.468 [main] INFO  org.ossreviewtoolkit.scanner.ScanResultsStorage - Read 0 scan result(s) for 'Yarn::package.json:' from PostgresStorage in 29.506844ms.
18:37:47.469 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Read the following stored scan result(s) in 50.062528ms:
18:37:47.473 [main] INFO  org.ossreviewtoolkit.scanner.Scanner -    ScanCode: Result(s) for 0 of 1 provenance(s).
18:37:47.474 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Scanning RepositoryProvenance(vcsInfo=VcsInfo(type=Git, url=ssh://git@git.intra.xyz.com/xyz/project.git, revision=e5a6f897d7bf0849a3e48f6ea282b941404b5709, path=), resolvedRevision=e5a6f897d7bf0849a3e48f6ea282b941404b5709) (1 of 1)...
18:37:47.476 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Scanning RepositoryProvenance(vcsInfo=VcsInfo(type=Git, url=ssh://git@git.intra.xyz.com/xyz/project.git, revision=e5a6f897d7bf0849a3e48f6ea282b941404b5709, path=), resolvedRevision=e5a6f897d7bf0849a3e48f6ea282b941404b5709) (1 of 1)...
18:37:47.599 [main] INFO  org.ossreviewtoolkit.utils.ort.OrtProxySelector - Proxy selector was successfully installed.
18:37:47.614 [main] INFO  org.ossreviewtoolkit.utils.ort.OrtAuthenticator - Authenticator was successfully installed.
18:37:47.747 [main] INFO  org.ossreviewtoolkit.downloader.vcs.Git - Trying to fetch only revision 'e5a6f897d7bf0849a3e48f6ea282b941404b5709' with depth limited to 50.
18:37:47.830 [main] INFO  org.apache.sshd.common.util.security.eddsa.EdDSASecurityProviderRegistrar - getOrCreateProvider(EdDSA) created instance of net.i2p.crypto.eddsa.EdDSASecurityProvider
18:37:47.937 [main] INFO  org.apache.sshd.common.io.DefaultIoServiceFactoryFactory - No detected/configured IoServiceFactoryFactory; using Nio2ServiceFactoryFactory
18:38:04.299 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout e5a6f897d7bf0849a3e48f6ea282b941404b5709' in '/tmp/ort-DefaultWorkingTreeCache1358180320316640570'...
18:38:11.933 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Scan of RepositoryProvenance(vcsInfo=VcsInfo(type=Git, url=ssh://git@git.intra.xyz.com/xyz/project.git, revision=e5a6f897d7bf0849a3e48f6ea282b941404b5709, path=), resolvedRevision=e5a6f897d7bf0849a3e48f6ea282b941404b5709) with path scanner 'ScanCode' started.
18:38:11.936 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'scancode --copyright --license --info --strip-root --timeout 300 --processes 31 /tmp/ort-DefaultProvenanceDownloader5088315794018978217 --json-pp /tmp/ort-ScanCode7866985949743986282/result.json' in '/home/ort'...
19:09:25.553 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Scan of RepositoryProvenance(vcsInfo=VcsInfo(type=Git, url=ssh://git@git.intra.xyz.com/xyz/project.git, revision=e5a6f897d7bf0849a3e48f6ea282b941404b5709, path=), resolvedRevision=e5a6f897d7bf0849a3e48f6ea282b941404b5709) with path scanner 'ScanCode' finished.
19:09:27.309 [main] WARN  com.zaxxer.hikari.pool.PoolBase - HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@b77b0a0 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
19:09:27.309 [main] WARN  com.zaxxer.hikari.pool.PoolBase - HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@22ff1372 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
19:09:27.310 [main] WARN  com.zaxxer.hikari.pool.PoolBase - HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@356341c9 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
19:09:27.310 [main] WARN  com.zaxxer.hikari.pool.PoolBase - HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@3a70575 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
19:09:27.310 [main] WARN  com.zaxxer.hikari.pool.PoolBase - HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@5d96d434 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
19:09:27.399 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Scanning 0 package(s) with 1 scanner(s).
19:09:27.399 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Resolving provenance for 0 package(s).
19:09:27.400 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Resolved provenance for 0 package(s) in 455.52us.
19:09:27.400 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Resolving nested provenances for 0 package(s).
19:09:27.400 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Resolved nested provenance for 0 package(s) in 187.885us.
19:09:27.400 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Reading stored scan results for 0 package(s) with 0 provenance(s).
19:09:27.401 [main] INFO  org.ossreviewtoolkit.scanner.Scanner - Read the following stored scan result(s) in 387.502us:
19:09:27.401 [main] INFO  org.ossreviewtoolkit.scanner.Scanner -    ScanCode: Result(s) for 0 of 0 provenance(s).
Writing scan result to '/project/build/oss-review-toolkit/yarn/scan/scan-result.yml'.
19:09:29.946 [main] INFO  org.ossreviewtoolkit.cli.commands.ScannerCommand - Wrote ORT result to 'scan-result.yml' (3.34 MiB) in 680.651033ms.
The scan took 31m 40.303165377s.
Resolved issues: 0 errors, 0 warnings, 0 hints.
Unresolved issues: 10 errors, 0 warnings, 0 hints.
There are 10 unresolved issues with a severity equal to or greater than the ERROR threshold.
SUCCESS: Created file:///builds/xyz/project/build/oss-review-toolkit/yarn/scan/scan-result.yml
INFO: Evaluate scanner results with OSS review toolkit
19:09:31,985 |-INFO in ch.qos.logback.classic.LoggerContext[default] - This is logback-classic version 1.4.5
19:09:31,999 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
19:09:32,001 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [jar:file:/opt/ort/lib/helper-cli-DOCKER-SNAPSHOT.jar!/logback.xml]
19:09:32,002 |-WARN in ch.qos.logback.classic.util.DefaultJoranConfigurator@7f8633ae - Resource [logback.xml] occurs multiple times on the classpath.
19:09:32,002 |-WARN in ch.qos.logback.classic.util.DefaultJoranConfigurator@7f8633ae - Resource [logback.xml] occurs at [jar:file:/opt/ort/lib/helper-cli-DOCKER-SNAPSHOT.jar!/logback.xml]
19:09:32,002 |-WARN in ch.qos.logback.classic.util.DefaultJoranConfigurator@7f8633ae - Resource [logback.xml] occurs at [jar:file:/opt/ort/lib/cli-DOCKER-SNAPSHOT.jar!/logback.xml]
19:09:32,006 |-INFO in ch.qos.logback.core.joran.spi.ConfigurationWatchList@68c87fc3 - URL [jar:file:/opt/ort/lib/helper-cli-DOCKER-SNAPSHOT.jar!/logback.xml] is not of type file
19:09:32,073 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - Processing appender named [STDOUT]
19:09:32,073 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
19:09:32,078 |-INFO in ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
19:09:32,094 |-INFO in ch.qos.logback.classic.model.processor.RootLoggerModelHandler - Setting level of ROOT logger to WARN
19:09:32,095 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [STDOUT] to Logger[ROOT]
19:09:32,095 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.http.headers] to ERROR
19:09:32,095 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.http.wire] to ERROR
19:09:32,095 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.eclipse.jgit.internal.storage.file.FileSnapshot] to ERROR
19:09:32,095 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.analyzer.managers.Yarn2] to INFO
19:09:32,096 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.clients.fossid.FossIdRestService] to INFO
19:09:32,096 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.reporter.reporters.fossid.FossIdReporter] to INFO
19:09:32,096 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.scanner.scanners.fossid.FossId] to INFO
19:09:32,096 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.scanner.scanners.fossid.FossIdConfig] to INFO
19:09:32,096 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.scanner.scanners.fossid.FossIdUrlProvider] to INFO
19:09:32,096 |-INFO in ch.qos.logback.core.model.processor.DefaultProcessor@bc0f53b - End of configuration.
19:09:32,096 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@8d7b252 - Registering current configuration as safe fallback point
 ______________________________
/        \_______   \__    ___/ The OSS Review Toolkit, version DOCKER-SNAPSHOT.
|    |   | |       _/ |    |
|    |   | |    |   \ |    |    Running 'evaluate' as 'root' under Java 17.0.5 on Linux
\________/ |____|___/ |____|    with 32 CPUs and a maximum of 32768 MiB of memory.
Environment variables:
ORT_CONFIG_DIR = /project/ort
ORT_DATA_DIR = /root/.ort
JAVA_HOME = /opt/java/openjdk
ANDROID_HOME = /opt/android-sdk
Looking for ORT configuration in the following file:
    /project/ort/config.yml
Looking for evaluator-specific configuration in the following files and directories:
    /project/ort/copyright-garbage.yml (does not exist)
    /project/ort/license-classifications.yml
    /project/ort/curations.yml
The evaluation of 1 script(s) took 6.218887641s.
Writing evaluation result to '/project/build/oss-review-toolkit/yarn/evaluation/evaluation-result.yml'.
Resolved rule violations: 0 errors, 0 warnings, 0 hints.
Unresolved rule violations: 0 errors, 0 warnings, 0 hints.
SUCCESS: Created evaluation result at file:///builds/xyz/project/build/oss-review-toolkit/yarn/evaluation/evaluation-result.yml
INFO: Generate report with OSS review toolkit
19:09:42,815 |-INFO in ch.qos.logback.classic.LoggerContext[default] - This is logback-classic version 1.4.5
19:09:42,828 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
19:09:42,830 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [jar:file:/opt/ort/lib/helper-cli-DOCKER-SNAPSHOT.jar!/logback.xml]
19:09:42,831 |-WARN in ch.qos.logback.classic.util.DefaultJoranConfigurator@7f8633ae - Resource [logback.xml] occurs multiple times on the classpath.
19:09:42,831 |-WARN in ch.qos.logback.classic.util.DefaultJoranConfigurator@7f8633ae - Resource [logback.xml] occurs at [jar:file:/opt/ort/lib/helper-cli-DOCKER-SNAPSHOT.jar!/logback.xml]
19:09:42,831 |-WARN in ch.qos.logback.classic.util.DefaultJoranConfigurator@7f8633ae - Resource [logback.xml] occurs at [jar:file:/opt/ort/lib/cli-DOCKER-SNAPSHOT.jar!/logback.xml]
19:09:42,835 |-INFO in ch.qos.logback.core.joran.spi.ConfigurationWatchList@68c87fc3 - URL [jar:file:/opt/ort/lib/helper-cli-DOCKER-SNAPSHOT.jar!/logback.xml] is not of type file
19:09:42,901 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - Processing appender named [STDOUT]
19:09:42,901 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
19:09:42,907 |-INFO in ch.qos.logback.core.model.processor.ImplicitModelHandler - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
19:09:42,923 |-INFO in ch.qos.logback.classic.model.processor.RootLoggerModelHandler - Setting level of ROOT logger to WARN
19:09:42,924 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [STDOUT] to Logger[ROOT]
19:09:42,924 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.http.headers] to ERROR
19:09:42,924 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.http.wire] to ERROR
19:09:42,924 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.eclipse.jgit.internal.storage.file.FileSnapshot] to ERROR
19:09:42,925 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.analyzer.managers.Yarn2] to INFO
19:09:42,925 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.clients.fossid.FossIdRestService] to INFO
19:09:42,925 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.reporter.reporters.fossid.FossIdReporter] to INFO
19:09:42,925 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.scanner.scanners.fossid.FossId] to INFO
19:09:42,925 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.scanner.scanners.fossid.FossIdConfig] to INFO
19:09:42,925 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.ossreviewtoolkit.scanner.scanners.fossid.FossIdUrlProvider] to INFO
19:09:42,925 |-INFO in ch.qos.logback.core.model.processor.DefaultProcessor@bc0f53b - End of configuration.
19:09:42,925 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@8d7b252 - Registering current configuration as safe fallback point
 ______________________________
/        \_______   \__    ___/ The OSS Review Toolkit, version DOCKER-SNAPSHOT.
|    |   | |       _/ |    |
|    |   | |    |   \ |    |    Running 'report' as 'root' under Java 17.0.5 on Linux
\________/ |____|___/ |____|    with 32 CPUs and a maximum of 32768 MiB of memory.
Environment variables:
ORT_CONFIG_DIR = /project/ort
ORT_DATA_DIR = /root/.ort
JAVA_HOME = /opt/java/openjdk
ANDROID_HOME = /opt/android-sdk
Looking for ORT configuration in the following file:
    /project/ort/config.yml
Generating the 'WebApp' report in thread 'DefaultDispatcher-worker-2'...
Generating the 'AdocTemplate' report in thread 'DefaultDispatcher-worker-4'...
Generating the 'NoticeTemplate' report in thread 'DefaultDispatcher-worker-3'...
Successfully created 'WebApp' report(s) at '/project/build/oss-review-toolkit/yarn/report/scan-report-web-app.html' in 869.019117ms.
Successfully created 'NoticeTemplate' report(s) at '/project/build/oss-review-toolkit/yarn/report/NOTICE_license_notice' in 321.553184ms.
Successfully created 'AdocTemplate' report(s) at '/project/build/oss-review-toolkit/yarn/report/AsciiDoc_bom_disclosure_doc.adoc' in 322.289174ms.
Created 3 of 3 report(s) in 893.538717ms.
INFO: Created report at file:///builds/xyz/project/build/oss-review-toolkit/yarn/report
SUCCESS: Did not find any license violations.
Saving cache for successful job
00:01
Creating cache default-4-non_protected...
.gradle: found 27 matching files and directories   
Archive is up to date!                             
Created cache
Uploading artifacts for successful job
00:01
Uploading artifacts...
build/oss-review-toolkit/*: found 11 matching files and directories 
Uploading artifacts as "archive" to coordinator... 201 Created  id=751798 responseStatus=201 Created token=HaJhSFE8
Cleaning up project directory and file based variables
00:01
Job succeeded

config.yml:

ort:
  addAuthorsToCopyrights: true
  enableRepositoryPackageCurations: true 
  enableRepositoryPackageConfigurations: true

  licenseFilePatterns:
    licenseFilenames: [ 'license*' ]
    patentFilenames: [ patents ]
    rootLicenseFilenames: [ 'readme*' ]

  severeIssueThreshold: ERROR
  severeRuleViolationThreshold: ERROR

  analyzer:
    allowDynamicVersions: true
    enabledPackageManagers: [ Gradle, Yarn, NPM ]
    packageManagers:
      Yarn:
        options:
          directDependenciesOnly: true
      Gradle:
        options:
          directDependenciesOnly: true
      NPM:
        options:
          directDependenciesOnly: true

  downloader:
    allowMovingRevisions: true
    # Only used if the '--license-classifications-file' option is specified.
    includedLicenseCategories:
      - copyleft
      - copyleft-provide-sourcecode
      - weak-copyleft
      - weak-copyleft-provide-sourcecode
      - proprietary
      - permissive
      - public-domain
      - no-assertion
      - not-for-commercial-use
      - include-in-notice-file
      - include-source-code-offer-in-notice-file

  scanner:
    skipConcluded: true

    archive:

      postgresStorage:
        connection:
          url: ${POSTGRES_URL}
          schema: public
          username: ort
          password: ${POSTGRES_PASSWORD}
          sslmode: disable

    createMissingArchives: true

    storages:
      postgres:
        connection:
          url: ${POSTGRES_URL}
          schema: public
          username: ort
          password: ${POSTGRES_PASSWORD}
          sslmode: disable
        type: PACKAGE_BASED

    storageReaders: [ postgres ]
    storageWriters: [ postgres ]

    provenanceStorage:

      postgresStorage:
        connection:
          url: ${POSTGRES_URL}
          schema: public
          username: ort
          password: ${POSTGRES_PASSWORD}
          sslmode: disable
sschuberth commented 1 year ago 
package_managers:
  Yarn:
    options:
      directDependenciesOnly: "true"
  Gradle:
    options:
      directDependenciesOnly: "true"
  NPM:
    options:
      directDependenciesOnly: "true"

Unrelated to your question, but please be informed that the only package manager that currently supports directDependenciesOnly is the DotNet/ NuGet one, so these lines will have no effect.

schvvarzekatze commented 1 year ago

I have just tested only to list the package. 

[#list packages as package]

    Package: [#if package.id.namespace?has_content]             ${package.id.namespace}:[/#if]${package.id.name}:${package.id.version}

[/#list]

So it seems that the content ist available in the reporter which looks ok: Package: abab:1.0.4

I will mark this issue as resolved and close it.

schvvarzekatze commented 1 year ago 
LicenseView.ONLY_DETECTED.filter(package.license.licenses)]

has been empty.