How do I actually run this tool?

[S3j5b0]

Hi, im trying to set up a gtihub action, and using your tool for scanning.

I have this yml file:

name: "Update and check dependency data"

on:
  push:
    branches: [main]

jobs:
  licensed:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - uses: jonabc/licensed-ci@v1
      - uses: jonabc/setup-licensed@v1
        with:
         version: 2.x
      - run: licensed list

Where I would like to run "licensed status" on my project. But of course this does not work, and the command "licensed" cannot be found.

How do I get to use the commands from your library? I dont really understand the examples on your marketplace page, none of them seem to be actually using the comands, they just run a few scripts and so on

[jonabc]

:wave: @S3j5b0 the issue here is that you've got @jonabc/licensed-ci running before @jonabc/setup-licensed. setup-licensed installs a distributable licensed+ruby executable to make it available for later actions. licensed-ci expects that licensed has been installed and can be found on $PATH, so these two actions are made to work together well. You can also run gem install licensed if ruby is available, or if you have a Gemfile that includes licensed then bundle install.

I dont really understand the examples on your marketplace page, none of them seem to be actually using the comands, they just run a few scripts and so on

I'm not sure what you mean, the examples on the marketplace page all use licensed-ci. The examples show how to use licensed based on two different methods for installing licensed itself. Is there something else you'd like to see that's missing?

[S3j5b0]

Thanks a lot for the answer!

I now have this setup:

name: "Update and check dependency data"

name: "Update and check dependency data"

on:
  push:
    branches: [main]

jobs:
  licensed:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - uses: jonabc/setup-licensed@v1
        with:
         version: 2.x

      - uses: jonabc/licensed-ci@v1
      - run: licensed list

Which fails with: ENOENT: no such file or directory, access '.licensed.yml'

Should i have a licensed.yml file somewhere?

If i completely remove 'licensed-ci@v1' then it gives me:

`find_config': Licensed configuration not found in /home/runner/work/malware/malware (Licensed::Configuration::LoadError)

[jonabc]

yes, licensed requires a configuration file to work. the default configuration file location is <repo root>/.licensed.yml and the config file documentation gives some information about the different configuration options.

Most of the configuration is optional, the only thing you should need to set to get up and running with licensed status to work is the allowed licensed list. The example configuration shows this configuration with a descriptive comment.

[S3j5b0]

Great, now it runs as it should

The only problem now, is that licensed status has this output: dependencies checked, 0 errors found.

And also when I delete my license file, the same output persists, how do I get it to check the license and dependencies of licenses?

[jonabc]

root: if you are in a git repository you don't need to set this, it will default to the repository root cache_path: this is the path that the metadata files that licensed-ci creates are stored. it defaults to a .licenses folder at the root of the repo, again you don't need to set it unless you want to override that value source_path: this is the path to the code you're finding licenses for and defaults to the directory that licensed is run from if not set. when using the bundler source this needs to be the directory path that contains Gemfile and Gemfile.lock.

The "happy path" is to have .licensed.yml at the root of the repository and to run licensed commands from the root of the repository, and not set root and cache_path. If Gemfile and Gemfile.lock are at the root of the repo, then you don't need to set source_path either. If they are not at the root of the repository, you'll need to set source_path to a path relative to the repository root of the folder containing Gemfile and Gemfile.lock

To preempt another possible point of confusion, licensed looks at dependencies that are installed on the local disk. You'll need to ensure that you've installed dependencies locally before running any licensed commands - i.e. bundle install or whatever local scripting you have to accomplish the same.

[S3j5b0]

My project is golang, would this mean that I would have to run something like:

uses: actions/setup-go@v2 run: go get "github.com/swaggo/http-swagger"

before calling licensed? running this toy example doesnt seem to have any effect

A gemfile looks to be something used for managing ruby dependencies. With this not being a ruby project, i'm guessing this wont help me much

[jonabc]

My project is golang, would this mean that I would have to run something like: uses: actions/setup-go@v2 run: go get "github.com/swaggo/http-swagger" before calling licensed?

Yep 👍 . In that case the issue may be that your source path is not set up properly. For go projects you'll need to set source path to the entry point of your app. A good way to test this is running go list -e -json -deps . in a directory and seeing what is output. That's the command licensed uses internally to determine what dependencies are available

[S3j5b0]

the file that I use to start the project is just in the root dir that would be the entry point right?, . running 'go list -e -json -deps .' gives me a really long list of dependencies in json-format

So the setup looks like this now:

name: "Update and check dependency data"

on:
  push:
    branches: [main]

jobs:
  licensed:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-go@v2
      - uses: actions/checkout@v1
      - uses: jonabc/setup-licensed@v1
        with:
         version: 2.x

      - run: go get "github.com/gorilla/mux"
      - run: go get "github.com/sirupsen/logrus"

      - run: licensed status

[jonabc]

👍 yeah the root dir would be the source_path in that case. I don't see anything immediately obvious in your action YML, maybe change the following to get all dependencies rather than just those two?

From:

      - run: go get "github.com/gorilla/mux"
      - run: go get "github.com/sirupsen/logrus"

To:

      - run: go get

Can you also paste the licensed configuration file contents here?

[S3j5b0]

good idea! here is licensed:

name: 'My application'

sources:
  bower: true
  bundler: false

allowed:
  - mit
  - apache-2.0
  - bsd-2-clause
  - bsd-3-clause
  - cc0-1.0
  - isc

ignored:
  bundler:
    - some-internal-gem

  bower:
    - some-internal-package

reviewed:
  bundler:
    - bcrypt-ruby

  bower:
  - classlist # public domain
  - octicons

also, just in case, this is the repo :D: https://github.com/S3j5b0/malware

[jonabc]

sources:
  bower: true
  bundler: false

If this isn't causing the issue, it soon will be! The sources key indicates which sources should be enabled. The important thing to note here is that if any source is set to true, all unlisted sources will be set to false by default. Otherwise, if no sources are explicitly set to true, unlisted sources default to true, docs.

In your case, try deleting every configuration key except for allowed name defaults to the repo name sources will default to using whatever is detected as available ignored specifies which dependencies will be ignored entirely and should be used to exclude 1st party dependencies. reviewed specifies dependencies whose licenses aren't automatically detected or aren't specified as allowed, but which you've otherwise decided are ok to use in the project

[S3j5b0]

progress! it does this for every of my dependencies!:

malware.go.github.com/prometheus/client_golang/prometheus filename: /home/runner/work/malware/malware/.licenses/go/github.com/prometheus/client_golang/prometheus.dep.yml

• cached dependency record not found

• malware.go.github.com/prometheus/client_golang/prometheus/internal filename: /home/runner/work/malware/malware/.licenses/go/github.com/prometheus/client_golang/prometheus/internal.dep.yml
  • cached dependency record not found

* malware.go.github.com/prometheus/client_golang/prometheus/promhttp
  filename: /home/runner/work/malware/malware/.licenses/go/github.com/prometheus/client_golang/prometheus/promhttp.dep.yml
    - cached dependency record not found

* malware.go.github.com/prometheus/client_golang/prometheus/push
  filename: /home/runner/work/malware/malware/.licenses/go/github.com/prometheus/client_golang/prometheus/push.dep.yml
    - cached dependency record not found

* malware.go.github.com/prometheus/client_model/go
  filename: /home/runner/work/malware/malware/.licenses/go/github.com/prometheus/client_model/go.dep.yml
    - cached dependency record not found

should I configure it to look for golang libraries?

[jonabc]

nope! that's expected if you've only running licensed status. the important part of the message is cached metadata - this tool caches metadata files into your repo with the licensed cache command and checks those cached files for existence, freshness, correctness and compliance with licensed status.

licensed is meant as an end to end solution for projects to manage their OSS licensing over time and does so by caching metadata about dependencies at development time, and checking that everything is 🆗 as part of a CI process. See a blog post when licensed went public for a bit more detail.

Specifically, this tool (jonabc/licensed-ci) provides an opinionated action workflow that automates the caching and checking of license metadata in a repo. It

1. runs licensed status to see whether any updates to cached metadata is needed
2. If there are any failures, run licensed cache to update metadata
3. if there are any cached metadata updates in the repo, either commit them to the tested branch or open a PR with the changes. The action it takes depends on whether the actions workflow configuration is set to push or branch

[S3j5b0]

ah ok, so the next logical step would be to run "licensed cache"? im not really sure how I stop the action from failing

[S3j5b0]

i Just added "licensed cache" right before "licensed status", it gives me this for 60 dependencies:

• malware.go.google.golang.org/protobuf/internal/encoding/tag filename: /home/runner/work/malware/malware/.licenses/go/google.golang.org/protobuf/internal/encoding/tag.dep.yml

  • license needs review: other

• malware.go.google.golang.org/protobuf/internal/encoding/text filename: /home/runner/work/malware/malware/.licenses/go/google.golang.org/protobuf/internal/encoding/text.dep.yml

  • license needs review: other

• malware.go.google.golang.org/protobuf/internal/errors filename: /home/runner/work/malware/malware/.licenses/go/google.golang.org/protobuf/internal/errors.dep.yml

  • license needs review: other

60 seems a bit excessive, is it likely that my project is just that messed up licensewise, or can something else be configured? Some of them are golangs native stuff, it seems unlikely that there would be issues there

[jonabc]

if you're not using the action from this repo jonabc/licensed-ci in your workflow, I'd recommend on your local machine to download/install licensed and run licensed cache to populate the local cache of files. Once those files are committed in the repo you'll start seeing different output from licensed status, though there could still be listed error depending on how well the license parsing/classification tool that github/licensed uses is able to detect each dependencies license.

Alternatively, in your action workflow replace - run: licensed status with - uses: jonabc/licensed-ci@v1 and push the change up to your repo in GitHub. The action should run and result in caching licenses, committing them to your branch and running a status check on them.

[jonabc]

@S3j5b0 those errors from googles dependencies being classified as other occur because Google includes an additional patents grant alongside the 3 clause BSD license, which causes the license classification tool to not know for certain whether to classify the dependency as bsd-3-clause or not.

I am not a lawyer and this is not legal advice on whether or not these dependencies are ok to use in your project, based on your own compliance concerns. That said, if you want to just quiet these warnings then you can add each dependency as reviewed in your configuration file.

# These dependencies have licenses not on the `allowed` list and have been reviewed.
# They will be cached and checked, but will not raise errors or warnings for a
# non-allowed license.  Dependencies on this list will still raise errors if
# license text cannot be found for the dependency.
reviewed:

For you this list would look something like

reviewed:
  go:
    - google.golang.org/protobuf/internal/encoding/tag
    - google.golang.org/protobuf/internal/encoding/text
    - ...

That can be tedious for a large number of dependencies, so licensed also supports glob patterns. Again this is not legal advice so please review your licensing compliance needs separately and be clear on whether the risk of a false positive on marking something as reviewed is ok for you.

reviewed:
  go:
    - google.golang.org/protobuf/internal/*

[S3j5b0]

I cant really get the "reviewed" thing to work. I have a dependency called mux that causes problems,even though it uses a simple mit license

 * malware.go.github.com/gorilla/mux
  filename: /home/runner/work/malware/malware/.licenses/go/github.com/gorilla/mux.dep.yml
    - license needs review: other

Ive tried ignoring it by: reviewed:

  go:
    - go.github.com/gorilla/mux
    - go.github.com/gorilla/mux*
    - /go/github.com/gorilla/mux.dep.yml
    - ./go/github.com/gorilla/mux.dep.yml

But none of it has any effect, am I writing the adress wrong?

EDIT:

it now works for gorilla lib, with this format

github.com/gorilla/mux*

But for other libs, this does not work?

[jonabc]

@S3j5b0 the format in the output <project>.<source>.<dependency name>, where the dependency name is what the reviewed list should contain. The dependency name for go dependencies match the import path of that dependency. In your example you should use github.com/gorilla/mux in that list.

[S3j5b0]

Yes but it seems to only work for some dependencies, for example When i get this output:

* malware.go.golang.org/x/crypto/blowfish
  filename: /home/runner/work/malware/malware/.licenses/go/golang.org/x/crypto/blowfish.dep.yml
    - license needs review: other

And I now try to add it to reviewed:

golang.org/x/*

But this has no effect. Trying a lot of variants doesnt work either

[jonabc]

The entries there use Ruby's Dir.glob to implement pattern matching. * will only match items at the current directory/path level, where ** will match one or more directories.

If you wanted to mark the entirety of golang.org/x as reviewed you'd need both golang.org/x/* and golang.org/x/**/*. If you are setting up licensed for true compliance purposes, these are risky patterns to use as any new dependencies will automatically be matched to the reviewed pattern even if they are not compliant with your list of allowed licenses.

[jonabc]

@S3j5b0 I'm going to close this as it sounds like maybe you've gotten everything figured out. Let me know if there are any remaining concerns