Breaking change after 7.2.0 ?

[sailro]

Hi,

We are using license finder to audit all our projects. We capture stdout to find new licences to check.

For maven-based projects, we use license_finder --no-prepare --enabled-package-managers=maven --maven-options=package --quiet --maven-include-groups=true

Since 7.2.0 released today, the output is polluted by messages coming from our test suite, so breaking everything.

Did something changed regarding this ? Can we pass extra options to license finder to either ignore tests (like -Dmaven.test.skip=true -DskipTests=true we commonly use when invoking maven directly) or simply ignore test output ?

Thank you !

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this. Unfortunately, the Pivotal Tracker project is private so you may be unable to view the contents of the story.

The labels on this github issue will be updated when the story is started.

[sailro]

okay so using :

license_finder --no-prepare --enabled-package-managers=maven --maven-options="package -Dmaven.test.skip=true -DskipTests=true" --quiet --maven-include-groups=true seems to fix the issue.

So not sure if it's really a bug in license finder but the behavior definitively changed between 7.1.0 and 7.2.0

[xtreme-shane-lattanzio]

Hey @sailro! There were quite a bit of maven changes for this release added by @rhuitl. I added more to the changelog to try to highlight a bit more. I am not sure what exactly caused the logs to change but im glad the workaround resolves it!

[rhuitl]

Hmm, there was a patch version update for Maven, and this: https://github.com/pivotal/LicenseFinder/commit/0da6d30b9a306cc8a7c88cc55f13acd0eb3f10b3#diff-b958f840c0529a7d4cc364d0e80267591bd50ed398b563d8d530513a5c9a3869R61. Can you see from your logs which Maven call produced the output? Can you make a minimal reproduction case maybe?

The actual call to Maven to extract the licenses did not change.

[ttomaszewski]

After upgrading to 7.2.0 we experienced the following Zip::Error with Maven. Reverted back to 7.1.0, which is working fine.

LicenseFinder::Maven: is active for '/home/circleci/repo/service'
/home/circleci/.rvm/gems/ruby-3.3.1/gems/rubyzip-2.3.2/lib/zip/file.rb:106:in `initialize': File /home/circleci/.m2/repository/io/netty/netty-resolver-dns-native-macos/4.1.107.Final/netty-resolver-dns-native-macos-4.1.107.Final-jakarta.jar not found (Zip::Error)
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/rubyzip-2.3.2/lib/zip/file.rb:121:in `new'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/rubyzip-2.3.2/lib/zip/file.rb:121:in `open'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package_utils/license_files.rb:46:in `candidates_from_zip'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package_utils/license_files.rb:41:in `candidate_files_and_dirs'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package_utils/license_files.rb:29:in `paths_of_candidate_files'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package_utils/license_files.rb:19:in `find'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package_utils/license_files.rb:11:in `find'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/packages/maven_package.rb:32:in `license_files'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package.rb:130:in `licensing'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package.rb:124:in `activations'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package.rb:120:in `licenses'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/decision_applier.rb:60:in `with_approval'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/decision_applier.rb:34:in `block in apply_decisions'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/decision_applier.rb:32:in `map'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/decision_applier.rb:32:in `apply_decisions'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/decision_applier.rb:8:in `initialize'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/core.rb:79:in `new'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/core.rb:79:in `decision_applier'
        from /home/circleci/.rvm/rubies/ruby-3.3.1/lib/ruby/3.3.0/forwardable.rb:234:in `any_packages?'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/license_aggregator.rb:17:in `block in any_packages?'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/license_aggregator.rb:15:in `map'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/license_aggregator.rb:15:in `any_packages?'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/cli/main.rb:121:in `action_items'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/thor-1.3.1/lib/thor/command.rb:28:in `run'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/thor-1.3.1/lib/thor/invocation.rb:127:in `invoke_command'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/thor-1.3.1/lib/thor.rb:527:in `dispatch'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/thor-1.3.1/lib/thor/base.rb:584:in `start'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/bin/license_finder:6:in `<top (required)>'
        from /home/circleci/.rvm/gems/ruby-3.3.1/bin/license_finder:25:in `load'
        from /home/circleci/.rvm/gems/ruby-3.3.1/bin/license_finder:25:in `<main>'
        from /home/circleci/.rvm/gems/ruby-3.3.1/bin/ruby_executable_hooks:22:in `eval'
        from /home/circleci/.rvm/gems/ruby-3.3.1/bin/ruby_executable_hooks:22:in `<main>'

[xtreme-shane-lattanzio]

@rhuitl @ttomaszewski I pushed a quick fix for this https://github.com/pivotal/LicenseFinder/commit/ba920d1670cf308df1cc6939d15bc0a62ad14486

Im not sure what the jakarta jar is so this at least maintains functionality like the previous release if needed

[akilleen]

Hello,

To add on to this, I am getting a similar error, just with a different zip file...

LicenseFinder::Maven: is active for '/home/runner/work/repo/repo'
/var/lib/gems/3.0.0/gems/rubyzip-2.3.2/lib/zip/file.rb:106:in `initialize': File /home/runner/.m2/repository/io/netty/netty-resolver-dns-native-macos/4.1.90.Final/netty-resolver-dns-native-macos-4.1.90.Final.jar not found (Zip::Error)
    from /var/lib/gems/3.0.0/gems/rubyzip-2.3.2/lib/zip/file.rb:121:in `new'
    from /var/lib/gems/3.0.0/gems/rubyzip-2.3.2/lib/zip/file.rb:121:in `open'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:46:in `candidates_from_zip'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:41:in `candidate_files_and_dirs'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:29:in `paths_of_candidate_files'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:19:in `find'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:11:in `find'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/packages/maven_package.rb:32:in `license_files'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package.rb:130:in `licensing'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package.rb:124:in `activations'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package.rb:120:in `licenses'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:60:in `with_approval'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:34:in `block in apply_decisions'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:32:in `map'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:32:in `apply_decisions'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:8:in `initialize'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/core.rb:79:in `new'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/core.rb:79:in `decision_applier'
    from /usr/lib/ruby/3.0.0/forwardable.rb:232:in `acknowledged'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:51:in `block in aggregate_packages'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:49:in `each'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:49:in `flat_map'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:49:in `aggregate_packages'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:11:in `dependencies'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/cli/main.rb:161:in `report'
    from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor/command.rb:28:in `run'
    from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor/invocation.rb:127:in `invoke_command'
    from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor.rb:527:in `dispatch'
    from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor/base.rb:584:in `start'
    from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/bin/license_finder:6:in `<top (required)>'
    from /usr/local/bin/license_finder:25:in `load'
    from /usr/local/bin/license_finder:25:in `<main>'

I also noticed that in my maven cache that file is either missing or just named differently.

% ls -l /Users/killead/.m2/repository/io/netty/netty-resolver-dns-native-macos/4.1.90.Final                
total 104
-rw-r--r--@ 1 killead  staff    267 Jun 11 13:40 _remote.repositories
-rw-r--r--@ 1 killead  staff  18724 Mar 14  2023 netty-resolver-dns-native-macos-4.1.90.Final-osx-x86_64.jar
-rw-r--r--@ 1 killead  staff     40 Jun 11 13:40 netty-resolver-dns-native-macos-4.1.90.Final-osx-x86_64.jar.sha1
-rw-r--r--@ 1 killead  staff  17999 Mar 14  2023 netty-resolver-dns-native-macos-4.1.90.Final.pom
-rw-r--r--@ 1 killead  staff     40 Jun 11 13:40 netty-resolver-dns-native-macos-4.1.90.Final.pom.sha1

For now I have downgraded back down to 7.1.0 and it works.

[knoxg]

^ I just encounted the same zip file error, although on a different file:

/var/lib/gems/3.0.0/gems/rubyzip-2.3.2/lib/zip/file.rb:106:in `initialize': File /root/.m2/repository/com/google/javascript/closure-compiler-rhino/v20140407/closure-compiler-rhino-v20140407.jar not found (Zip::Error)
        from /var/lib/gems/3.0.0/gems/rubyzip-2.3.2/lib/zip/file.rb:121:in `new'
        from /var/lib/gems/3.0.0/gems/rubyzip-2.3.2/lib/zip/file.rb:121:in `open'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:47:in `candidates_from_zip'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:41:in `candidate_files_and_dirs'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:29:in `paths_of_candidate_files'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:19:in `find'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:11:in `find'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/packages/maven_package.rb:32:in `license_files'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package.rb:130:in `licensing'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package.rb:124:in `activations'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package.rb:120:in `licenses'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:60:in `with_approval'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:34:in `block in apply_decisions'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:32:in `map'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:32:in `apply_decisions'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:8:in `initialize'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/core.rb:79:in `new'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/core.rb:79:in `decision_applier'
        from /usr/lib/ruby/3.0.0/forwardable.rb:232:in `acknowledged'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:51:in `block in aggregate_packages'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:49:in `each'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:49:in `flat_map'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:49:in `aggregate_packages'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:11:in `dependencies'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/cli/main.rb:161:in `report'
        from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor/command.rb:28:in `run'
        from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor/invocation.rb:127:in `invoke_command'
        from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor.rb:527:in `dispatch'
        from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor/base.rb:584:in `start'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/bin/license_finder:6:in `<top (required)>'
        from /usr/local/bin/license_finder:25:in `load'
        from /usr/local/bin/license_finder:25:in `<main>'

I was able to get past this by patching license_files.rb so that it doesn't attempt to open non-existing files. Can ignore the commented-out puts line, it was just there to add some logging to see what was happening:

--- /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb.orig        2024-07-11 02:02:00.826350646 +0000
+++ /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb     2024-07-11 02:01:40.415265274 +0000
@@ -43,6 +43,8 @@
     end

     def candidates_from_zip
+      # puts install_path.to_s
+      return [] if !File.exist?(install_path.to_s)
       Zip::File.open(install_path.to_s) do |zip_file|
         zip_file.glob(CANDIDATE_PATH_WILDCARD, File::FNM_EXTGLOB)
       end

[knoxg]

Following up from that last comment, I also needed to apply the following patch in order to generate a CSV report:

--- /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/notice_files.rb.orig 2024-07-11 06:04:55.697786572 +0000
+++ /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/notice_files.rb      2024-07-11 06:37:45.816310897 +0000
@@ -43,6 +43,7 @@
     end

     def candidates_from_zip
+      return [] if !File.exist?(install_path.to_s)
       Zip::File.open(install_path.to_s) do |zip_file|
         zip_file.glob("*/#{CANDIDATE_PATH_WILDCARD_STRICT}", File::FNM_EXTGLOB)
       end