search

aboutcode-org / scancode-toolkit

:mag: ScanCode detects licenses, copyrights, dependencies by "scanning code" ... to discover and inventory open source and third-party packages used in your code. Sponsored by NLnet project https://nlnet.nl/project/vulnerabilitydatabase, the Google Summer of Code, Azure credits, nexB and others generous sponsors!
https://aboutcode.org/scancode/
2.15k stars 553 forks source link

Incorrect RPM license detected #2767

Open pombredanne opened 3 years ago

pombredanne commented 3 years ago

In http://rpmfind.net/linux/fedora/linux/development/rawhide/Everything/aarch64/os/Packages/g/GConf2-devel-3.2.6-31.fc36.aarch64.rpm we get this and the "unknown" is incorrect:

headers:
    -   tool_name: scancode-toolkit
        tool_version: 30.0.0
        options:
            input:
                - GConf2-devel-3.2.6-31.fc36.aarch64.rpm
            --license: yes
            --license-text: yes
            --license-text-diagnostics: yes
            --package: yes
            --yaml: '-'
...
packages:
    -   type: rpm
        namespace:
        name: GConf2-devel
        version: 3.2.6-31.fc36
...
        license_expression: (lgpl-2.0-plus AND gpl-2.0-plus) AND unknown
        declared_license: LGPLv2+ and GPLv2+
        notice_text:
...
adityasangave commented 2 years ago

I'd like to work on this

pombredanne commented 2 years ago

@adii21-Ux great!

the resolution could include:

or both.

The first step would be a small unit test IMHO.

adityasangave commented 2 years ago

Ok Got it

adityasangave commented 2 years ago

I guess the reason for unknown license expression is we have 10 rules for LGPLv2+ AND GPLv2+ and no rule file for (LGPLv2+ AND GPLv2+) AND another expression so I guess we have to add another rule, I don't see any code changes here, what do you think?

pombredanne commented 2 years ago

@adii21-Ux this is an approach... another one may be to craft a list of RPM license symbols and use the packaged.licensing code to parse RPM license tags as a license expression using these symbols(such as LGPLv2+ , GPLv2+)

pombredanne commented 2 years ago

you can try to go first with a few rules for a start.

nighteous commented 2 years ago

whats the status of this issue? seems like it hasnt been resolved yet

adityasangave commented 2 years ago

I am working on it

pombredanne commented 2 years ago

BTW here is a mapping of Fedora license ids https://github.com/fedora-modularity/check_modulemd/blob/55555db5796d92311acea65273f7536fd3a9663e/valid_sw_licenses.txt by @bgoncalv ... well not a mapping bu a list. A mapping to normalized licenses would need to be built from this!

Valid Software Licenses according to https://fedoraproject.org/wiki/Licensing:Main

AAL # Attribution Assurance License Abstyles # Abstyles License Adobe # Adobe Systems Incorporated Source Code License Agreement ADSL # Amazon Digital Services License AFL # Academic Free License Afmparse # Afmparse License AGPLv1 # Affero General Public License 1.0 AGPLv3 # Affero General Public License 3.0 AGPLv3+ # Affero General Public License 3.0 or later AGPLv3 with exceptions # Affero General Public License 3.0 with Zarafa trademark exceptions AMDPLPA # AMD's plpa_map.c License AML # Apple MIT License AMPAS BSD # Academy of Motion Picture Arts and Sciences BSD APAFML # Adobe Postscript AFM License App-s2p # App::s2p License APSL 2.0 # Apple Public Source License 2.0 ARL # Aspell-ru License Artistic 2.0 # Artistic 2.0 Artistic clarified # Artistic (clarified) ASL 1.0 # Apache Software License 1.0 ASL 1.1 # 4Suite Copyright License ASL 1.1 # Apache Software License 1.1 ASL 1.1 # MX4J License ASL 1.1 # Neotonic Clearsilver License ASL 1.1 # QuickFix License ASL 2.0 # Apache Software License 2.0 Bahyph # Bahyph License Barr # Barr License Beerware # Beerware License BeOpen # BeOpen Open Source License Agreement Version 1 Bibtex # Bibtex License BitTorrent # BitTorrent License Boost # Boost Software License Borceux # Borceux license BSD # BSD License (no advertising) BSD # BSD License (two clause) BSD # Creative Commons BSD BSD # Cryptix General License BSD # Eclipse Distribution License 1.0 BSD # Metasploit Framework License (post 2006) BSD Protection # BSD Protection License BSD with advertising # BSD License (original) BSD with advertising # NRL License BSD with attribution # BSD with attribution CATOSL # Computer Associates Trusted Open Source License 1.1 CC0 # Creative Commons Zero 1.0 Universal CDDL # Common Development Distribution License CeCILL-B # CeCILL-B License CeCILL-C # CeCILL-C License CeCILL # CeCILL License v1.1 CeCILL # CeCILL License v2 CNRI # CNRI License (Old Python) Condor # Condor Public License Copyright only # Copyright Attribution Only CPAL # CPAL License 1.0 CPL # Common Public License CRC32 # CRC32 License Crossword # Crossword License Crystal Stacker # Crystal Stacker License Cube # Cube License diffmark # diffmark license DMIT # Docbook MIT License DOC # DOC License Dotseqn # Dotseqn License DSDP # DSDP License dvipdfm # dvipdfm License DWPL # DO WHATEVER PUBLIC LICENSE ECL 1.0 # Educational Community License 1.0 ECL 2.0 # Educational Community License 2.0 eCos # eCos License v2.0 EFL 2.0 # Eiffel Forum License 2.0 eGenix # eGenix.com Public License 1.1.0 Entessa # Entessa Public License EPICS # EPICS Open License EPL # Eclipse Public License 1.0 ERPL # Erlang Public License 1.1 EU Datagrid # EU Datagrid Software License EUPL 1.1 # European Union Public License 1.1 Eurosym # Eurosym License Fair # Fair License FSFUL # FSF Unlimited License FSFULLR # FSF Unlimited License (with License Retention) FTL # Freetype License Giftware # Giftware License GL2PS # GL2PS License Glide # 3dfx Glide License Glulxe # Glulxe License gnuplot # gnuplot License GPL+ # GNU General Public License (no version) GPL+ # GNU General Public License v1.0 or later GPL+ or Artistic # Perl License GPLv1 # GNU General Public License v1.0 only GPLv2+ # Creative Commons GNU GPL GPLv2 # GNU General Public License v2.0 only GPLv2+ # GNU General Public License v2.0 or later GPLv2 or Artistic # Perl License (variant) GPLv2+ or Artistic # Perl License (variant) GPLv2 with exceptions # Fedora Directory Server License GPLv2 with exceptions # GNU General Public License v2.0 only, with Classpath exception GPLv2 with exceptions # GNU General Public License v2.0 only, with font embedding exception GPLv2+ with exceptions # GNU General Public License v2.0 or later, with Classpath exception GPLv2+ with exceptions # GNU General Public License v2.0 or later, with font embedding exception GPLv2 with exceptions # MySQL License GPLv3 # GNU General Public License v3.0 only GPLv3+ # GNU General Public License v3.0 or later GPLv3 with exceptions # GNU General Public License v3.0 only, with Classpath exception GPLv3 with exceptions # GNU General Public License v3.0 only, with font embedding exception GPLv3+ with exceptions # GNU General Public License v3.0 or later, with Classpath exception GPLv3+ with exceptions # GNU General Public License v3.0 or later, with font embedding exception GPL+ with exceptions # GNU General Public License (no version), with Classpath exception GPL+ with exceptions # GNU General Public License (no version), with font embedding exception HaskellReport # Haskell Language Report License HSRL # Henry Spencer Reg-Ex Library License IBM # IBM Public License IJG # Independent JPEG Group License ImageMagick # ImageMagick License iMatix # iMatix Standard Function Library Agreement Imlib2 # Imlib2 License Intel ACPI # Intel ACPI Software License Agreement Interbase # Interbase Public License ISC # ISC License (Bind, DHCP Server) Jabber # Jabber Open Source License JasPer # JasPer License JPython # JPython License (old) Julius # Julius License Knuth # Knuth License Latex2e # Latex2e License LBNL BSD # Lawrence Berkeley National Labs BSD variant license Leptonica # Leptonica License LGPLv2+ # Creative Commons GNU LGPL LGPLv2+ # GNU Lesser General Public License (no version) LGPLv2 # GNU Lesser General Public License v2 (or 2.1) only LGPLv2+ # GNU Lesser General Public License v2 (or 2.1) or later LGPLv2+ or Artistic # Perl License (variant) LGPLv2 with exceptions # FLTK License LGPLv2+ with exceptions # GNU Lesser General Public License v2 (or 2.1) or later, with exceptions LGPLv2 with exceptions # GNU Lesser General Public License v2 (or 2.1), with exceptions LGPLv2+ with exceptions # Qwt License 1.0 LGPLv3 # GNU Lesser General Public License v3.0 only LGPLv3+ # GNU Lesser General Public License v3.0 or later LGPLv3 with exceptions # GNU Lesser General Public License v3.0 only, with exceptions LGPLv3+ with exceptions # GNU Lesser General Public License v3.0 or later, with exceptions Lhcyr # Lhcyr License libtiff # libtiff License LLGPL # Lisp Library General Public License Logica # Logica Open Source License LOSLA # LEGO Open Source License Agreement LPL # Lucent Public License (Plan9) LPPL # LaTeX Project Public License MakeIndex # MakeIndex License mecab-ipadic # mecab-ipadic license midnight # midnight License MirOS # MirOS License MIT # Adobe Glyph List License MIT # CMU License (BSD like) MIT # enna License MIT # feh License MIT # Historical Permission Notice and Disclaimer MIT # MIT license (also X11) MIT # mpich2 License MITNFA # MIT +no-false-attribs license MIT # SGI Free Software License B 2.0 MIT # Standard ML of New Jersey License MIT with advertising # Enlightenment License (e16) MIT with advertising # Nunit License mod_macro # mod_macro License Motosoto # Motosoto License MPLv1.0 # FreeImage Public License MPLv1.0 # Mozilla Public License v1.0 MPLv1.1 # CUA Office Public License Version 1.0 MPLv1.1 # Mozilla Public License v1.1 MPLv2.0 # Mozilla Public License v2.0 MS-PL # Microsoft Public License MS-RL # Microsoft Reciprocal License MTLL # Matrix Template Library License Mup # Mup License Naumen # Naumen Public License NCSA # NCSA/University of Illinois Open Source License NetCDF # NetCDF license Netscape # Celtx Public License (CePL) Netscape # Netscape Public License Newmat # Newmat License Newsletr # Newsletr License NGPL # Nethack General Public License NLPL # No Limit Public License Nmap # Nmap License Nokia # Nokia Open Source License NOSL # Netizen Open Source License Noweb # Noweb License OGL # Open Government License OML # Open Market License OpenLDAP # OpenLDAP License OpenPBS # OpenPBS License OpenSSL # OpenSSL License OReilly # OReilly License OSL 1.0 # Open Software License 1.0 OSL 1.1 # Open Software License 1.1 OSL 2.0 # Open Software License 2.0 OSL 2.1 # Open Software License 2.1 OSL 3.0 # Open Software License 3.0 Par # Par License Phorum # Phorum License PHP # PHP License v3.0 PlainTeX # PlainTeX License Plexus # Plexus Classworlds License PostgreSQL # PostgreSQL License psfrag # psfrag License psutils # psutils License Public Domain # Public Domain Python # Python License Qhull # Qhull License QPL # Q Public License Rdisc # Rdisc License REX # REX License RiceBSD # Rice BSD Romio # Romio License RPSL # RealNetworks Public Source License V1.0 Rsfs # Rsfs License Ruby # Ruby License Saxpath # Saxpath License SCEA # SCEA Shared Source License SCRIP # SCRIP License Sendmail # Sendmail License Sequence # Sequence Library License SISSL # Sun Industry Standards Source License Sleepycat # Sleepycat Software Product License SLIB # SLIB License SNIA # SNIA Public License 1.1 softSurfer # softSurfer License SPL # Sun Public License STMPL # SciTech MGL Public License SWL # Scheme Widget Library (SWL) Software License Agreement TCGL # Trusted Computing Group License TCL # TCL/TK License Teeworlds # Teeworlds License TGPPL # Transitive Grace Period Public Licence Threeparttable # Threeparttable License TMate # TMate Open Source License Tolua # Tolua License TORQUEv1.1 # TORQUE v2.5+ Software License v1.1 TOSL # Trusster Open Source License TPDL # Time::ParseDate License TPL # Thor Public License TTWL # Text-Tabs+Wrap License UCAR # UCAR License UCD # Unicode Character Database Terms Of Use Unicode # Unicode License Unlicense # Unlicense Vim # Vim License VNLSL # Vita Nuova Liberal Source License VOSTROM # VOSTROM Public License for Open Source VSL # Vovida Software License v. 1.0 W3C # W3C Software Notice and License Webmin # Webmin License Wsuipa # Wsuipa License WTFPL # Do What The F*ck You Want To Public License wxWidgets # wxWidgets Library License Xerox # Xerox License xinetd # xinetd License xpp # XPP License XSkat # XSkat License YPLv1.1 # Yahoo Public License v 1.1 Zed # Zed License Zend # Zend License v2.0 zlib with acknowledgement # zlib/libpng License with Acknowledgement zlib # zlib/libpng License ZPLv1.0 # Zope Public License v 1.0 ZPLv2.0 # Zope Public License v 2.0 ZPLv2.1 # Zope Public License v 2.1

Another avenue could be https://github.com/org-metaeffekt/metaeffekt-universe ... but I cannot see ids explicitly from Fedora there ... @karsten-klein do you track these specifically ?

See also https://github.com/maxhbr/LDBcollector @maxhbr you would not happen to have built a mapping of ScanCode license keys to Fedora license ids by chance?

bgoncalv commented 2 years ago

I'm not sure if helps, but rpminspect contains a list of supported Licenses in Fedora: https://github.com/rpminspect/rpminspect-data-fedora/blob/master/licenses/fedora.json

pombredanne commented 2 years ago

@bgoncalv It will surely help as between the SPDX ids and the text URL for the ones that do not have an SPDX id we should be able to map these alright. We have likely most if not all these licenses somehow already detectable as text and notices, and this would get us the Fedora id and name I stumbled on this too which is similar yet subtly different: https://gitlab.com/redhat/centos-stream/ci-cd/rpminspect-data-centos/-/blob/master/licenses/centos.json