search

rpm-software-management / spec-cleaner

spec-cleaner
BSD 3-Clause "New" or "Revised" License
27 stars 33 forks source link

license-update.sh is broken #293

Closed marxin closed 1 year ago

marxin commented 2 years ago

Running the script results in the following stats:

 README.md                 | 412 ---------------------------------------------------------------------------------------------------------------------
 data/licenses_changes.txt | 841 +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Apparently the script is a partial copy of: https://github.com/openSUSE/obs-service-format_spec_file/blob/master/fetch-licenses.sh

that seems be working fine.

marxin commented 2 years ago

@Conan-Kudo @DimStar77 @lkocman Note rpmlint licenses are taken from spec-cleaner right now.

marxin commented 2 years ago

Slightly related to #294.

marxin commented 2 years ago

If I see correctly, the only difference is that spec-cleaner also adds licenses with +:

diff -u obs-service-format_spec_file/licenses_changes.txt ./spec-cleaner/data/licenses_changes.txt:

--- obs-service-format_spec_file/licenses_changes.txt   2021-11-29 20:29:35.856271745 +0100
+++ ./spec-cleaner/data/licenses_changes.txt    2021-11-29 20:31:17.663533879 +0100
@@ -1,45 +1,74 @@
-First line
 0BSD   0BSD
+0BSD+  0BSD+
 AAL    AAL
+AAL+   AAL+
 ADSL   ADSL
+ADSL+  ADSL+
 AFL-1.1    AFL-1.1
+AFL-1.1+   AFL-1.1+
 AFL-1.2    AFL-1.2
+AFL-1.2+   AFL-1.2+
 AFL-2.0    AFL-2.0
+AFL-2.0+   AFL-2.0+
 AFL-2.1    AFL 2.1
 AFL-2.1    AFL-2.1
 AFL-2.1    Academic Free License 2.1
+AFL-2.1+   AFL-2.1+
 AFL-3.0    AFL-3.0
+AFL-3.0+   AFL-3.0+
+AGPL-1.0-only  AGPL-1.0
 AGPL-1.0-only  AGPL-1.0-only
+AGPL-1.0-only+ AGPL-1.0-only+
+AGPL-1.0-or-later  AGPL-1.0+
 AGPL-1.0-or-later  AGPL-1.0-or-later
+AGPL-1.0-or-later+ AGPL-1.0-or-later+
 AGPL-3.0-only  AGPL-3.0
 AGPL-3.0-only  AGPL-3.0-only
 AGPL-3.0-only  AGPLv3
 AGPL-3.0-only  Affero GPL
+AGPL-3.0-only+ AGPL-3.0-only+
 AGPL-3.0-or-later  AGPL-3.0+
 AGPL-3.0-or-later  AGPL-3.0-or-later
 AGPL-3.0-or-later  AGPLv3+
 AGPL-3.0-or-later  SUSE-AGPL-3.0+
+AGPL-3.0-or-later+ AGPL-3.0-or-later+
 AMDPLPA    AMDPLPA
+AMDPLPA+   AMDPLPA+
 AML    AML
+AML+   AML+
 AMPAS  AMPAS
+AMPAS+ AMPAS+
 ANTLR-PD   ANTLR-PD
-ANTLR-PD-fallback  ANTLR-PD-fallback
+ANTLR-PD+  ANTLR-PD+
 APAFML APAFML
+APAFML+    APAFML+
 APL-1.0    APL-1.0
 APL-1.0    APL-1.0 (spdx.org/licenses)
+APL-1.0+   APL-1.0+
 APSL-1.0   APSL-1.0
+APSL-1.0+  APSL-1.0+
 APSL-1.1   APSL-1.1
+APSL-1.1+  APSL-1.1+
 APSL-1.2   APSL-1.2
+APSL-1.2+  APSL-1.2+
 APSL-2.0   APSL-2.0
+APSL-2.0+  APSL-2.0+
 Abstyles   Abstyles
+Abstyles+  Abstyles+
 Adobe-2006 Adobe-2006
+Adobe-2006+    Adobe-2006+
 Adobe-Glyph    Adobe-Glyph
+Adobe-Glyph+   Adobe-Glyph+
 Afmparse   Afmparse
+Afmparse+  Afmparse+
 Aladdin    Aladdin
+Aladdin+   Aladdin+
 Apache-1.0 Apache-1.0
+Apache-1.0+    Apache-1.0+
 Apache-1.1 ASL 1.1
 Apache-1.1 Apache 1.1 (Indiana University)
 Apache-1.1 Apache-1.1
+Apache-1.1+    Apache-1.1+
 Apache-2.0 ASL 2.0
 Apache-2.0 ASL2
 Apache-2.0 ASLv..
@@ -57,6 +86,7 @@
 Apache-2.0 Apache-2.0
 Apache-2.0 The Apache Software License
 Apache-2.0+    ASLv2.0+
+Apache-2.0+    Apache-2.0+
 Apache-2.0+    SUSE-Apache-2.0+
 Artistic-1.0   Artistic
 Artistic-1.0   Artistic 1.0
@@ -64,13 +94,18 @@
 Artistic-1.0   Artistic License ..
 Artistic-1.0   Artistic license
 Artistic-1.0   Artistic-1.0
-Artistic-1.0 OR GPL-1.0-or-later   Perl License
+Artistic-1.0 or GPL-1.0+   Perl License
+Artistic-1.0+  Artistic-1.0+
 Artistic-1.0+  SUSE-Artistic-1.0+
 Artistic-1.0-Perl  Artistic-1.0-Perl
+Artistic-1.0-Perl+ Artistic-1.0-Perl+
 Artistic-1.0-cl8   Artistic-1.0-cl8
+Artistic-1.0-cl8+  Artistic-1.0-cl8+
 Artistic-2.0   Artistic 2.0
marxin commented 2 years ago

The script is broken because it parses HTML output from 2 w3m command invocations. I really believe we should take canonical licenses from https://github.com/openSUSE/obs-service-format_spec_file.

marxin commented 2 years ago

For rpmlint, I prefer moving the script to the repository: https://github.com/rpm-software-management/rpmlint/pull/767 and taking the licenses from the obs-service-format_spec_file repo.

kubat-david commented 1 year ago

I tried running license-update.sh but it crashed

$ ./license-update.sh
sort: cannot read: license_exceptions.ntxt: No such file or directory
rm: cannot remove 'license_exceptions.ntxt': No such file or directory

It turns out that the w3m -dump -cols 1000 http://spdx.org/licenses/exceptions-index.html | grep "License Exception Text" command on line 38 returns nothing so the license_exceptions.ntxt never gets created. I'm wondering if the contents of the URL has changed?

marxin commented 1 year ago

Yes, it was renamed here https://github.com/openSUSE/obs-service-format_spec_file/commit/862eb75add9a0f1d7a7425a29f2e7719c60d3bdc.

kubat-david commented 1 year ago

I see, thank you. So instead of http://spdx.org/licenses/exceptions-index.html the script should be using https://raw.githubusercontent.com/spdx/license-list-data/master/json/exceptions.json Is that correct?

marxin commented 1 year ago

I guess so, parsing a JSON file would be much more appropriate. It's actually what obs-service-format_spec_file does: https://github.com/openSUSE/obs-service-format_spec_file/blob/master/fetch-licenses.sh#L14