Utilize the OSI API's to automatically populate the isOsiApproved flag in the listed license

goneall commented 6 years ago

https://api.opensource.org/licenses/ can access the SPDX license ID and OSI status. This can be used to do one of the following:

Fill in the OSI approved text on spdx.org/licenses based on JavaScript and real time access to the OSI API and deprecate the isOsiApproved attribute in the license list XML
Set the value for osiApproved in the listed licenses based on the OSI API information at the time the license list is generated and deprecate the isOsiApproved attribute in the license list XML
Continue to use the isOsiApproved attribute in the license list XML, but generate a warning if the OSI API does not agree with the isOsiApproved XML attribute value.

goneall commented 6 years ago

Suggested by @wking on SPDX tech email dist. list

goneall commented 6 years ago

My current preference is solution #2 since #1 depends on the OSI API site being available. The frequency of license updates should be sufficient to keep things in sync.

goneall commented 6 years ago

From @wking

On Fri, Oct 13, 2017 at 09:20:56PM +0000, goneall wrote:

https://api.opensource.org/licenses/ can access the SPDX license ID and OSI status.

The API is backed by OpenSourceOrg/licenses, and there's still a non-canonical warning up there 1. See also OpenSourceOrg/licenses#47. Hopefully serious SPDX interest (and assistance? I have some open PRs over there) will encourage them to push through to something authoritative.

Fill in the OSI approved text on spdx.org/licenses based on JavaScript and real time access to the OSI API and deprecate the isOsiApproved attribute in the license list XML

I like this way for public HTML, although I think we'll want to go with (2) if we distribute text/plain or similar versions of the list. While there is a risk that the OSI site could go down, I'm fine just telling consumers that the site is down. With the JavaScript approach, you wouldn't have to update the vOld page as the OSI approves new licenses.

But if we plan on periodically rebuilding pages for all versions of the license list to pick up new approvals, then baking the approval status into the built pages is fine.

goneall commented 6 years ago

Moved from https://github.com/spdx/tools/issues/111

goneall commented 3 years ago

@swinslow @jlovejoy Any opinion on this issue? Should we remove the XML OSI Approved from the XML and use the API? At a minimum, I think we should generate a warning.

goneall commented 3 years ago

The following warnings are generated when comparing the OSI metadata to the license-list-XML metadata on OSI approved:

    License AFL-2.1 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License AFL-1.2 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License AFL-1.1 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License AFL-2.0 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License 0BSD is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License AGPL-3.0-only is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License AGPL-3.0-or-later is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License APSL-1.0 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License APSL-1.1 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License Artistic-1.0-cl8 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License APSL-1.2 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License Artistic-1.0-Perl is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License BSD-2-Clause-Patent is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License BSD-1-Clause is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License BSD-3-Clause-LBNL is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License CAL-1.0-Combined-Work-Exception is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License CAL-1.0 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License CERN-OHL-P-2.0 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License CERN-OHL-S-2.0 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License CERN-OHL-W-2.0 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License EPL-2.0 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License EUPL-1.2 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License GPL-2.0-only is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License GPL-2.0-or-later is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License GPL-2.0+ is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License CECILL-2.1 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License GPL-3.0+ is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License GPL-3.0-only is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License GPL-3.0-with-GCC-exception is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License GPL-3.0-or-later is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License LGPL-2.0-only is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License LGPL-2.0-or-later is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License LGPL-2.1-only is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License LGPL-2.1-or-later is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License LGPL-2.0+ is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License LGPL-2.0 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License LGPL-3.0-or-later is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License LGPL-3.0-only is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License LGPL-3.0+ is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License LiLiQ-Rplus-1.1 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License LiLiQ-R-1.1 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License LiLiQ-P-1.1 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License MIT-Modern-Variant is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License MPL-2.0-no-copyleft-exception is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License LGPL-2.1+ is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License MulanPSL-2.0 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License OFL-1.1-RFN is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License OFL-1.1-no-RFN is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License OLDAP-2.8 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License OSET-PL-2.1 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License OSL-2.0 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License PHP-3.01 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License UCL-1.0 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License Unlicense is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License UPL-1.0 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License Unicode-DFS-2016 is not included in the OSI metadata, but is marked as OSI approved in the License XML
    License wxWindows osiApproved is set to true by OSI, but is not marked as OSI approved in the License XML
    License MIT-0 is not included in the OSI metadata, but is marked as OSI approved in the License XML

goneall commented 3 years ago

The vast majority of the warnings are due to inconsistencies in the OSI data. The repo hosting the API may no longer be maintained.

See https://github.com/OpenSourceOrg/licenses/issues/62 for the list of inconsistencies.

goneall commented 3 years ago

Warnings not related to OSI data inconsistencies include:

The GPLxx-or-later, GPLxx-only ID's not being included in the OSI data
Some APSL license versions are not on the OSI list - this could be intentional, so I added https://github.com/spdx/license-list-XML/issues/1328
Some AFL license versions are not on the OSI list - this could be intentional, so I added https://github.com/spdx/license-list-XML/issues/1327
wxWindows is deprecated and replaced by an exception. I added a pull request to resolve: https://github.com/spdx/license-list-XML/pull/1329

I did not create a PR for the following remaining warnings. I think they can be safely ignored - but @swinslow and/or @jlovejoy should review just to be sure:

OSL-2.0 is not on the OSI website, but OSL-1.0, and 2.1 are. I don't think this is intentional, so I didn't create an issue
Artistic-1.0-cl8 and Artistic-1.0-Perl identifiers are not listed on the OSI website - I recall several discussions on this, so I'm just ignoring these warnings
MIT-Modern-Variant
MPL-2.0-no-copyleft-exception
OFL-1.1-RFN
OFL-1.1-no-RFN

goneall commented 3 years ago

Summary - the following SPDX ID's with a warning should be ignored:

 0BSD
 AGPL-3.0-only
 AGPL-3.0-or-later
 Artistic-1.0-cl8
 Artistic-1.0-Perl
 BSD-2-Clause-Patent
 BSD-1-Clause
 BSD-3-Clause-LBNL
 CAL-1.0-Combined-Work-Exception
 CAL-1.0
 CERN-OHL-P-2.0
 CERN-OHL-S-2.0
 CERN-OHL-W-2.0
 EPL-2.0
 EUPL-1.2
 GPL-2.0-only
 GPL-2.0-or-later
 GPL-2.0+
 CECILL-2.1
 GPL-3.0+
 GPL-3.0-only
 GPL-3.0-with-GCC-exception
 GPL-3.0-or-later
 LGPL-2.0-only
 LGPL-2.0-or-later
 LGPL-2.1-only
 LGPL-2.1-or-later
 LGPL-2.0+
 LGPL-2.0
 LGPL-3.0-or-later
 LGPL-3.0-only
 LGPL-3.0+
 LiLiQ-Rplus-1.1
 LiLiQ-R-1.1
 LiLiQ-P-1.1
 MIT-Modern-Variant
 MPL-2.0-no-copyleft-exception
 LGPL-2.1+
 MulanPSL-2.0
 OFL-1.1-RFN
 OFL-1.1-no-RFN
 OLDAP-2.8
 OSET-PL-2.1
 OSL-2.0
 PHP-3.01
 UCL-1.0
 Unlicense
 UPL-1.0
 Unicode-DFS-2016
 MIT-0

jlovejoy commented 3 years ago

I don't quite have my head around all the warnings that should be ignored (will need to think and look more closely, as well as go into attic of memory...) But generally speaking I am in favor of using the OSI data and your #2 proposal IF: 1) we can confirm the OSI is maintaining this; and 2) perhaps they can add some of the missing stuff so we don't have to "ignore" various warnings

Maybe we should wait to see if you get a response on the issue you logged in due time. If not, then reach out to OSI board directly?

goneall commented 3 years ago

Maybe we should wait to see if you get a response on the issue you logged in due time. If not, then reach out to OSI board directly?

How about we reach out to the OSI board in 2 weeks if we don't hear back.

Haven't heard anything yet - but its only been a few days.

goneall commented 3 years ago

There have been some updates from OSI in their repo - cross referencing them here:

There is a discussion on the process for accepting pull requests for fixes to their data in this PR: https://github.com/OpenSourceOrg/licenses/pull/65
The AFL licenses issue doesn't look like it will be addressed soon - the issue tracking the inconsistency was closed, but I did request it to be re-opened: https://github.com/OpenSourceOrg/licenses/issues/66
I added a PR for a script which copies SPDX license list data to the OSI repo: https://github.com/OpenSourceOrg/licenses/pull/68

spdx / LicenseListPublisher

Utilize the OSI API's to automatically populate the isOsiApproved flag in the listed license #20