search

spdx / change-proposal

Repository for change proposal for the SPDX project
3 stars 3 forks source link

Change Proposal: AdditionRef- #4

Open jlovejoy opened 2 years ago

jlovejoy commented 2 years ago

Please review a new Change Proposal for adding an ExceptionRef here: https://github.com/spdx/change-proposal/blob/main/proposals/ExceptionRef.md submitted by @zvr

To be reviewed by SPDX-legal and SPDX-tech team.

Please indicate in this issue, if you think this proposal should move forward or not at this time, which a brief supporting statement for your decision.

If there is support to move forward, then we will move the discussion to the appropriate team for discussion related implementation.

goneall commented 2 years ago

+1 on the change proposal.

One suggestion - we should also support external document references to ExceptionRef's in the same way we support external document references to LicenseRef's.

jlovejoy commented 2 years ago

I think this would be a good addition. I like the distinction of ExceptionRef- rather than allowing LicenseRef- on both sides of the WITH operator, as it seems to avoid confusion.

Implications for spec:

richardfontana commented 2 years ago

+1 consistent with what I said a couple of years ago. However, I urge that this not be tied to the existing narrow, FSF-doctrinally-influenced SPDX notion of what an "exception" is, which does not match up with real-world uses of FOSS/FOSS-like licenses coupled with additional terms. An "ExceptionRef" construct for use with WITH would be useful to represent terms that are clearly or possibly not characterizable as "additional permissions" or "exceptions" in the orthodox GPL sense. See the examples in my comment on the issue linked above (though I have no recollection of where I encountered them).

Update: I think @pombredanne's counterproposal is preferable because it seems to address (in this one limited context) my basic objection to SPDX's tying itself to a narrow, normative understanding of what an "exception" is, instead of attempting to devise an objective way of representing the actual kinds of license term supplementation that occurs in practice, which would be more consistent with how SPDX treats license representation. I am still +1 for the proposal because there is a pressing need for user-defined supplementary term identifiers and this would get us there provided the use of the ExceptionRef- construct with non-FSF-normative types of additional terms is not treated as improper.

Further update: Having just dealt with a real-world case of some terms that could be characterized either as an orthodox GPL exception or as an alternative (non-FOSS) license, and in the process concluding that we needed to have two data files in the Fedora license data repository (with two different SPDX expressions) to represent both interpretations, I am now mildly -1 on this proposal and instead support @pombredanne's suggestion for allowing LicenseRef- to be used in WITH subexpressions.

Essentially, the concern that 'LicenseRef-1 WITH LicenseRef-2' (to use @Pizza-Ria's example) is undesirable is IMO misplaced because in some situations you might want to use LicenseRef-2 outside of a WITH context.

@pombredanne says:

there is nothing in the SPDX standards to define the validity of license elements within a license expression, nor does it make sense for SPDX to cover that. There is no specific meaning attached to the exception identifiers to reliably determine if a license id on the SPDX license list is for a license or an exception short of getting more data beyond the identifier alone.

This is related to why I find SPDX's treatment of exceptions confusing given what I thought was a "just the facts" principle. The SPDX spec says of WITH: "Sometimes a set of license terms apply except under special circumstances." And the SPDX exception list says: "These exceptions grant an exception to a license condition or additional permissions beyond those granted in a license; they are not stand-alone licenses." This implies SPDX is making some kind of nontrivial legal interpretation of what are often informal and confusing terms. In reality, we have things in FOSS/pseudo-FOSS characterized as 'exceptions' that are probably not really "granting an exception" in any meaningful sense (see for example the old license of Liberation Fonts). We also might have highly restrictive terms that have been constructed to look like an orthodox GPL exception. SPDX should not deciding whether supplementary terms, whether or not characterized by the licensor as an 'exception', are actually additional permissions or something else.

Pizza-Ria commented 2 years ago

+1 - Seems to align with already documented exceptions provided at https://spdx.org/licenses/exceptions-index.html and offer an easier way to identify them especially considering the final note in the writeup of the ambiguity of only using "with" (i.e., "However, this would result in an impossibility to distinguish between Licenses and Exceptions, therefore a construct like LicenseRef-1 WITH LicenseRef-2 might be valid (if it were similar to Apache-2.0 WITH LLVM-exception) or not (if it were similar to Apache-2.0 WITH BSD-3-Clause)."

silverhook commented 2 years ago

I agree with both @zvr ’s suggestion and @richardfontana ’s comment above.

(as pretty much always, likely won’t be able to make the call due to a clash)

goneall commented 2 years ago

Providing a link to an existing issue in the SPDX spec proposing same: https://github.com/spdx/spdx-spec/issues/153

pombredanne commented 2 years ago

-1 I do not agree with this proposed change.

This is consistent with my previous comment on the topic https://github.com/spdx/spdx-spec/issues/153#issuecomment-989699504 a few years back. Having a separate prefix for an identifier did not make sense then and it does not make more sense today.

The latest guideline is to "use a single LicenseRef- to represent the entire license terms, including the exception." As the representative of one of the primary open source projects that actively collects, defines and uses exceptions that are not listed in the SPDX license list, it would wreak havoc on the existing user base for ScanCode licenses where we consistently have used a LicenseRef prefix in these cases. These users represent a large number of the open source tools in the space. We track about 100 such exceptions that are not on the SPDX list at this time.

I also wrote back then: https://github.com/spdx/spdx-spec/issues/153#issuecomment-989699504 : “My guess is that the majority, if not all, of the LicenseRef- used in license expressions as an exception are from ScanCode.” Introducing an ExceptionRef prefix would break tools and validators including tools that merely parse and validate license expressions. The syntax would need to be updated https://spdx.github.io/spdx-spec/SPDX-license-expressions/

Counter proposal

The only thing that seems reasonable is to allow the use of LicenseRef values in the WITH part of a license expression. Some key supporting points:

  1. This approach is already used in practice

  2. It requires only a minor relaxation of the spec and validation tools and does not introduce new constructs nor would it break existing license expression handling tools.

Note that this approach is already considered in https://github.com/spdx/change-proposal/blob/main/proposals/ExceptionRef.md#not-introducing-new-prefix. A concern expressed here is: “However, this would result in an impossibility to distinguish between Licenses and Exceptions, therefore a construct like LicenseRef-1 WITH LicenseRef-2 might be valid (if it were similar to Apache-2.0 WITH LLVM-exception) or not (if it were similar to Apache-2.0 WITH BSD-3-Clause).”

This is already the case because there is nothing in the SPDX standards to define the validity of license elements within a license expression, nor does it make sense for SPDX to cover that. There is no specific meaning attached to the exception identifiers to reliably determine if a license id on the SPDX license list is for a license or an exception short of getting more data beyond the identifier alone.

Existing exception identifiers are not prefixed or suffixed consistently in the license list, so adding an arbitrary convention for non-SPDX listed exceptions does not make sense. If someone mistakenly records “Apache-2.0 WITH BSD-3-Clause” instead of “Apache-2.0 AND BSD-3-Clause”, most people would understand the meaning. A good way to think of this is that WITH is a special case of AND to alert you to the fact that the next license in the license expression is an exception.

Using a consistent convention to include the word "exception" in the license id would be a good thing and is probably all that is needed. See suggestion by @sschuberth in https://github.com/spdx/spdx-spec/issues/153#issuecomment-989735668

swinslow commented 2 years ago

I haven't had a chance to think about this in detail, but based on an initial review of the proposal and the comments above, here are my thoughts:

Overall: I'm +1 to the original proposal. Open to some modifications to it, but as a baseline I think there needs to be some way to support user-defined exceptions within the existing SPDX license expression syntax and license data model.

Exceptions on the license list itself:

For purposes of the exceptions actually on the SPDX License List, to respond to @richardfontana I do think there has been value in keeping it to the narrow definition of "additional permissions" or "exceptions."

Briefly, by limiting it to those exceptions, you can look at an expression using WITH, like GPL-2.0-only WITH GCC-exception-2.0, and know that you at least have the rights in GPL-2.0-only itself. A WITH expression might indicate that fewer restrictions apply, but at a minimum the recipient can elect to disregard the WITH and know that they could just use it under the original license. By contrast, if exceptions could be any modifier -- e.g. if you could have GPL-2.0-only WITH Commons-Clause -- then this presumption no longer holds.

So I think there's value in it having worked this way in the past. All that said, I could also be open to an argument that the value there is outweighed by replacing "exceptions" with a more general "modifier" concept. But I think that's outside the scope of this particular change proposal.

If the question is just whether custom ExceptionRef- exceptions should be required to be strict "exceptions" under this definition, I'm less tied to that and I could be okay with flexibility here. If the ExceptionRef- prefix is present, then as with LicenseRef- licenses, all bets are off regarding stability, structure, etc., since there's no community review / curation for custom identifiers and their content.

Prefix for custom exceptions:

I'm -1 to the comments above indicating that LicenseRef- should be used as the prefix for BOTH licenses and exceptions.

Licenses are different from exceptions to licenses (and/or modifiers to licenses). They are defined differently on the SPDX license list (licenses vs. exceptions); in the location and permitted syntax for the license-list-XML templates (schema); and in the RDF objects that represent them in the underlying SPDX data model (licenses vs. exceptions).

Changing the license expression syntax to permit LicenseRef-'s to be used in WITH expressions would, I suspect, require treating licenses and license exceptions as identical in the underlying SPDX data model. I think this would break existing assumptions for tools and communities that have relied on the expression syntax and object model. I also think this could introduce confusion by conflating AND/OR and WITH. I don't really see the benefit that would outweigh these.

I would suggest that the discussion here be focused on the specific proposal at hand, and whether to adopt this (as-is or with minor/moderate changes) in the near term. If people want to propose more significant changes to the underlying SPDX data model, license expression syntax, etc., then I would suggest it's probably appropriate for those to be submitted as a separate change proposal so that it can be similarly considered and discussed.

jlovejoy commented 1 year ago

NOTE - we will discuss this Change Proposal in terms of 1) proceeding with it or not (yes or no); and 2) if yes, then specifics as to implementation.

On Thursday, Jan 12th at noon US eastern time using the Legal team dial-in. I will also send an email reminder to both the legal and tech mailing lists.

jlovejoy commented 1 year ago

Having re-read this thread, I think there are a few tangents and I want to make sure we stay focused on our call Thursday. To use an illustrative example:

If I found a package licensed under Apache-2.0 (which is on the SPDX License List), and it also applied the Commons Clause (which is not on the SPDX License List):

1) Status quo: as the spec currently stands (see section 10) I would represent this as: LicenseRef-somename in 10.1 License identifier field and capture the total of the text of Apache-2.0 and the Commons Clause in 10.2 Extracted text field

2) This Change Proposal proposes to instead, be able to use: 'Apache-2.0 WITH ExceptionRef-CommonsClause' in 10.1 License identifier field and capture the text of only the Commons Clause in 10.2 Extracted text file

2a) An alternative implementation option (suggested by @pombredanne and which, if I understand correctly, is what Scancode has already been doing?) would to be able to use: 'Apache-2.0 WITH LicenseRef-CommonsClause' in 10.1 License identifier field and capture the text of only the Commons Clause in 10.2 Extracted text field.

vargenau commented 1 year ago

Will this proposed change be in 2.3.1, 2.4 or 3.0?

jlovejoy commented 1 year ago

Discussed on Jan 12th legal call https://github.com/spdx/meetings/blob/main/legal/2023-01-12.md

Everyone in favor of allowing more extensible ability to represent things not on SPDX License List on right side of WITH operator.

Still need to decide on implementation and meaning.

Next steps:

To revisit once PRs are posted after FOSDEM - probably mid-Feb

pombredanne commented 1 year ago

Dennis/Phillippe to prepare PR for how spec would change if extended use of LicenseRef to allow that to represent things on right side of WITH

I drafted this PR https://github.com/spdx/spdx-spec/pull/829/

zvr commented 1 year ago

During all the discussions, it seems that there was support for having a way to reference things not in the SPDX Licene List.

There were two alternatives of what these "things" could be:

  1. License exceptions, that are not (yet) present in the SPDX License List; or
  2. Additional text to the original license text, that might (or might not) change the terms in any way.

I don't think we had consensus on what was the goal. Therefore, I've prepared two alternative PRs for both of these cases:

  1. ExceptionRef syntax in https://github.com/spdx/spdx-spec/pull/838
  2. AdditionRef syntax in https://github.com/spdx/spdx-spec/pull/839

I can join one of the next Legal Team calls and discuss them, if needed.

goneall commented 1 year ago

My 2 cents:

swinslow commented 1 year ago

Thanks for this and apologies for my slow reply. Here are my thoughts on the proposals from @pombredanne and @zvr following from the prior meeting about this change proposal:

So overall, I'm +1 to move forward with #839 in concept, though will take a closer look at the specific wording in that PR.

Pizza-Ria commented 1 year ago

Still a hesitant +1 - I'm afraid this is going to result in a lot of clutter but hopefully it will be more restrained than my fears.

pombredanne commented 1 year ago

I stand by -1 as explained in https://github.com/spdx/change-proposal/issues/4#issuecomment-1284356573 Note that there is nothing that distinguishes an exception from a license id string today. So there is no precedent to introduce this for additional exceptions and I see this only as a source of un-needed churn.

zvr commented 1 year ago

Without repeating all the pros and cons of all alternatives, I'm in favor of AdditionRef as proposed in https://github.com/spdx/spdx-spec/pull/839

richardfontana commented 1 year ago

Leaving aside my earlier comments: Of the two proposals, I prefer AdditionRef-. I agree with @swinslow that "it would be helpful to have a way to represent user-defined pieces of text that are intended as additions to licenses -- including those that are not formally "exceptions" under a strict reading".

I would assume that, as proposed, AdditionRef- can be used for all of the following:

swinslow commented 1 year ago

I would assume that, as proposed, AdditionRef- can be used for all of the following:

@richardfontana Yes -- my expectation is that AdditionRef- is appropriate for all of these use cases. The only criteria for AdditionRef- in my mind (and stated informally) is that it would be used for text that is added onto another separate license, and is not itself a standalone license. The substantive contents of what that additional text means, or what effect it has, is irrelevant, and it is not limited to traditional "exceptions".

pombredanne commented 1 year ago

@swinslow

The only criteria for AdditionRef- in my mind (and stated informally) is that it would be used for text that is added onto another separate license, and is not itself a standalone license. The substantive contents of what that additional text means, or what effect it has, is irrelevant, and it is not limited to traditional "exceptions".

How is this different from a LicenseRef then? I still have not seen a convincing argument of why using a LicenseRef for these would not be good enough.

There is no way that any meaning attached to the content of what's in such a file can be enforced, so using a special new prefix does not help with anything I can fathom.

swinslow commented 1 year ago

@pombredanne In my view:

LicenseRef- is a fully-fledged license and can be used in expressions as parts of AND, OR, etc. By contrast, AdditionRef- would be used only on the right-hand side of a WITH expression.

My view is that these are two distinct things, with different semantic and syntactic meanings. Therefore, it is worthwhile to use different prefixes for them.

Note that this reflects the way that SPDX has handled licenses vs. exceptions for at least as long as the Exceptions List has existed. This long pre-dates my own involvement with SPDX, but from looking at prior versions of the spec it appears that at least since the license expression syntax was introduced in SPDX 2.0 in 2015, there has been a dichotomy between "licenses" and "things that are added to licenses." So this is not a new concept.

There is no way that any meaning attached to the content of what's in such a file can be enforced, so using a special new prefix does not help with anything I can fathom.

That's true, but only in the sense that this applies to any other free text field in the specification. We can say "the intent for this field or this class is X", and people may or may not comply with it, but that doesn't stop us from establishing a specification for how things should be used.

jlovejoy commented 1 year ago

I'm +1 to AdditionRef- for the reasons already stated. Most notably, I like that AdditionRef- can be used for any of the things @richardfontana listed above, some of which wouldn't qualify under the criteria for an "exception" on the SPDX License List. This consistent with how LicenseRef- is intended for things not found on the SPDX License List (and implicitly, may not be eligible for inclusion), AdditionRef- can/should be defined in a similar way.

I would note that none of the PRs, in my opinion, are complete. As noted in a comment above, I foresee this change impacting the Appendix, Section 10 of the spec and possibly other collateral - this will need to be fleshed out, but given the complete change in format for 3.0, I won't fault @pombredanne or @zvr for submitting a partial PR.

@pombredanne - I'm a bit perplexed by your comments above - you cite the current state of the spec to "use a single LicenseRef- to represent the entire license terms, including the exception." But then seem to say that Scancode has used LicenseRef-x WITH LicenseRef-y which is not valid with the current spec for some time?

As someone who look at many SPDX license expressions, I would find LicenseRef-x WITH LicenseRef-y to be confusing. AND and OR are differently and distinctly defined than WITH, in terms of what can be on the right and left side of said operators. To allow LicenseRef- to be used for something that is not a standalone license and on the right side of WITH is counter intuitive. Using a differently named "ref" keeps things consistent with the original concept of the operators.

As for messing things up - I recall that when we adopted the operators, there was some concern that people would use the + with licenses for which that should not apply and there was nothing to stop them from doing so. There was a bit of discussion around this. To my knowledge, this has not proved to be a problem.

Point being if we articulate the intention for the use of AdditionRef- well enough, which I think involves changes to section 10 as well, then we can likely avoid any such issues.

karsten-klein commented 1 year ago

I personally thing that AdditionRef- is not going far enough. I would propose to change and thereby generalize it to ModifierRef- instead. So its semantically <License> WITH <Modifier>. An addition, for me, is already a specific type of modifier. See also change proposal https://github.com/spdx/change-proposal/issues/6.

Regards, Karsten

zvr commented 1 year ago

Since it's been a month and some more arguments were made, let me state again that I am personally in favor of AdditionRef as proposed in spdx/spdx-spec#839

@jlovejoy I'd like to hear what other parts of the spec are impacted, in your view. I thought I had went through all of the text and made a comprehensive PR. In short, "license expressions" are only explained and expanded in Annex D and that's the only thing that gets changed.

@karsten-klein it was decided that each ChangeProposal should have a specific scope to stop us discussing ever-expanding issues. I understand your proposal is much more general, changing the way we model licenses and affecting the spec, the license expression syntax, the License List, the inclusion principles, and possibly other aspects. This is not what is being discussed here.

For this Change Proposal, would you prefer:

jlovejoy commented 1 year ago

@zvr - yes, license expressions are only explained in Annex D, but I would think this would need to be addressed in Section 10 as well, or maybe a separate section. Assuming the text of the additional text for the AdditionalRef- need be captured

swinslow commented 1 year ago

Following from the discussions in this thread, the mailing list and on the January call and today's call, we concluded on a few items:

More details included in the minutes from today's meeting, which I'll post in the meetings repo shortly.

Next steps are for me to revise the licensing-profile branch in the spdx-3-model repo to implement these changes. I'll follow up here and will close the issue after that is done.

swinslow commented 1 year ago

This has now been merged into the initial SPDX 3.0 licensing model:

Classes:

Properties:

Ancillary materials such as the License Expression Syntax annex will still need to be updated. Not yet clear to me how the annexes will be carried over from SPDX 2.3 to 3.0, so I'll leave this issue open as a placeholder until this is complete

fviernau commented 1 year ago

The question whether to use LicenseRef- or AdditionRef- for additons to a license to me boils down to the question:

Should the information whether the text is an addition or a stand-alone license be encoded as part of the ID string or not? (So far, no properties / interpretations of a text referred to by the ID are encoded as part of the ID string)

drawbacks I see with AdditionRef:

Looking just at ScanCode: It offers meta-data it associates with the IDs, which includes license category and also an is_exception flag. If you use LicenseRef- for additions, for ScanCode you can just go to that meta-data and figure out whether that LicenseRef- is an exception or not. In fact any tool is free to do that. And if someone's interpretation differs from the meta-data of ScanCode, he is free to use its own.

Given that I believe it'd be less of a hazzle to just allow LicenseRef- and be save as it makes no interpretation at all. (At least for SPDX 2.x, as I believe using custom IDs for exceptions is desperately missing)

maxhbr commented 1 year ago

FYI: as visible from the linked PR above, some tools start supporting LicenseRef- on the right side of a WITH, and just warn that it is not compliant . There is a need for this feature, and the longer a decision takes, the more non-consistent implementations might be out there.

swinslow commented 1 year ago

@fviernau @maxhbr Just for clarity, the decision to require AdditionRef- as the prefix for custom license additions was already concluded in April 2023, and is not up for revision. There is no decision remaining pending about which prefix will be used.

This issue is just remaining open as a placeholder pending determination about where ancillary materials (e.g., the Annexes from SPDX 2.3 and earlier) will be inserted for SPDX 3.0, so that the annexes relating to the License Expression Syntax can be updated to reflect this decision.