search

dart-lang / pub-dev

The pub.dev website
https://pub.dev
BSD 3-Clause "New" or "Revised" License
789 stars 146 forks source link

Add "Filter by license" #4373

Open jayoung-lee opened 3 years ago

jayoung-lee commented 3 years ago

Add “Filter by license” to the search results page for the developers working on commercial apps and want a permissive license (e.g., BSD, MIT). In a recent user study some users mentioned that they won't even want to look at packages that miss license information.

jayoung-lee commented 3 years ago

We can consider exposing the license information to the search result, until SPDX license detection becomes available.

ChristianKleineidam commented 3 years ago

When it comes to Filter by license, GPL style licenses would also be a category that exist and that people might want to filter out.

jonasfj commented 2 years ago

We actually have a design for this now:

image

Should probably be spelled "License" and use radio buttons.

Options should probably be:

With "More" expanding to a subset of the licenses we can detect, or maybe just a high-level family of licenses.

Perhaps we can do an offline grep to identify common licenses and hard code the most popular licenses.

Analyzing licenses using unstable API end-points:

get_license() { 
  curl -s "https://pub.dev/api/packages/$1/metrics" | jq .scorecard.panaReport.licenseFile.name -r ;
}
export -f get_license

curl -s https://pub.dev/api/package-names | jq .packages[] -r | parallel -j 50 get_license > /tmp/detected-licenses

cat /tmp/detected-licenses | sort | uniq -c | sort -n -r

Yields a list of licenses as follows:

  11176 MIT
   4378 unknown
   4351 null
   4124 BSD-3-Clause
   2116 Apache-2.0
    949 BSD-2-Clause
    420 GPL-3.0
     83 MPL-2.0
     73 LGPL-3.0
     69 Unlicense
     58 AGPL-3.0
     25 BSD-2-Clause-Views
     21 LGPL-2.1
      7 WTFPL
      7 EUPL-1.2
      6 CPL-1.0
      6 CC0-1.0
      5 OFL-1.1
      4 Zlib
      4 BSL-1.0
      3 EPL-2.0
      3 CC-BY-4.0
      3 Artistic-2.0
      2 OpenSSL
      2 MulanPSL-2.0
      2 MulanPSL-1.0
      2 Hippocratic-2.1
      2 CC-BY-SA-4.0
      2 AFL-3.0
      1 X11
      1 W3C-20150513
      1 UPL-1.0
      1 MS-PL
      1 EPL-1.0
      1 CC-BY-SA-3.0
      1 CC-BY-NC-SA-3.0
      1 BSD-4-Clause

Hence, I would suggest we support, the following subset in the more menu.

  11176 MIT
   4124 BSD-3-Clause
   2116 Apache-2.0
    949 BSD-2-Clause
    420 GPL-3.0
     83 MPL-2.0
     73 LGPL-3.0
     69 Unlicense
     58 AGPL-3.0
     25 BSD-2-Clause-Views
     21 LGPL-2.1

If people really want more licenses to filter by than that, can't they just do license:<spdx-identifier> in the search query? (similar to how terms can be negated).

We could also just have radio button options for:

(we might need to consult someone who knows about licenses to ensure these categories makes sense)

And "more" could be:

From https://opensource.org/proliferation-report which identifies these as:

Licenses that are popular and widely used or with strong communities (9)

jayoung-lee commented 2 years ago

Developers want this feature because they want to see whether they can use a package for their commercial projects or not. So the first value that this feature provides is excluding all the packages with null/unknown license, and the next thing is conveying some high-level family information like BSD/MIT/LGPL/... (If we use this kind of list, we might want to keep the checkboxes instead of converting it to radio buttons, so that developers can choose all the permissive licenses, etc.)

But I agree that we need someone to verify the list of license categories. :)

jonasfj commented 2 years ago

Internally google categorizes licenses into the following types: https://opensource.google/documentation/reference/thirdparty/licenses

I don't think these categories make sense to everybody. These are looking at things from the perspective of Google being the consumer.

I think "OSI approved" Is a useful category of licenses. It generally means "open source". But whether a license falls into categories such as "permissive" and "copyleft" is a little harder determine (maybe ambiguous).

Maybe, we could invent our own license categories:

Warning "Share-alike" and "by-attribution" are terms borrowed from CreativeCommons, so it might be confusing to use those.

From the perspective of package consumer I guess there really only is 3 categories, most people are interested in:

I'm guessing that many open source projects will be willing to use anything from (b) and probably most things from (a). And most commercial enterprises will be willing to use things from (b) and maybe (c), but they might be careful with the subset from (a) that isn't part of (b) because those are likely to be copyleft (or similar).

So the question is if we can use the word "permissive" for all licenses that generally fall into category (b), or if there is some other word. I guess we can thirdparty/licenses to categories which licenses we consider permissive / notice-required / noticed-only (or whatever word we come up with for (b)).

jayoung-lee commented 2 years ago

If "(c) commercial packages" mean that the users have to pay for the package, I think it can be under another filter category, because it's not really about any licenses.

And regarding (a), would you find the list of packages with all OSI Approved licenses more useful than the second category (namely restricted & reciprocal & notice-only)? I don't have enough experience to feel strongly about anything here, but to me it feel natural to keep (a) and (b) at the same level of granularity.

But at the same time, I think the biggest interest would be "can I closed-source by app if I use this package?", which led me to think of an alternative like

License [ ] Exclude unknown licenses [ ] Exclude noncommercial licenses [ ] Exclude copyleft licenses

Maybe we can add a tooltip that explains what the label means, and shows some examples, when the users hover over..?

sigurdm commented 2 years ago

Marking commercial packages needs a way of indicating that your license is commercial (we currently cannot detect that, it is not part of the spdx database).

Within the current license detection, is there any more filters we could do except for the current "osi-approved licenses"?

sigurdm commented 2 years ago

Maybe we need filters for

leighajarett commented 2 years ago

+1 This came up in Flutter Vikings product research from multiple developers

sigurdm commented 2 years ago

@leighajarett did we get any specific details about the kind of searches people are looking for?

dkbast commented 2 years ago

Regarding what is "permissive" I just came across this overview which classifies OSI approved licenses in "risk" to commercial projects, e.g. MIT low risk, GPL high risk: https://www.synopsys.com/blogs/software-security/top-open-source-licenses/

isoos commented 2 years ago

@dkbast: I don't think we can judge or present any risk indication on any license, we'd like to keep the presented information factual and not speculative (in any direction).

dkbast commented 2 years ago

@isoos I do agree - maybe we could use the pub.dev accounts for saving custom "filters" where we can then have presets which can easily be applied (to me for a commercial project that would probably be "recently maintained" e.g. not longer than 12 months since last update and a collection of licenses)

Also at flutter vikings we had the discussion that it's currently quite difficult to have proprietary packages on pub.dev - this is another discussion, but having a "commercial license" flag would probably be great in the scope of this filter functionality

timsneath commented 2 years ago

In many larger companies, there are a set of licenses that are approved for usage without further legal review. And some licenses are banned altogether (e.g. GPL).

For that reason, I prefer your original design, @jonasfj, which presents the most popular licenses individually, rather than trying to make an editorial judgement on which licenses are "equivalent".

dkbast commented 2 years ago

Looks like we can't query for multiple licenses at the moment, so this would need to be resolved first: #6176

jonasfj commented 2 years ago

If we did https://github.com/dart-lang/pub-dev/issues/6176, then the original design would actually start to make sense. The checkboxes would have to behave differently, because it wouldn't be license:mit license:gpl-3, but rather license:mit,gpl-3.

However, I don't think this is necessarily too inconsistent or surprising.

dkbast commented 1 year ago

I understand we don't want to make any assumptions in terms of grouping licenses, but it might be worth looking into how golang handles exactly what we are discussing here:

https://pkg.go.dev/github.com/google/licensecheck#section-readme

The google go-lang team displays the following groups as a tag on pkg.go.dev which is golangs pub.dev equivalent.

As an non-googler I would assume that this topic has already been discussed internally and 2nding the way golang uses should be fine for dart?


const (
    // Unknown indicates that the license requirements are unknown.
    Unknown Type = 0

    // Unrestricted indicates that the license carries no requirements.
    // Examples: 0BSD, CC0, Unlicense.
    Unrestricted Type = 1 << iota

    // Notice indicates that the license carries notice or advertising requirements.
    // Examples: MIT, BSD, Apache 1.1, Apache 2.0
    Notice

    // ShareChanges indicates that source and binary distributions must include
    // source code for any modifications made to the original code itself,
    // made available under the same license as the original.
    // Example: LGPL.
    ShareChanges

    // ShareProgram indicates that source and binary distributions must include
    // source code for the entire program using the original code library,
    // made available under the same license as the original.
    // Example: GPL.
    ShareProgram

    // ShareServer indicates that any server program making use of the source code
    // must provide a download link for the entire source code of the server.
    // Examples: AGPL, CPAL, EUPL, SSPL.
    ShareServer

    // NonCommercial indicates that commercial use of the software is disallowed.
    // Examples: CommonsClause, CC-BY-NC-4.0, Prosperity 3.0.0.
    NonCommercial

    // Discouraged indicates that there is some legal problem with the license,
    // making it difficult to comply with or vague about what it permits.
    // Examples: Beerware, SISSL, WTFPL.
    Discouraged
)
1mtiyxz commented 1 year ago

I want to solve this issue, please assign me this issue.

dkbast commented 1 year ago

I want to solve this issue, please assign me this issue.

I believe nobody would have anything against you opening a PR with a proposal for a solution but as of now it looks like there hasn't been a decision on what the Flutter teams stance is on this, so I doubt anybody will assign it to you, for now. Opening a working PR could change this though ;)?

isoos commented 1 year ago

@dkbast @1mtiyxz: This seems like a feature where upfront consensus on the design would be required, and currently we are not there yet.

dkbast commented 10 months ago

@mit-mit would it be reasonable to assume that we could use the same logic as pkg.go.dev does as I outline above?