mlcommons / mobile_app_open

Mobile App Open
https://mlcommons.org/en/groups/inference-mobile/
Apache License 2.0
37 stars 19 forks source link

Collecting licenses of the 3rd party components we used #873

Open freedomtan opened 1 month ago

freedomtan commented 1 month ago

Let's try to list all the 3rd party components and corresponding licenses we used.

RSMNYS commented 1 month ago

I've used anchor/syft to generate the SBOM (Software list of materials). More info is here: https://github.com/anchore/syft Next, by using sbom-utility: https://github.com/CycloneDX/sbom-utility, I've generated the list of the licenses. The only drawback is the duplication, which we can resolve.

Please find attached sbom, and licenses files licenses.txt sbom.json

freedomtan commented 1 month ago

@RSMNYS Please try to use syft and sbom-util, so that we can automatically generate "licenses.txt" in our current CI/CD. Please try to send a PR for this.

RSMNYS commented 3 weeks ago

here are the instructions how to generate sbom and then convert to the list of licenses:

Install syft

curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin

Run syft from the directory

syft -o cyclonedx-json . --exclude './react' > sbom-cyclonedx.json --verbose

convert sbom to the licenses list

./sbom-utility license list -i /Volumes/work/Programming/ScopicSoftware/MLCommons/mobile/sbom-cyclonedx.json --summary --quiet -o licenses.txt

While this works to some degree, I found another tool that is used for the licenses check: scancode-toolkit ( https://github.com/nexB/scancode-toolkit). It scans through the files and trying to find the licenses in those. And looks like it identifies more licenses than the approach above. The output is in the json, so we can use some script to extract the info.

RSMNYS commented 3 weeks ago

@freedomtan @anhappdev I found yet another tool, I'm using a free version of it, and results are really good. And we can use the free version for our needs. I'm attaching the list that the tool is able to generate. Furthermore we can use the API to fully automate it. Please check.

https://fossa.com

mlcommons_licenses.txt

freedomtan commented 2 weeks ago

@RSMNYS please try to group packages with the same license together and then talk to Scott (@nathanw-mlc I don't know Scott's handle).

nathanw-mlc commented 2 weeks ago

I don't know Scott's handle

@swasson488

freedomtan commented 1 week ago

Let's ping Scott @swasson488 after @RSMNYS finish the grouping of licenses.

RSMNYS commented 6 days ago

Hi guys! Here is the grouped licenses. Prepared the python script which parses the output from fossa service, and groups packages by license type. grouped_packages_licenses.txt

freedomtan commented 5 days ago

@RSMNYS to check with some legal guys to see if we can further reduce the file size.

swasson488 commented 5 days ago

Thanks, folks. Do let me know if there's any further consolidation possible. Otherwise, this is still very helpful and yeah, lots of different components with at least two different licenses.

Scott

On Tue, May 14, 2024 at 12:16 AM Koan-Sin Tan @.***> wrote:

@RSMNYS https://github.com/RSMNYS to check with some legal guys to see if we can further reduce the file size.

— Reply to this email directly, view it on GitHub https://github.com/mlcommons/mobile_app_open/issues/873#issuecomment-2109302688, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFC3WA5BPNH2DIQKEDLCXKTZCGM4FAVCNFSM6AAAAABFWL2WSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBZGMYDENRYHA . You are receiving this because you were mentioned.Message ID: @.***>

anhappdev commented 4 days ago

I spent some time testing the https://fossa.com mentioned by @RSMNYS. I think the HTML version of the report looks quite good and we can include it either in the app as HTML file or as a web link: https://app.fossa.com/reports/eb25eabe-7e15-45fb-ac34-b1f1cd848b03

They also has warnings for possible licensing issue: https://app.fossa.com/projects/custom%2B44937%2Fmobile_app_open-master