Pub package repository support - for Flutter and Dart

jonjanego commented 6 months ago

Greetings from the GitHub supply chain team! We'd love for ClearlyDefined to support license information for pub, because it would help GitHub customers to get license information for any dependencies that they are getting from it in their dart projects. We already support pub within dependabot but don't have license information in it, which is used and reported on within dependency graph. GitHub relies on data from ClearlyDefined to resolve license information, so supporting pub within ClearlyDefined would be a key enabler to building out this support.

Following the ClearlyDefined guides for adding harvest information here is the initial set of information about pub:

Discoverability – how are the packages for this language discovered? Is the repository searched by the build tooling without the user having to customize their client?

Pub is the default package manager for Dart. More technical investigation is needed to understand precisely how packages are discovered, but dart docs highlight pub as the official package repository.

Primary Source – is this the primary repository that the package is published to? Or is this repository a mirror of an existing repository? We should always harvest from primary sources.

Yes. https://pub.dev/ "The official package repository for Dart and Flutter apps"

Reputability – is this repository operated by a reputable organization? What is the purpose behind running this repository? Is there an identifiable team that can be reached in the event of any issues?

Pub is supported by Google.

Security – how secure is the repository? Is there a team that is available to handle issues in a timely manner when they arise? How fast do they respond to issues, such as when a security vulnerability is planted as a backdoor in a package?

More investigation is needed to understand its security posture and policies.

Automation – does the repository support an API to support pulling of information? If not, is the package index organized in a schematized format that can programmatically queried using the package name and version and queried using HTTP(s). When using HTTP to mine data, ClearlyDefined should check for the existence of robots.txt or robot headers that indicate such mining is unacceptable. How much effort is it to automate the process?

There is an API for pub. Specific detailed requirements will need more investigation.

Relationship – reach out to the organization that maintains the repository to indicate that ClearlyDefined wishes to harvest data from their repository, with an explanation on how harvesting is done, what the data is used for and how much additional traffic this could result in. Identify/Resolve any concerns and provide a contact from ClearlyDefined in the event they need to support in case of an issue.

GitHub can likely assist in making introductions to Google maintainers. They're also contactable at https://github.com/dart-lang/pub/

Thank you for the consideration, and we're happy to help!

nickvidal commented 5 months ago

Hi @jonjanego, this is a great suggestion. Thanks!

Would you or any of your colleagues be able to join our monthly community call next Monday (April 8th) at 11am Eastern? We would love to hear more about pub!

jonjanego commented 5 months ago

hi @nickvidal unfortunately i am going to be traveling at that time on Monday, but perhaps @elrayle might be able to join? otherwise i'm happy to join the next month

elrayle commented 5 months ago

@jonjanego I plan to be there. It's one of my regular meetings.

jonjanego commented 5 months ago

update, per https://github.com/dart-lang/pub-dev/issues/7475 this is probably a nonstarter until there is an API from pub that returns this data

clearlydefined / service

Pub package repository support - for Flutter and Dart #1088