Catalog existing rulesets

alexeagle commented 2 years ago

Find whatever rulesets we can, or based on some criteria.

Assess their current state, such as:

active
perform releases and how often
quality/reliablity
who are the maintainers

This is like a conceptual merge of

Note that we want to avoid overlap with the effort Bazel core team is making in https://github.com/bazelbuild/bazel-central-registry and https://registry.bazel.build - in particular we shouldn't try to map the dependencies between rulesets.

jsharpe commented 2 years ago

On thing we should probably suggest is a standard repository tagging structure; this is how buildkite discovers plugins; we could do something similar with rules? e.g. bazel-rules github repository tag?

alexeagle commented 2 years ago

Maybe we also want to take over a page like https://docs.bazel.build/versions/main/rules.html#recommended-rules as part of this issue.

alexeagle commented 2 years ago

I think an "MVP" to resolve this issue should have:

a web view like https://docs.aspect.build/stats.html (I'm happy to donate what little code exists to run that: https://gist.github.com/alexeagle/8d40bd35debc89f0450ebc5f3b610470)
automation to refresh the data periodically
sortable columns [stretch]
usage stats [if @aiuto can get us the data]
publish the result on our existing GitHub Pages under https://bazel-contrib.github.io/SIG-rules-authors/

hobofan commented 2 years ago

So from what I've gathered from today's SIG meeting, it seems like having the catalog is still something that has use for the SIG, and will only be partially supplemented/replaced by the BCR.

On top of that, it seems like having an up-to-date list of all rulesets and their existing (pre-bzlmod) dependencies would also help in bootstrapping the BCR by giving insight into which are the "most blocking" rulesets for wider bzlmod adoption (= rulesets that many other rulesets depend on).

So as the first step toward that, my plan would be to:

Try to extend the existing catalog to be as comprehensive as possible regarding the number of rulesets
Extend the information about the existing rulesets with:
- Dependencies (via manual inspection of WORKSPACE/README)
- Name in the BCR (if already published there)
- Whether automatic publishing to the BCR is set up (to differentiate from sporadic one-off publications that you can't depend on yet as a rules author; I've seen that a GH Action for that is in the works)
- Whether it's bzlmod module has feature parity (which I understand isn't a given for everything that exposes repository rules?)

@alexeagle WDYT?

aiuto commented 2 years ago

I'll once again repeat my assertion that we need more than a catalog.

There should be a repository that does nothing but cross rule-set CI.

it requires a specific bazel track (4.x, 5.x, head)
has a WORKSPACE or MODULE.bazel file that depends on the top N (let's say 10-15) common rules at specific versions
bazel build //... should just work without flags
bazel test //... should pass without flags
it includes both android and ios applications that depends on native C++ code.
It has a web UI that interoperates with the same back end as the mobile apps.

The goal is to go beyond saying that rules_go passes its tests with Bazel 5.x. It is to say that rules_go works with rules_proto + rules_apple to build a complex product.

In the past I had presumed that Google should own this, but perhaps it is a worthy community project. Or, perhaps Google pays for the CI machines, but does not own the content of the test projects.

On Tue, May 31, 2022 at 4:22 PM Maximilian Goisser @.***> wrote:

So from what I've gathered from today's SIG meeting, it seems like having the catalog is still something that has use for the SIG, and will only be partially supplemented/replaced by the BCR.

On top of that, it seems like having an up-to-date list of all rulesets and their existing (pre-bzlmod) dependencies would also help in bootstrapping the BCR by giving insight into which are the "most blocking" rulesets for wider bzlmod adoption (= rulesets that many other rulesets depend on).

So as the first step toward that, my plan would be to:

Try to extend the existing catalog to be as comprehensive as possible regarding the number of rulesets

Extend the information about the existing rulesets with:

Dependencies (via manual inspection of WORKSPACE/README)

Name in the BCR (if already published there)

Whether automatic publishing to the BCR is set up (to differentiate from sporadic one-off publications that you can't depend on yet as a rules author; I've seen that a GH Action for that is in the works)

@alexeagle https://github.com/alexeagle WDYT?

— Reply to this email directly, view it on GitHub https://github.com/bazel-contrib/SIG-rules-authors/issues/2#issuecomment-1142604790, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXHHHCN6ECV2OOO64KW6Z3VMZYH3ANCNFSM5FQFP5RA . You are receiving this because you were mentioned.Message ID: @.***>

alexeagle commented 2 years ago

That sounds great, but running that CI sounds expensive and hard, and the SIG has barely done any meaningful engineering so far, so I don't think we should try to expand the scope of this one. Also, the BCR has CI for interoperability as an explicit goal and has corporate funding to operate it, so maybe this one should be on the Google side of the "fence"?

alexeagle commented 1 year ago

Update: @ashi009 has made some progress and we now have https://github.com/bazel-contrib/bazel-catalog as a home for it.

alexeagle commented 1 year ago

Design doc: https://docs.google.com/document/d/1SVYOrKRJ6jZaK43-wSJfk16ghwx5q_8GpeQpSzrhFa4/edit#heading=h.lqnathh99lzv

alexeagle commented 1 year ago

A "minimal viable product" for this is now at https://bazel-contrib.github.io/bazel-catalog/

bazel-contrib / SIG-rules-authors

Catalog existing rulesets #2