pl-strflt / ipdx

2 stars 0 forks source link

IPDX Roadmap H2 2023 #81

Open galargh opened 1 year ago

galargh commented 1 year ago

eta: 2023-12-31


description: https://github.com/pl-strflt/ipdx/blob/master/ROADMAP.md children:

IPDX Roadmap H2 2023

Given our mode of operation and the fact that we are highly affected by current needs of teams, our H2 roadmap serves mainly us a guideline of where our interests lie and where we'd like to focus our efforts. We view our adaptability as one of our main strengths but it also means we have to be flexible when it comes to our planning. If you already know of things you'd like our involvement for in H2 2023, please let us know - we're happy to adjust accordingly. We're here for you ❤️

Drive GitHub Observability Usage Internally

Whenever applicable, every project we start in H2 2023 should have a Grafana Dashboard associated with it that tracks relevant metrics for the project. Each project issue should have a link to the dashboard, description of the metrics we care about and the expected outcomes.

Additionally, we'd like to use GitHub Observability to identify more workflows that would benefit from PL Self-Hosted Runners.

Release Gateway Conformance v1

In H2 we want to conclude IPDX's active development of Gateway Conformance. Ideally, we'd like to attract Gateway implementation maintainers to become active contributors to the project. We're going to continue to serve as maintainers, review PRs and provide guidance.

To be able to call Gateway Conformance ready for this, we should:

In terms of promotion of Gateway Conformance, we'd like to:

As stretch goals for the Gateway Conformance initiative, we identified:

Increase PL Self-Hosted Runners Relevance

With the recent $ cost reduction efforts in the PL self-hosted runners space the project is well positioned to serve the needs of PL developers already i.e. we feel confident it's the correct solution to apply to increase developer speed when faced with GitHub hosted runners bottleneck problem. As of now there are 2 areas that we'd like to look more closely into:

Expand IPDX Comms

We want to continue living close to our developers. That's why we're going to continue releasing What's new in GitHub? issues on a monthly basis. What's more, we'd like to start a new regular newsletter about productivity tips. Finally, we'd like to start tracking developer happiness among IP Stewards. To begin with, we're going to prepare surveys for IP Stewards developers that are going to be issued on a regular basis.

When it comes to external comms, we'd like to actively participate in IPFS Camp and GitHub Universe conferences.

Release Unified CI 2.0

In August, we're going to release new version of Unified CI with Go 1.21 support. With that release, we'd like to transition to Unified CI 2.0 which should encompass all the learning that we gathered by running Unified CI in its' current form for years. This includes but is not limited to:

All of the above will make Unified CI more robust, easier to manage and it will increase it chances to last for another couple of years. Moreover, it will open up doors for even more automation across hundreds of repositories, like using dependabot (instead of web3-bot that has to be maintained by IPDX) for Unified CI updates, or maintaining multiple variants of releaser workflows (e.g. release-please vs version.json vs PR).

Immerse Into the Day to Day of IP JS Developer

As one of the most under-resourced projects within IP Stewards group, it is the best target for our research. We want to embed with the team for a while to uncover areas of potential DX improvements. By the end of this exercise, we want to have a concrete, prioritised list of improvements that would make the life of a IP JS developer easier.

Improve the Day to Day of Kubo/Boxo Developer

We've been focusing on Kubo/Boxo developers in H1 2023. In H2, we'd like to work on the work items that we identified throughout the process. The main projects we envision are:

Explore DX of Building on IPFS Stack

A major push around debuggability within our stack is expected to happen. This is a great opportunity to kick start a joint effort between Kubo, Helia, IPDX, and probelab.

Through our work on Gateway Conformance we discovered that building on IPFS stack is not easy, even for people close to the ecosystem. We'd like to start a standalone project that would help us explore this further. The very rough game plan is:

  1. Start a simple project that cuts through all the layers of IPFS stack (working idea - decentralised DropBox).
  2. Consume the publicly available documentation to implement the project.
  3. Document usability sharp edges and documentation shortcomings.
  4. Contribute usability fixes to the IPFS stack.

Our thinking is that this exercise could attract more active contributors to the IPFS stack thus reducing strain on IP Stewards.

This is a project that sparks joy for Laurent 💖

Increase GitHub Security

We want to continue building up our partnership with RSS by continuous collaboration on Security Guides for Software Engineers. Moreover, we'd like to explore the possibility of:

R&D

We want to continue bringing innovation and improvement to our development stack. To be able to do that, we need to be up-to-date with the latest tech in the field. We'll definitely continue exploring AI tooling throughout the rest of the year and evaluate its' relevance for our projects. Moreover, we'd like to gain practical experience with GitHub merge queues by integrating them in GitHub Management workflows.

BigLep commented 1 year ago

Below are some quick thoughts during my first read that I didn't want to lose. I'll come back through again. Good stuff - thank you!

Release Gateway Conformance v1 - creation of dashboard encompassing all gateway implementations

I wonder if there is something "cheap" that can be done here where it becomes self-service to link into the CI results for anyone who has gateway conformance tests running. If there is a list of these projects then someone can open al the tabs and cycle through to see the conformance.

Increase PL Self-Hosted Runners Relevance - With the recent cost-reduction effort

Let's be clear about what costs we're reducing with self-hosted runners. I assume we're talking about increasing developer productivity (rather than saving infra dollars).

Release Unified CI 2.0 - using dependabot for Unified CI updates

I thought dependabot already does push out updates?

Improve the Day to Day of Kubo/Boxo Developer - create a general solution for identifyin flaky tests within Go ecosystem

This seems much broader than Kubo/Boxo (which is great). In a similar vein, I could see value in helping out the go monorepos we have in our midst: https://github.com/pl-strflt/ipdx/issues/80

Explore DX of Building on IPFS Stack

I like the direction of this one. It depends on the IPFS 2023H2 planning, but I am anticipating a major push around debuggability within our stack (which could be a joint effort between Kubo, Helia, IPDX, and probelab). Basically, how to crack open the IPFS black box so its self-service for anyone to understand why "IPFS isn't working" or "IPFS is slow".

galargh commented 1 year ago

@BigLep Seeing your comment I realised that we couldn't postpone automating roadmap issue generation any longer because discussing individual points in GitHub issue is just painful. Sooo... we're moving the discussion to this PR - https://github.com/pl-strflt/ipdx/pull/82. The roadmap can be found in the ROADMAP.md file. I also copied all your comments and posted them inline (already with our answers). Once we merge the PR, this issue is going to get updated with the content that we land on + children issues for all the milestones are going to get created.

BigLep commented 1 year ago

@galargh : this syncing between the roadmap.md and the issues is awesome - I like where this is going - awesome!

A couple of things:

  1. Can we have two-way linking between the markdown file and issues? For example, https://github.com/galargh/.github/issues/42 links to https://github.com/galargh/.github/blob/master/ROADMAP.md#drive-github-observability-usage-internally (great). Can we get a link in the other direction?
  2. You all know the conventions better than I, but should we pull the sync "script" out of yaml and into its own file so it can be unit tested and shared?
galargh commented 1 year ago

To both - yes! That's what I was getting at in https://github.com/pl-strflt/ipdx/pull/82#discussion_r1252180018. When we tackle 2-way sync, we're gonna extract the code because it's going to be easier to work with that way (plus, we'll be able to make it testable and shareable more easily).