roadmap: move WebTransport to Done section

[marten-seemann]

I'm not quite sure what our done criteria are (in this case, and in general).

In general, I'd consider an item "done" from a specs standpoint once the specs is stable, and an interoperable implementation has been rolled out. Usually, a single of our (big) implementations implementing a spec would fulfill that threshold.

For WebTransport, one could argue that interoperability is limited as long as we only have go-libp2p support, since WebTransport is a browser protocol. Maybe this means that we should wait for the js-libp2p release in this case.

[MarcoPolo]

In general, my intuition is: "It's done when there's no more work left to do for it (except maintenance/bug fixes) and users can use it."

Right now this only works in go-libp2p. Tracking issue for js-libp2p is here: https://github.com/libp2p/js-libp2p-webtransport/issues/1.

I always thought that this Roadmap was the roadmap of the libp2p project as a whole. The progress relating to the progress of the project, not necessarily just the specs. Am I wrong? If this roadmap is just a specs-only thing then this is done, but otherwise I don't think this is done since it doesn't match my intuition.

[p-shahi]

I'm not quite sure what our done criteria are (in this case, and in general).

Imo we can establish a basic definition of Done for items in this roadmap using what's outlined in the spec lifecycle doc. The Candidate Recommendation stage describes exactly I would want to be completed before moving an item to done:

The specification document itself is technically complete. It defines wire level formats for interoperability, error codes, algorithms, data structures, heuristics, behaviours, etc., in a way that it is sufficient to enable contributors to develop an interoperable implementation. There is at least ONE implementation conforming to the specification. That implementation serves as the Reference Implementation.

• [ ] Spec is merged and classified as "Candidate Recommendation"
• [ ] (by virtue of the above) At least one major reference implementation exists
• [ ] A well established testing criteria (defined at the outset of the project) is met (testing via Testground or otherwise)
• [ ] Public documentation (on docs.libp2p) exists

If we consider this roadmap an overarching product roadmap then I would push for those 4 points. There's room for flexibility and we can add additional points specific to certain roadmap items (like interoperability for this one.)

So I'd say we block this on https://github.com/libp2p/specs/pull/404, https://github.com/libp2p/docs/pull/198, and optionally https://github.com/libp2p/js-libp2p-webtransport/issues/1

[BigLep]

My head isnt' fully in this but a couple of quick thoughts:

1. I agree it's premature to mark it "done".
2. On first glance,, @p-shahi's suggested framing seems good to me.
3. When we do mark something "done", lets also be clear about what isn't done. In the webtransport case, we're likely going to mark it "done" even though rust doesn't have an implementation. We should add a note like "At least as of date X, there is a go (link) and js (link) implementation. There isn't a rust implementation. You can track the progress here (link)."
4. We do need to figure out how this "vision" roadmap fits with implementation roadmaps.
5. While we won't merge this PR as is, it's probably worth updating the "in progress WebTransprot section" with links on where to follow the items that are being worked on and which must be satisfied before we mark it as done. Lets make sure we have a tracking issue in libp2p/test-plans for the interop portion.

Thanks!

[p-shahi]

When we do mark something "done", lets also be clear about what isn't done. In the webtransport case, we're likely going to mark it "done" even though rust doesn't have an implementation. We should add a note like "At least as of date X, there is a go (link) and js (link) implementation. There isn't a rust implementation. You can track the progress here (link)."

I agree here, we need track this information too and measure progress across implementations but imo not inside this repo.

This is the specs repo and imo should not be coupled with project managing downstream implementations. We can have pointers to complete implementations within a spec itself (for example, the WebTransport spec could point to go-libp2p as the "first" reference impl) but where possible, I would prefer to decouple specs (and spec repo) from implementation details.

Which leads me to my second point that I think is relevant to these two points/questions:

I always thought that this Roadmap was the roadmap of the libp2p project as a whole. The progress relating to the progress of the project, not necessarily just the specs. Am I wrong?

We do need to figure out how this "vision" roadmap fits with implementation roadmaps.

This document is "bigger" than specs and is a long term product roadmap. I propose we organize work on the roadmap and do project management of product roadmap items outside of this repo at a higher, "product" level. So, I think this doc should live in its own repository (libp2p/roadmap***) and not in this specs repo.

That repo would be the appropriate place to join the product roadmap with implementation roadmaps without coupling the specs repo with project management work. Each product roadmap item is currently a header and we can make it link to a product requirement GitHub Issue. The tracking information can be added to issues (we can think of them as Epics) and will help us establish requirements traceability (trace an issue across repos from Product Roadmap <-> Spec <-> Implementation <-> Testing <-> Docs.)

I know this is a beyond the scope of this PR but after reading the above comments, I wanted to bring this up.

***I proposed calling this new repo libp2p/roadmap but it could house not just the roadmap but also a set of improvement proposals to the libp2p project (akin to FIPs, IPIPs, etc.) I think the roadmap is the entrypoint for non PL contributors to suggest changes to the long term direction of the libp2p project, so we also need to consider the process by which community members/other organizations contribute to it. Maybe we introduce a libp2p improvement proposal which updates the roadmap and then links to the necessary spec and implementation work that needs to occur.

[MarcoPolo]

I'd be hesitant to add another repo to our collection without answering if we really need it. There are a couple of downsides to an extra repo:

1. We have to manage multiple repos.
2. History isn't synchronized across repos. A single repo has a canonical timeline, multiple repos do not.
3. Having multiple repos for similar things (specs/roadmap) I think will be confusing to community contributors. Imagine someone opening an issue in specs, and then having it moved to the roadmap repo. Do we expect focus to keep track of the specs repo, roadmap repo, and possibly many implementation repos?

We could do all the suggested things within the specs repo, no? I personally like the single "source of truth" repo for the libp2p project. We can add a header for this roadmap that clarifies this is the roadmap of the project, not just the specs.

[BigLep]

Thanks for bringing up the discussion @p-shahi. Maybe before we get into the idea of creating a libp2p/roadmap repo, we should first get alignment of the problems with the current ROADMAP.md file we have in this repo as it exists today.

Problems I see:

1. Often doesn't make clear what the done criteria are
2. For items that are in progress, often doesn't make clear where someone can follow along with the work, especially once it starts getting implemented (although someone can probably piece that together from a related spec issue).
3. Related to the above, for implementations that aren't in progress, there isn't usually a tracking item where can watch.
4. Doesn't give indication of expected completion dates.
5. (minor) Summary view is limited to what is written out and it's a manual typed effort. For example, we have a table of contents at the top. This gives a view on the roadmap but it doesn't allow filtering down to "items that are in progress", "items for 2022". We could create those views with other lists if they are of use, but they would be maintained (and right now many of the links within the ToC are not working).

Are there more/different problems to add?

Looking at that problem set, I can understand to turn items like "✈️ WebTransport" or "⏱ Full Observability" into GitHub issues. We still need to add a description of what it is, the done criteria, etc. but the nature of being a github issue gives other powers like:

1. Easy two-way linking to other issues/PRs (e.g., the implementation specific tracking items).
2. Labels
3. Commenting
4. Adding to "project board" which itself can have different views

I can personally get behind the idea of the items on the current ROADMAP.md into an issue and I think you can even do that in this repo to start and move them if it makes sense. But best to align that we have the problem understood and defined.

It probably makes sense too to move this discussion about the ROADMAP.md out of this particular PR :)

[marten-seemann]

I've updated the PR. We should now be in a position to call the project "done". As a point of reference, we recently closed https://github.com/libp2p/go-libp2p/issues/1827.

@p-shahi @MarcoPolo Would you mind reviewing again?

[p-shahi]

Hooray :tada: