Closed teamdandelion closed 5 years ago
bvohaska
commented
5 years ago @decentralion: great work! Maybe we can move this into "instructions.md" or something like that in the repo.
We should also make a distiction between specs and technical documentation. Where that distinction is that the technical documentation will contain all the context and additional exploration of other ideas as well as link any references to the defining Filecoin component spec.
@whyrusleeping: what do you think?
wchargin
commented
5 years ago Thanks for writing this up!
@bvohaska:
Maybe we can move this into "instructions.md" or something like that in the repo.
My understanding is that this issue is intended to be an instance of the
very process that it describes. So, once it progresses from “proposal”
to “draft”, it can be merged into drafts/113-metaprocess.md or
similar. Then, a root README or instructions.md could link to that
file.
We should also make a distiction between specs and technical documentation. Where that distinction is that the technical documentation will contain all the context and additional exploration of other ideas as well as link any references to the defining Filecoin component spec.
Agreed in the long term; in the short term, I’d propose keeping these in one artifact for now. Writing, expanding, and maintaining the spec alone will be a fair amount of effort. Adding separate tech docs would additionally require keeping those in sync with the spec and the go-filecoin implementation—three potential stores of information, not just two. I worry that this wouldn’t carry its weight in the short term. Note also that @phritz has expressed the importance of communicating design intent in the spec itself for implementers, so the distinction won’t be as black-and-white as “math here, prose there”.
@decentralion:
Proposals do not need to be at a very high level of detail, and should be written early in the process towards changing the spec.
I think that I agree, but let’s flesh this out a bit more. Here are some concrete variations on the same theme:
If I discover a problem in the spec, but I’m not sure of the solution, should I post a “proposal”? (In #110 @whyrusleeping suggested that this should at least be an issue on the repository, as opposed to, say, a Discourse post.)
If a group of people are working over an extended period of time on developing a solution to a well-defined problem, should these conversations take place in a “proposal”?
Is it the case that every proposal MUST specify the design intent, SHOULD specify the requirements, and MAY specify a proposed solution? (in the sense of RFC 2119)
Should this very discussion take place in the proposal issue (as it currently is)?
As I understand it, the answer to each of these questions is “yes”.
Drafts should merge easily (i.e. consensus from the team is NOT required to merge the PR).
To confirm: what’s implicit here is that the drafts/*.md file is a
“living document” that is intended to be discussed in and updated by
pull requests, until it reaches a state where maintainers agree that it
should be merged. But although the draft may evolve over time, it
remains at the same (high) level of detail throughout its lifecycle.
Is this all correct?
The draft can be left in the repository for historical tracking.
Yes, and: (a) to the draft should be prepended a header that lists the commit hash in which the draft was accepted into the spec, and (b) the draft file should thenceforth be frozen. This way, people can continue to make sense of the diff proposed in the draft even as the actual spec changes structurally.
Once we have a v1 spec, we'll tighten up the requirements for merging a draft,
…and dedicate some effort to fixing up the existing spec, which will be at varying levels of detail, to the desired precision, yes?
phritz
commented
5 years ago Thanks! This feels like the right direction. Some suggestions:
Re some of the other comments:
wchargin
commented
5 years ago
- i want to know proposals as they happen
Suggestion: “Watch” the repository. You’ll be notified of new issues and pull requests, which correspond to proposals and drafts respectively.
- i want to know about drafts as proposals graduate to them
As above, watch the repository. (This doesn’t provide a way to watch for only proposals or drafts.)
- i want to know when the spec changes so i can go implement
Not sure about this. Phabricator has Herald, which is what you want. I’m not familiar with a GitHub equivalent. You could probably implement a pure Git solution with a post-receive that would pick up changes to the spec file or subtree, but this would only trigger when you fetch from your local copy of the repository.
- i'm making a proposal and i want the explicit input of some specific people or WGs
Suggestion: @-reference them in the proposal issue. For working groups, we could create GitHub teams, which can also be @-referenced.
- i feel strongly about a proposal (also, draft) and want to know when/how the decision is going to be made so i can come and make a fuss
Suggestion: For proposals, subscribe to the proposal issue. For drafts, same comments as in “I want to know when the spec changes so I can go implement”.
- i'm a WG captain and planning work way ahead i want to know about potential changes that aren't even at the proposal stage yet, for example if we are thinking of radically reworking a part of the protocol i might want to limit investment in the current implementation until the dust settles
Not sure about this. If the answer to “are all issues in the repository proposals?” is “no”, and discussion of things that might become proposals tends to happen in the repository, then watching the repository suffices.
- i think there's a problem with the protocol where do i file a bug?
What we said earlier was: general questions (even serious ones) go to Discourse; if you can point out a particular part of the spec that looks wrong, then post an issue on this repository. (I’m not necessarily endorsing this, just recalling it as a starting point.)
- i'd like to ask for clarification about part of the protocol, where do i ask?
As above: Discourse. If the clarification is generally useful, it can give rise to a pull request (see next item).
- suppose i want to clear up some language in the spec, can I just propose a PR?
Suggestion: Yes, assuming that the change does not change the semantics of the protocol.
- suppose i want to make one small change, say rework some enums in a message. do i have to make a proposal?
Suggestion: Yes. Anything that changes the semantics of the protocol should go through the proposal process so that people watching the repository can provide input.
- i'm going to implement something new that just landed in the spec and it's not going to work exactly like the spec says, we have to make some changes. do i stop my work and make a proposal? do i do the implementation and then make the proposal or draft? ideally for something like this i would do the implementation and then propose the deltas directly to the spec, kind of more like an update than a new proposal.
This is an interesting one. +1. I’m not sure.
- are all issues in that repo proposals? i would suggest the repo is most useful if the answer is yes, but it's unclear from reading what your intent is and where bugs if any would go in that case.
Why is the repository more useful if the answer is “yes” than if the answer is “no, but all proposal issues are tagged with [proposal] and all draft update PRs are tagged with [draft-new] or [draft-update] or [draft-acceptance] as the case may be (or just [draft])?
As noted above, a benefit of also using issues for discussions of things that may become proposals is that people watching the repository can have more lead time.
- we should have a template for proposals. it should probably read like a super light weight design sketch with a clear problem statement, set of requirements, and a proposed solution and maybe alternatives. These four sections get at the core of intent. Here's how I've articulated it elsewhere: https://github.com/filecoin-project/notes/blob/master/designdocs.md#what-is-a-designdoc
+1.
- you need to say what happens to everything that is in specs now, all the PRs and whatnot. Seems like they either get moved to proposals, to drafts, into the spec, or get shot in the head. i would love to see this happen soon after we have the item above.
+1. @whyrusleeping said that most open PRs can die, save about two of
them. As to the fate of spec.md and drafts/, I have no comment.
phritz
commented
5 years ago are all issues in that repo proposals? i would suggest the repo is most useful if the answer is yes, but it's unclear from reading what your intent is and where bugs if any would go in that case.
Why is the repository more useful if the answer is “yes” than if the answer is “no, but all proposal issues are tagged with [proposal] and all draft update PRs are tagged with [draft-new] or [draft-update] or [draft-acceptance] as the case may be (or just [draft])?
To clarify I meant that watching the repo is most useful if each issue corresponds to a proposal or draft and there are no other uses for issues. This as opposed to using issues to also track bugs, have discussions that are not specific to a proposal/draft, etc. If issues are exactly one of a proposal or draft then the signal is really clear. If issues are a mixture of all kinds of spec-related stuff (eg, "write OKRs") then I have to filter/process it and figure out what is or is not relevant to my interests. Since I'm already doing this for a zillion repos I was kind of hoping there would be a super clear, high fidelity signal from the specs repo's issues. I'm not sure it's possible to restrict the use cases this narrowly or whether it's even a good idea, I'm just pointing out the signal/noise problem. We want the lifecycle events to be really easy to notice and find -- this is one way of achieving it.
i'm making a proposal and i want the explicit input of some specific people or WGs
Suggestion: @-reference them in the proposal issue. For working groups, we could create GitHub teams, which can also be @-referenced.
This doesn't work very well for busy people, they get so many issue updates and @s they tend to get lost in the shuffle. We've been trying https://github.com/filecoin-project/pm#how-to-use-this-repo to accommodate that fact for cross-WG communication, to mixed results. I'm not sure we have a great solution for this problem.
I would really love to see a fast-and-loose spec update protocol available to devs for stuff that is still emergent. In the next few months we are the only implementation and it'll be so much faster to go make small changes as direct spec PRs (eg, re-organizing enums or updating some details that changed because of implementation concerns). I can understand the thought that we should practice the way it should be now, but there's a real cost and I'm doubtful on the benefit of practicing extra process before we need it.
Beyond the next few months I would also like to see a fast-and-loose process for emerging / unsettled protocol features: maybe when something is in draft an implementation should work on it and they can feed back any changes that were necessary into the draft as direct PRs? Perhaps we're getting a little ahead of ourselves here, but I want to make it easy to make small changes because we're going to have a lot of them before the protocol reaches 1.0.
Basically: we should embrace the fact that any time we go to implement a draft there are going to be some things we overlooked that we only catch at implementation time. We should make feeding those changes back in easy because it's a good thing to make improvements to the spec based on what we learn, and if you make good things hard/onerous they happen less often.
wchargin
commented
5 years ago Thanks for the responses; this is really helpful.
Since I'm already doing this for a zillion repos I was kind of hoping there would be a super clear, high fidelity signal from the specs repo's issues.
Do email filters suffice? Notifications for this repository should have
subjects prefixed with [filecoin-project/specs] and also the List-ID
email header.
If this doesn’t suffice, how do other projects in this space handle this issue? We surely don’t want to invest the time right now into switching to Phabricator or setting up a bespoke notification system. I hope that we can combine the existing primitives in ways that are good enough to unblock us in the short term.
Basically: we should embrace the fact that any time we go to implement a draft there are going to be some things we overlooked that we only catch at implementation time.
Certainly. In some cases, it may be reasonable to keep a draft in draft stage until the go-filecoin implementation is complete, so that once the draft is accepted into the spec the implementation can immediately be updated. But this probably doesn’t apply in all cases, so…
We should make feeding those changes back in easy because it's a good thing to make improvements to the spec based on what we learn, and if you make good things hard/onerous they happen less often.
…+1.
phritz
commented
5 years ago Do email filters suffice? Notifications for this repository should have subjects prefixed with [filecoin-project/specs] and also the List-ID email header.
I'm talking about signal within the issues themselves. If an issue in the specs repo always contains either a draft or a (pointer to a?) proposal then every issues is relevant to people who want to keep track of what's going on with the spec. However if issues are drafts and proposals, but there are also issues that are discussions about OKRs, notes from meetings, debate about where to host the next meetup, ideas we should re-visit in the future, discussions of when and how to shave why's head, etc then it's really hard for me as someone interested in the spec to actually see what's happening with the spec. I have to sift through / ignore / filter all those other issues. I'm saying one way to ensure that it's really easy for people to locate and keep track of proposals and drafts is to have only proposals and drafts in the issues. In that model, issues in the specs repo are for drafts and proposals and only drafts and proposals. Super focused, clean, clear, no clutter.
wchargin
commented
5 years ago However if issues are drafts and proposals, but there are also issues that are [other things] then it's really hard for me as someone interested in the spec to actually see what's happening with the spec. I have to sift through / ignore / filter all those other issues
Sorry, I’m not following. Certainly any single issue should not be both
a proposal and a meeting note transcript. If you want to filter the list
of issues to see what’s actually happening with the spec, just filter by
the label (e.g., label:WG-go-filecoin on the PM repo); that’s why
we have labels. This just seems like a total non-issue to me—surely
clicking on the “proposal” label is not too much to ask. I must be
misunderstanding something; what am I missing?
phritz
commented
5 years ago what am I missing?
Usability.
bvohaska
commented
5 years ago Link to draft meta-spec:
https://github.com/filecoin-project/specs/blob/spec-update-process/drafts/spec-update-process.md
Feel free to take a crack at adding/removing content. I added a note about having a draft spec template. Might make it easier for people to collect their thoughts and not focus on formatting too much. Let's get all these thoughts into a draft that we can merge with master. I'm thinking about having something in drafts/ master within a week or so. It'll be only a draft so it will not be taken as truth until we have consensus from all stakeholders which may take a while.
ZenGround0
commented
5 years ago Some thoughts prompted by:
i'm going to implement something new that just landed in the spec and it's not going to work exactly like the spec says, we have to make some changes. do i stop my work and make a proposal? do i do the implementation and then make the proposal or draft? ideally for something like this i would do the implementation and then propose the deltas directly to the spec, kind of more like an update than a new proposal.
In my head solving this type of problem seems out of scope for the spec process itself to handle. It feels so much cleaner if all protocol level changes come about by the same dumb interface: make a proposal, work to move it to a draft, merge to the spec. As an observer I don't need to worry about how the protocol is being worked on to anticipate what is going to happen to the spec, I can simply rely on all new ideas passing through the pipeline. Adding additional spec-modifying channels to cater to unblocking work will lead to a spec process tightly tied to our initial working patterns. My guess is this leads to a brittle spec process and an increase in the chance that some subtle and critical mistake makes its way into the spec.
The universal dumb interface approach seems feasible to me because I don't think the spec process will end up being the best and only interface for us to synchronize the efforts of protocol and implementation developers, at least not until the project has matured and slowed down a lot. We don't need a spec process that handles issues of this fine level of granularity if we have some "layer 2" process, i.e. something like that used by @porcuquine in proofs land, to sync research and dev in other areas of the protocol like consensus or repair. We can work together to write proposals and code fairly concurrently (i.e. wait to write a draft until some code seems to work) and minimize the number of patching proposals submitted to the spec.
Without such "layer 2" higher context working relationships, i.e. once OSS contributors propose a higher percentage of spec changes, the dumb "just make a new proposal" interface might get annoyingly laggy. But maybe that comes with the territory for a stable project?
These are just initial thoughts, interested to see if anyone disagrees strongly.
nicola
commented
5 years ago Personal request:
phritz
commented
5 years ago Do we have agreement on this?
Sure, I'm all for whatever the team doing the work decides is the right thing.
Do we need an aligning call
Not from my end, you have my bits.
However one final word on usability / "how hard can it be to just click a label": you can't filter on labels in your email, so if labels are the thing that differentiates signal (updates to drafts, proposals, specs) from noise (background/coordination chatter) my email notifications do not give me the useful signal that I want. I have to manually filter out the noise, either by opening notifications or going to the web site and clicking around.
teamdandelion
commented
5 years ago Hi all! Sorry I've been absent on this thread the past week--was on vacation. I've created a pull request that merges a more fleshed out proposal to master; you can see it here: https://github.com/filecoin-project/specs/pull/119
Some features of the proposal:
In keeping with the proposed process, I'd ask that someone approve this pull swiftly, so that we can use master/drafts as the source of truth. If you would like to see specific changes, please send pull requests rather than bikeshedding on this thread. If you have major concerns/questions, then posting in this thread is appropriate.
If people are broadly happy with this proposal, then we can merge it as official as soon as @whyruslepeing and @bvohaska approve. Then we can setup issue templates to discourage people from posting non-proposal issues, and redirect them to discuss.filecoin.io.
pooja
commented
5 years ago We now have one.
bvohaska
commented
5 years ago @pooja: Where is this documented?
teamdandelion
commented
5 years ago @bvohaska: here i believe: https://github.com/filecoin-project/specs/blob/master/process.md
pooja
commented
5 years ago Yup! ^ @whyrusleeping If you were thinking of something else here
Spec Update Process
This is a proposal for creating a standardized process around documenting and updating the Filecoin spec.
Intent
We need to produce a technical specification of the Filecoin network and protocol. Without a spec, it will be difficult to accurately communicate (or come to consensus on) how Filecoin works, and it will be nearly impossible to write additional valid Filecoin clients.
Producing a spec is challenging. Like all documentation, writing it takes considerable effort, and it must be continually maintained to stay current. This proposal aims to describe a v0 process for writing and maintaining the spec. This is explicitly not the final process; see EIP 1 for an example of what a fully developed spec process would look like. This spec proposal is much lighter weight.
Requirements
Proposed Process
Updates to the spec will happen through a 3 part process:
Proposal
A proposal is a GitHub issue posted in this repository filecoin-project/specs. The purpose of a proposal is to communicate high-level intent to change the protocol. Proposals do not need to be at a very high level of detail, and should be written early in the process towards changing the spec. The GitHub issue number assigned to the proposal will be used to identify and reference to it.
Draft
A draft is a markdown document located in the master branch of this repository, within the
/draftsfolder. The title of the markdown document should start with the number of the proposal that spawned it.A draft is a concrete suggestion for a change to the spec, and should be written at the same level of detail prevailing in the spec. Drafts should merge easily (i.e. consensus from the team is NOT required to merge the PR).
Each draft should be identified by the GitHub issue number of the proposal that spawned it.
Spec
Once a draft has been officially accepted into the spec, the specs owners (@whyrusleeping and @bvohaska) will merge the draft into the spec code. The draft can be left in the repository for historical tracking.
Catching Up
We need to synchronize the spec with go-filecoin, and write it at a high level of detail. As of Oct 2018, we have a lot of catching up to do. For now, there is a lot of Filecoin behavior that is unspecified.
Until we have a v1 spec, the requirements for the spec are relaxed. We will not require that every draft be implemented at a high level of detail: it must only improve on the current document. Also, we are not yet blocking feature development in go-filecoin on their details being fully specified, although we should encourage anyone involved in developing unspecified features to go through spec review first.
Once we have a v1 spec, we'll tighten up the requirements for merging a draft, and will start blocking any unspecified changes to go-filecoin.
Note this is just a proposal. Comments are welcome, once we have some consensus we can make it into a draft and then get it merged as an early piece of the spec.