BIP 64 is in the repo, but no clients support it.

[zander]

BIP 64 doesn't seem to be something I'd expect in this repo.

1. it is not a consensus rule.
2. most (if not all) cliens on the BCH network have not activated this proposal.

[thesquaregroot]

After investigating this, I believe BIP-64 was added as a reference point for the service bit with index 1 ("NODE_GETUTXO").

Documenting the service bits is a bit awkward since the historical use of these bits has clearly had an impact on which ones were used for new purposes. The fact that this bit is apparently no longer used but was also not later repurposed, could be seen as allowing for nodes to re-implement that feature as-is in the future. Regardless, it's relevant historical context.

But I agree that since the BIP is also listed on the home page without that context, it could potentially be misleading. What are you thoughts on moving BIP-64 to a path like /protocol/forks/unimplemented/bip-0064 and removing it from the home page? That way it's not listed with all the other forks (both in the repo and via a webpage) but is still available for linking to on the version message page and could easily be moved back to its current location should a node decide to implement it in the future.

This could also establish a good precedent for CHIPs, during the time between when they're proposed and implemented.

[zander]

Since you ask.

My background is that I worked with standard setting organizations and teams, there is an ISO spec with my name in it.

From this background I find it important to specify the agreed-upon behavior, typically seen because it is implemented, and the spec reflects the result of cooperation and the spec does not show where cooperation didn't finish or failed. The specification lives on a different plane than implementations, the goal is that if two implementations disagree then the one that doesn't follow the spec is wrong. The spec is always right.

This is my personal background and this is what guides my advice as you ask for it.

So, as you probably can predict, I would say that a BIP that didn't get activated should not be in the spec. There are other repos and historical places for this that are more approrpriate. If you start to include all the things that might make it, didn't make it or similar then you're no longer the independent observer. You are making judgement calls. And people can abuse the influence of being listed in a spec by talking you into things. Slipperly slope.

Its not implemented by the majority, nor is there any clear very wide agreement that nodes will implement it soon, therefore it doesn't belong in the spec.

[thesquaregroot]

How often in your experience were those standard written without a sense of determining what "should be" a part of the standard? In "standard setting," are you implicitly making decisions about what should or should not be included based on your judgement at that time? In turn, once the standard is published, haven't you almost always defined something new that must be implemented by those that seek to comply with it? Because if so, it's important to recognize that that's not the situation we have with BCH. In many ways we are documenting the protocol, not defining it. It has a messy history that is hard to capture in a way that is both accurate and logical.

On top of that, and perhaps because of it, it is often easiest to include all of the information we come across in documenting the existing behavior. Selectively excluding information also requires judgement calls and can also be abused by those who would wish to silence a minority or discredit an idea (not saying you're doing this, just that it's also a slippery slope). If something is incorrect or misleading, we're happy to adjust it to make the reality more clear. But removing information that could be used for better collaboration or historical understanding isn't a priority when we still have so much to document.

We are primarily using code bases as the source of truth about what is a part of the protocol. Collaboration between node developers has historically not been perfect and, honestly, I would argue that this approach has also improved the code of several nodes (e.g. see your recent changes/pull requests regarding op codes). So when two implementations disagree, we don't get to write the spec how we want it and then say one of the nodes is wrong. So we've decided to call it out and move on. Hopefully that fosters more future collaboration than just ignoring it would.

[zander]

are you implicitly making decisions about what should or should not be included based on your judgement at that time?

Is a good question and the answer may surprise you (clickbait sounding answer, haha :)

To decide if something would end up in the spec there was a group of people, basically stakeholders, that review, improve and ultimately decide on what to include. The main editor then took the drafts and included it into the main spec-document.

There are very strong similarities with BCH where the decision to include it is indeed done by stakeholders. Typically by writing code to support it.

The point here is that to decide what goes into the spec is identical to the decision on what the protocol looks like, as implemented by stakeholders. The spec makers are not making decisions, they document decisions already taken.

In many ways we are documenting the protocol, not defining it. It has a messy history that is hard to capture in a way that is both accurate and logical.

Ok, but here is the important part: you are not documenting the history, you are documenting the protocol as it exists now. Sure, when possible say that some stuff entered the spec then and then. Sure, have a 'forks' dir for historical data. But you are documenting what actually is in production today.
More to the point, your audience mostly don't care at all about the history, your audience is people that want to interoperate with the network as it is today.

it is often easiest to include all of the information we come across in documenting the existing behavior. Selectively excluding information also requires judgement calls

I respect that, and the work is not glorious or easy :-)
The judgment calls won't go away entirely, the point is to have a clear policy that minimizes abuse and minimizes drama and confusion. I think that the position I wrote in my previous reply is the best in that regard as you can clearly state the policy and most judgement calls will be easy to support based on easy to find evidence. Someone wants to counter and say BIP64 should stay? Let me connect to a random selection of nodes and see if it works. Simple proof based decision.

when two implementations disagree, we don't get to write the spec how we want it and then say one of the nodes is wrong. So we've decided to call it out and move on. Hopefully that fosters more future collaboration than just ignoring it would.

A Noble idea, pointing out problems. Also a quite political one. One I would personally avoid as you get to be the centre of drama in short order.

Lets look at the reality. I added double spend proofs. It took some years before that was accepted to the point that it entered this spec. In your view that means that during those years this spec should have been documented in the BCH-spec. Even though the protocol I was writing was still changing.

I think that it is accomplishing the opposite of what you are setting out to accomplish. Either the developer get annoyed with the spec for inevitably being outdated, or she may get evil ideas and add code that shows she is extremely productive and I'm hogging all the names.
Point being, and I think you agree, you guys are documenting the spec. You should not be part of the decision making of what makes it in into the spec. That is what community / ecosystem etc is supposed to do.

I would personally suggest to set a simple guideline to what makes it into the spec. Mostly by being accepted in the marketplace. My DSProof didn't make it until the majority of the full nodes either merged it or gave a strong indication they would.

That makes it no longer your (=the specification dudes) decision. That saves you from having to make another judgement call. That avoids the spec being abused by people just adding some commits/features in their repo and thus bypassing the need to actually work together. That makes the spec less biased.

[thesquaregroot]

I appreciate your insights; thank you for taking the time to explain your thoughts in more detail. That said, for the time-being my plan is to address the core complaint of this issue while sticking to the way we've generally been proceeding. This is for a few reasons:

1. This is a worthwhile discussion, but probably not the right setting or audience for it.
2. As your link suggests, the decision keep this open and modifiable so that anyone can host their own repos and adjust the content as they see fit, has potentially allowed for a more organic solution to this problem, even if the community is unable to come to an explicit agreement about how this should work.
3. This is way above my pay grade. :)

I'll let you know once I've pushed up the relevant changes and close this issue at that point.

[zander]

Thank you!

[thesquaregroot]

Changes in 8b1108e.