Rust Docs + refactor: `protocols` crates

[plebhash]

We recently published our protocols crates to crates.io. We need to make sure the APIs provided by those crates are properly documented under the Rust Best Practices.

This is also the perfect opportunity to map and address technical debt on those crates. As we go through the code and try to understand it, we take the chance to write down a list of ways the code could be improved. Things such as:

• unintuitive, redundant, broken or suboptimal APIs
• unsafe error handling (unwrap, panic!, expect)
• typos, bad variable names
• // todo comments, todo! macros, edge cases with unimplemented! or panic!

---

Task list

Rust Docs

• [x] v1 (already documented but worth a look before proceding)
  • INPUT: Q&A Discussion:: https://github.com/stratum-mining/stratum/discussions/897
  • OUTPUT: not planned
• [ ] v2/binary-sv2/binary-sv2
  • [ ] INPUT: Q&A Discussion: todo
  • [x] INPUT: Rust Docs Issue #1008
  • [ ] OUTPUT: Rust Docs PR: #1231
  • [ ] OUTPUT: Technical Debt issue: #1131
  • [ ] OUTPUT: Refactoring PRs: todo
• [ ] v2/binary-sv2/no-serde-sv2/codec
  • [ ] INPUT: Q&A Discussion: todo
  • [x] INPUT: Rust Docs Issue: #1009
  • [ ] OUTPUT: Rust Docs PRs: #1046 (small doc additions) #1231
  • [ ] OUTPUT: Technical Debt issue: #1132
  • [ ] OUTPUT: Refactoring PRs: todo
• [ ] v2/binary-sv2/no-serde-sv2/derive_codec
  • [ ] INPUT: Q&A Discussion: todo
  • [x] INPUT: Rust Docs Issue: #1010
  • [ ] OUTPUT: Rust Docs PR: #1231
  • [ ] OUTPUT: Technical Debt issue: #1010
  • [ ] OUTPUT: Refactoring PRs: todo
• [x] v2/binary-sv2/serde-sv2
  • [x] INPUT: Q&A Discussion: not planned
  • [x] INPUT: Rust Docs Issue: not planned #1011
  • [x] OUTPUT: Rust Docs PR: not planned
  • [x] OUTPUT: Technical Debt issue: not planned #1134
  • [x] OUTPUT: Refactoring PRs: not planned
• [ ] v2/const-sv2
  • [x] INPUT: Q&A Discussion: https://github.com/stratum-mining/stratum/discussions/1159
  • [x] INPUT: Rust Docs Issue: #1016
  • [x] OUTPUT: Rust Docs PR: https://github.com/stratum-mining/stratum/pull/1160
  • [ ] OUTPUT: Technical Debt issue: #1135
  • [ ] OUTPUT: Refactoring PRs: todo
• [x] v2/framing-sv2
  • [x] INPUT: Q&A Discussion: https://github.com/stratum-mining/stratum/discussions/858
  • [x] INPUT: Rust Docs Issue: no tracker issue was created (we should avoid skipping for other crates)
  • [x] OUTPUT: Rust Docs PR: https://github.com/stratum-mining/stratum/pull/848
  • [ ] #1178: Make #848 conform with doc standards in #970.
  • [x] OUTPUT: Technical Debt issue: https://github.com/stratum-mining/stratum/issues/903
  • [ ] OUTPUT: Refactoring PRs:
  • https://github.com/stratum-mining/stratum/pull/1033
  • https://github.com/stratum-mining/stratum/pull/1049
  • https://github.com/stratum-mining/stratum/pull/976
  • https://github.com/stratum-mining/stratum/pull/982
• [ ] v2/codec-sv2
  • [x] INPUT: Q&A Discussion: https://github.com/stratum-mining/stratum/discussions/873 (includes refactoring output, which should be eventually converted into issue)
  • [x] INPUT: Rust Docs issue: #1012
  • [x] OUTPUT: Rust Docs PR: https://github.com/stratum-mining/stratum/pull/1040
  • [ ] OUTPUT: Technical Debt issue: #1129
  • [ ] OUTPUT: Refactoring PRs: todo
• [ ] v2/noise-sv2
  • [x] INPUT: Discussion: https://github.com/stratum-mining/stratum/discussions/916
  • [x] INPUT: Rust Docs Issue: #1013
  • [x] OUTPUT: Rust Docs PR: #1111
  • [ ] OUTPUT: Technical Debt issue: #1130
  • [ ] OUTPUT: Refactoring PRs: todo
• [ ] v2/subprotocols (already documented but worth a look before proceeding)
  • [ ] INPUT: Q&A Discussion: todo
  • [x] INPUT: Rust Docs Issue: #1201
  • [ ] OUTPUT: Rust Docs PR: #1242 #1243 #1257
  • [ ] OUTPUT: Technical Debt issue: #1202
  • [ ] OUTPUT: Refactoring PRs: todo
• [ ] v2/roles-logic-sv2
  • [x] INPUT: Q&A Discussion: https://github.com/stratum-mining/stratum/discussions/1256
  • [x] INPUT: Rust Docs Issue: #1014
  • [ ] OUTPUT: Rust Docs PR: todo
  • [ ] OUTPUT: Technical Debt issue: todo
  • [ ] OUTPUT: Refactoring PRs: todo
• [ ] v2/sv2-ffi
  • [x] INPUT: Q&A Discussion: https://github.com/stratum-mining/stratum/discussions/895
  • [ ] INPUT: Rust Docs Issue: #1015
  • [ ] OUTPUT: Rust Docs PR: todo
  • [ ] OUTPUT: Technical Debt issue: #1136
  • [ ] OUTPUT: Refactoring PRs: todo
• [ ] utils/buffer
  • [x] INPUT: Q&A Discussion: https://github.com/stratum-mining/stratum/discussions/1185
  • [x] INPUT: Rust Docs Issue: #1184
  • [ ] OUTPUT: Rust Docs PR: #1232
  • [ ] OUTPUT: Technical Debt issue: #1247
  • [ ] OUTPUT: Refactoring PRs: todo

Refactor Implementation

• [ ] #67
• [ ] TODO

Dependencies graph

• [ ] OUTPUT: Current protocols' DEPENDENCIES.md PR: https://github.com/stratum-mining/stratum/pull/1233
• [ ] OUTPUT: Future protocols' updated DEPENDENCIES.md PR (after refactoring): todo

[rrybarczyk]

We can leverage the work done in #854 in this effort when we start listing the technical debt for each crate.

[jbesraa]

What do you think about merging all the v2/binary-sv2/** into a single v2/binary-sv2 for the following reason https://github.com/stratum-mining/stratum/issues/1134#issuecomment-2333691340 ?

[Fi3]

I don't think that this is a good idea as we wouldn't be able to opt out serde anymore

[jbesraa]

Oh sorry maybe my comment was not clear. I was just suggesting to merge them in terms of the effort to document and refactor them, not merge the crates into a single one.

[plebhash]

What do you think about merging all the v2/binary-sv2/** into a single v2/binary-sv2 for the following reason #1134 (comment) ?

As discussed in last week's dev call (sep 10th 2024), binary-sv2/serde-sv2 is a candidate for deprecation. This crate is the only reason we maintain the with_serde feature flag, which is known to be causing some trouble on downstream projects. Therefore it is highly desirable that we eliminate the with_serde feature flag by getting rid of this crate.

So spending time documenting this crate would probably be wasted effort.

The only thing preventing us from deprecating it right now is the fact the MG uses it as a dependency.

[plebhash]

we are starting to write many ideas to our Technical Debt issues

we should allow ourselves room to brainstorm and propose ideas, this is the time to get creative

but when we start doing the actual refactoring PRs, it's of strategic importance that we cultivate a disciplined mindset around Goodhart's Law so we can make the right engineering tradeoffs and compromises

[Georges760]

maybe #1233 can fit in this issue ?