Restructure codebase for a module-based separation of concerns

[cwgoes]

At the moment, we split the codebase into separate directories and sub-packages based on technical aspects:

• apps/ includes CLI code, wallet code, and a bunch of ABCI handling code
• core/ includes types, storage layouts, and some miscellaneous state machine logic
• ethereum-bridge/ contains a bunch of Ethereum bridge-related code
• proof-of-stake/ contains a bunch of proof-of-stake related code
• sdk/ contains the SDK code (client code)
• shared/ contains validity predicate logic, miscellaneous types and events logic, some protocol structure, and some WASM VP wrapper functions
• tx_prelude, vp_prelude, and vm_env contain some WASM standard library code (as far as I can tell)

At least personally, I find this separation somewhat confusing, for a few reasons:

• it's difficult to find all of the code associated with a particular feature (e.g. governance), since it's split into many places
• it's not always entirely clear whether some piece of code is in the right place, and some locations seem inconsistent
• I think it probably creates more file conflicts than necessary, since code for unrelated features is not always clearly abstracted

I propose for discussion a rework of the structure of the codebase focused on feature modules, e.g.:

• A module is a defined interface which must implement a bunch of functions, such as:
  • a validity predicate
  • logic to be run at the beginning/end of blocks/epochs
  • transaction builder logic for the sdk
  • commands to expose in the cli
  • genesis fields required
• Governance/PGF, proof-of-stake, IBC, the Ethereum bridge, MASP all become modules
  • if convenient, these can be implemented as separate crates in some modules/ directory
  • all of these modules should implement this interface
  • no feature-specific code should live outside its module

Some precedent for this abstraction can be found in the modules system of the Cosmos SDK, which, although it has a lot of boilerplate, implements this basic module abstraction in a reasonably clean manner.

I think that if we do this well, it could make the codebase much easier to navigate, reduce file conflicts, and make it easier to add new modules in the future. Commentary & discussion welcome.

The dependency graph from 0.31.8: (generated with cargo depgraph --workspace-only --dedup-transitive-deps | dot -Tpng)

Module dependencies that should be removed:

• trans_token <-
  • [ ] governance
• parameters <-
  • [ ] governance
  • [ ] shielded_token
• account <-
  • [ ] proof_of_stake
• governance <-
  • [ ] proof_of_stake
  • [ ] ibc
• token <-
  • [ ] ibc
• proof_of_stake <-
  • [ ] ethereum_bridge

[cwgoes]

Looking over our current work, particularly @tzemanovic's PRs such as https://github.com/anoma/namada/pull/2312, I thought it would be helpful to try to define the target dependency graph of our internal structure. Based on my understanding, it should look something like this (top -> bottom = deepest dependency -> shallowest):

@tzemanovic @sug0 @batconjurer What do you think? Is this achievable?

[cwgoes]

Updated diagram per discussion with @tzemanovic @batconjurer

Agreed on order of operations:

1. General modularization & separation of modules with a clear interface.
2. Remove unnecessary WASM usage (specifically in MASP and IBC, which will both change to fully-native VPs).
3. Reassess and consider further changes.