Need high level specifications for standalone SVM

[pgarg66]

Problem

Standalone SVM project's goal is to extract reusable code out of Solana transaction processing pipeline into its own crate (and eventually to its own GitHub repo). The project needs a high level specifications writeup to define the following.

• Scope of the functionality that is included in and excluded from standalone SVM
  • High level API and data structures exposed by standalone SVM
  • Interfaces (traits) that the users of standalone SVM must implement and provide to SVM during init or API calls (e.g. AccountsDB)
• High level block diagram showing how standalone SVM fits into Solana's validator client
• High level block diagram showing how a client can interact with standalone SVM

[anoushk1234]

@pgarg66 @dmakarov As discussed I've collected a list of applications from multiple ecosystem teams that they are interested in, this would help us define the spec and API.

Applications:

• SVM Fraud Proofs for Diet Clients
  • A succinct proof of an invalid state transition by the supermajority (SIMD-65)
• SVM Rollups
  • Rollups that need to execute the block but don’t need the other components of the validator can benefit from this as it can reduce hardware requirements and decentralize the network. This is especially useful for Ephemeral Rollups since the cost of compute will be higher as a new rollup is created for every user session in applications like gaming.
• Modified SVM (SVM+) An SVM type with all the current functionality and extended instructions for custom use cases. This would form a superset of the current SVM.
• Validator Sidecar for JSON-RPC
  • The RPC needs to be separated from the validator, simulateTransaction requires replaying the transactions and have access to necessary account data.
• SVM-based Avalanche subnet
  • The SVM would need to be isolated to run within a subnet since the consensus and networking functionality would rely on Avalanche modules.

[riptl]

Modified SVM (SVM+) An SVM type with all the current functionality and extended instructions for custom use cases. This would form a superset of the current SVM.

The SVM+ idea seems like a recipe for disaster. A large part of the current mainnet functionality is deprecated or redundant, and highly complex to replicate in external environments. The vote/stake programs are unlikely to be used in custom app chains, yet they introduce unbounded memory use. Which makes them unsuitable for the four other use cases named above. An SVM+ cannot be correctly specified before the entire mainnet protocol is specified first too, so it's not practical either.

The only production SVM so far (Pythnet) does the opposite instead by removing most of the current functionality. I suggest standardizing a "Solana Base Runtime", which is the minimum scaffold required for any SVM. (Slot context, blocks, generic transaction processing, and the system program)

[pgarg66]

SVM+ seems out of scope for the current issue. It might be best to create a new issue for SVM+ to discuss its merits and potential implementation.

[SOELTH]

We should specify the specs of the existing implementation first. After that we should do everything we can to simplify the implementation. Only then should we start thinking about (conservatively) adding complexity where needed, imo.

[pgarg66]

I believe some clarification might be useful here. The goal is not to reimplement SVM. The current transaction processing code in Solana validator client was not designed to be used outside the client. The code is intertwined with the non-tx processing code. We are attempting to split the code and put an interface on top of it. This will establish a cleaner boundary between tx processing code (SVM) and non-SVM code.

This issue is to track the creation of a spec for the interface of SVM. The current code doesn't have a well defined API, and it's impossible to create a spec for it.

It's also understood that some clients of SVM could reuse other Solana code blocks, while other clients might want to implement their own. If there's enough interest to split those code blocks, it's best to create their individual issues at some point of time. Just so that the discussions can continue even after this issue is closed.

[anoushk1234]

Modified SVM (SVM+) An SVM type with all the current functionality and extended instructions for custom use cases. This would form a superset of the current SVM.

The SVM+ idea seems like a recipe for disaster. A large part of the current mainnet functionality is deprecated or redundant, and highly complex to replicate in external environments. The vote/stake programs are unlikely to be used in custom app chains, yet they introduce unbounded memory use. Which makes them unsuitable for the four other use cases named above. An SVM+ cannot be correctly specified before the entire mainnet protocol is specified first too, so it's not practical either.

The only production SVM so far (Pythnet) does the opposite instead by removing most of the current functionality. I suggest standardizing a "Solana Base Runtime", which is the minimum scaffold required for any SVM. (Slot context, blocks, generic transaction processing, and the system program)

The ideas listed were only to guage interest from ecosystem teams and not something we should implement.

if it is hard to define a spec currently then I think we should first make a minimal SVM i.e the minimum amount of individual components required to execute transactions and be compatible with mainnet-beta while being separated from the validator, similar to the Minimal-AccountsDB where someone tried to implement it within 300 LOC. This might help gain clarity on what a spec looks like. We don't need to re-implement the SVM, just extract the important components out and put it all together.