Entangling the runtime with specific loader instructions is inflexible and requires unnecessary overhead

[jackcmay]

Problem

This issue encapsulates an extension of the following PR (which has been reverted). https://github.com/solana-labs/solana/pull/6470

Solana runtime requires that all loaders implement LoaderInstruction and wraps the caller's data with InvokeMain. Doing so constrains program writers from extending LoaderInstruction and artificially restricts all non-native programs to using LoaderInstruction

The main goal is to get rid of this: https://github.com/solana-labs/solana/blob/a2a9d54985c140d9b710f2892c95cffe12afd579/runtime/src/message_processor.rs#L144

6470's solution was a partial solution but kept the InvokeMain instruction resulting in the following issues:

• Pushed the construction of the 'InvokeMain' variant to the clients retaining an unnecessary wrapper
• Resulted in different invoke mechanisms between native and non-native programs

Proposed Solution

• Remove the runtime's special case where the client data is wrapped with InvokeMain and call both native and non-native programs via the same code flow.
• Return the loaders to checking for an executable account as they see fit and calling the program entry point if found. Keeps the loaders detangled from the runtime, maintains a consistent and clean program invocation interface with clients, and allows the loaders to do whatever they need at the program entry point without the additional wrapper. (They can even use InvokeMain if they so chose)
• Remove loader_instruction.rs from the SDK. Instead, refer to the bpf_loader and move_loader's own instruction definitions
• These changes would also unwind this ugliness: https://github.com/solana-labs/solana/blob/73c31d873ebe6818bb23c6ce6c8e89bcb102fc58/programs/move_loader_api/src/processor.rs#L342. Instead CreateGenesis would be defined as another move loader instruction type alongside Load and Finalize and InvokeMain::RunProgram would be executed directly based on the following check if keyed_accounts[PROGRAM_INDEX].account.executable

[jackcmay]

@garious @mvines

[garious]

How would this change the transaction format? Currently, I write Instruction { program_id, data } and that implies InvokeMain of the program with program_id, passing it data. Would InvokeMain need to wrap Instruction::data in the transaction?

[jackcmay]

That was my original proposal but after discussions with @mvines we agreed that removing InvokeMain altogether was a better idea.

[garious]

Have you considered what a general Invoke instruction would look like then? How do I tell the Move VM, "hey, call function "foo" with this data" instead of the main entrypoint?

[jackcmay]

That would be part of ix data sent. The InvokeMain is implicit when there is an executable account. The client/loader are free to decide on the format of the ix data. Maybe the ix data contains something an InvokeMain (like) variant(s), maybe it doesn't

[garious]

Are you saying Move modules shouldn't be marked executable?

[garious]

Yeah, I think that's exactly what you're saying. That there's a difference between an executable Move module that defines main(), and a library that defines a bunch of functions. The latter can be RO, but not executable - that executable is reserved for telling the loader it implemented the known entrypoint and nothing more.

[garious]

So just a thought, but does the runtime have any real dependency on the executable bit? Seems like it only depends on Account::owner. If so, can we make executable the first byte in Account::data for a loader's accounts?

[jackcmay]

(two comments ago) Yes,that would be possible for a loader to do, except in Move's case a library comes bundled in a Move program so main is the entrypoint either way.

[jackcmay]

Runtime does utilize the executable bit both during loading as well as instruction verification. Leaving the Account::data contents up to the loader/programs is probably more flexible

[garious]

Okay, I'm on board. Do it!