Open lorisleiva opened 3 months ago
This would be super useful for us. For context, we have a program with over 120 instructions, with most of those instructions existing as part of a bundle like this.
I propose to use JSON-Schema for validation since it also it has the ability to have resolvable $ref links to both in-document targets and URLs.
What I imagine we'd need:
Here's how I imagine it vaguely (This doesn't yet address how we refer to IDL, although it isn't outrageous to just establish a $ref
convention to refer to a program within the IDL):
{
"bundleName": "createAndInitializeMintAccount",
"accounts": [
{
"name": "fromPubkey",
"description": "The public key of the account funding the creation"
},
{
"name": "toPubkey",
"description": "The public key of the new account to be created"
},
{
"name": "owner",
"description": "The public key of the owner of the new account"
},
{
"name": "tokenProgramId",
"description": "The public key of the token program"
},
{
"name": "mintAuthorityPubkey",
"description": "The public key of the mint authority",
"default": {
"$ref": "#/accounts/0"
}
},
{
"name": "freezeAuthorityPubkey",
"description": "The public key of the freeze authority (optional)",
"isOptional": true
}
],
"args": [
{
"name": "lamports",
"type": "number",
"description": "Number of lamports to fund the new account"
},
{
"name": "space",
"type": "number",
"description": "Space to allocate for the new account"
},
{
"name": "decimals",
"type": "u8",
"description": "Number of decimals for the mint"
}
],
"instructions": [
{
"name": "createAccount",
"program": "11111111111111111111111111111111 ", // This one is of course challenging. But I'd imagine that being able to reference to programs by public key will be needed
"params": {
"from_pubkey": {
"$ref": "#/accounts/0"
},
"to_pubkey": {
"$ref": "#/accounts/1"
},
"owner": {
"$ref": "#/accounts/2"
},
"lamports": {
"$ref": "#/args/0"
},
"space": {
"$ref": "#/args/1"
}
}
},
{
"name": "initializeMint",
"program": "ATokenGPvbdGVxr1b2hvZbsiqW5xWH25efTNsLJA8knL",
"params": {
"token_program_id": {
"$ref": "#/accounts/3"
},
"mint_pubkey": {
"$ref": "#/accounts/1"
},
"mint_authority_pubkey": {
"$ref": "#/accounts/4"
},
"freeze_authority_pubkey": {
"$ref": "#/accounts/5"
},
"decimals": {
"$ref": "#/args/2"
}
}
}
]
}
Thanks for that Danny. I love the idea of defining the expectations for the "bundle" first and then linking these expectations to actual instruction accounts/arguments. My only comment here is, since we already have InstructionNodes
, can we just point to these instead — maybe using a new InstructionLinkNode
and provide a mapping for their accounts/arguments?
EDIT:
I'm using $ref
here, but we might as well just be idiomatic to the way codama
does things and use an InstructionLinkNode
(similar to stuff like https://github.com/codama-idl/codama/blob/main/packages/nodes/docs/linkNodes/ProgramLinkNode.md), so just read it as that. I used $ref
for my example, because they're generic JSON-Schema (which is a standard) , but I realize that in Codama we'd use link nodes instead.
EDIT 2: Looks like we'll need roughly the following:
InstructionLinkNode
(to be used where I use $ref
in my example pseudo-IDL)InstructionBundleNode
There's an open question here @lorisleiva about where the InstructionBundleNode[]
would go? On RootNode
?
I think since we have about 120 instructions that need this, it'd make sense for me to start prototyping in a PR (with some internal happy path testing), and we can reason about how it feels once I have a working prototype?
Thanks for that Danny. I love the idea of defining the expectations for the "bundle" first and then linking these expectations to actual instruction accounts/arguments. My only comment here is, since we already have
InstructionNodes
, can we just point to these instead — maybe using a newInstructionLinkNode
and provide a mapping for their accounts/arguments?
Yes! I was just thinking about it as you typed this.
Basically, the use cases I see are: 1) Multiple IX needing some arg to default to an account / arg from a previous IX (but keeping it use-overridable) 2) Overriding some sensible defaults for a use case?
Any IX that defaults in one of its' accounts / args to another previous IX account / arg would then render that specific arg optional.
For instance, a lazy default could look like:
{
"bundleName": "createAndInitializeMintAccount",
"instructions": [
{
"$ref": "system#/instructions/create_account" // assuming that this is the name of the ProgramNode in an IDL?
// another variation could imagine everything as one big JSON and then do:
"$ref": "#/programs/0/instructions/create_account"
},
{
"$ref": "token#/instructions/initialize_mint"
}
]
}
Then a more elaborate one could be:
{
"bundleName": "createAndInitializeMintAccount",
"instructions": [
{
"$ref": "#/program/instructions/create_account"
},
{
"$ref": "#/additionalPrograms/0/instructions/initialize_mint"
}
],
// this could let us customize some of the parameters
"accounts": [
{
"name": "mint_pubkey",
"replace": [
{
"$ref": "#/program/instructions/create_account/arguments/2"
},
{
"$ref": "#/additionalPrograms/0/instructions/initialize_mint/arguments/3"
}
]
},
{
// let's say that here we're replacing/overriding just for the sake of defaulting to something else
"name": "mint_authority_pubkey",
"replace": {
"$ref": "#/additionalPrograms/0/instructions/initialize_mint/arguments/2"
},
"default": {
"$ref": "#/program/instructions/create_account/arguments/2"
}
}
]
}
In this case I'd imagine we will then have to pass on accounts / params grouped by the IX name:
{
"createAccount": {
"accounts": {
// ... all the accounts for this IX
},
"args": {
// ... all the args for this IX
}
},
"initializeMint": {
"accounts": {
// ... all the accounts for this IX
},
"args": {
// ... all the args for this IX
}
}
}
Great! I love your idea of using “overrides” for accounts/arguments of the underlying linked instructions. I think this is a great way to build bundles bottom up. We could even consider nesting bundles by having a InstructionBundleLinkNode
.
Regarding your question on where these bundles should live: under the ProgramNode
would be my answer. Everything should be under the context of a program IMO. For instance, the createMint
may use the createAccount
instruction from the system program but that operation is very much a construct of the token program.
I also think using the additionalPrograms
array for that purpose is perfect because that is completely idiomatic to how Codama currently handles programs depending on each other. In the future, we may push this further by downloading these additional programs on-demand (say when that information is available on-chain) so when we do, this bundle logic will automatically benefit from that change.
Tracking progress here: https://github.com/codama-idl/codama/pull/143
There is still a lot to explore here so this issue mainly offer a space to discuss this in more detail before committing to this.
The idea here is to describe bundles of instructions that are typically used together to create more complex operations.
For instance
createAccount
from the System program andinitializeMint
from the Token program could both be part of acreateMint
bundle which would link to these two instructions and provide a mapping regarding the accounts and arguments that refer to the same reference on both instructions. For instance, the createdaccount
on thecreateAccount
instruction would also match themint
account of theinitializeMint
instruction.By describing these instruction bundles, we allow renderers to offer helper methods for these, CLIs to offer more useful commands and documentation to provide more useful examples.