metaplex-foundation / shank

Extracts IDL from Solana Rust contracts
https://docs.rs/crate/shank_macro/latest
130 stars 26 forks source link

Can non-borsh accounts be shanked? #28

Closed gagliardetto closed 1 year ago

gagliardetto commented 2 years ago

Considering shanking SPL programs; many (most?) don't use borsh for encoding their accounts.

Would shank work on those accounts, too? Or that's a feature that would need to be added to shank?

Theorically, regardless the encoding, the IDL is the same.

thlorenz commented 2 years ago

They can however there are two caveats:

If those accounts have non-primitive typed fields then those types also need to be shanked. Currently all types annotated with BorshSerialize or BorshDeserialize are also shanked. However for non-borsh types we'd need to add an annotation like ShankType to include them that way.

However those may not be properly de/serialized when the SDK from it is generated via solita as that uses beet under the hood which is compatible with the borsh de/serializer.

Thus we'd need to include information in the IDL that shank extracts which let's solita know to use another de/serializer instead. Implementing + making solita use that alternate de/serializer could be a larger effort.

gagliardetto commented 2 years ago

Thanks @thlorenz

The use of encoding standards other than Borsh is the one of the main blockers towards making the anchor IDL a standard for the whole Solana ecosystem.

Right now, the task of handling the different encoding/decoding ways is on the shoulders of the various anchor generators.

In the lack of a standardized way to express that in the IDL, I think that should continue to remain a concern of the tools that read and use the IDL to generate clients.


My only concern and question here is this: given a type A, would it be possible to shank all other types that type A depends on without having to do that manually?

Example:



#[derive(ShankAccount)]
A {
   Foo B
   Bar []C
}

B {
  Hello D
  World E
}

...
thlorenz commented 2 years ago

The use of encoding standards other than Borsh is the one of the main blockers towards making the anchor IDL a standard for the whole Solana ecosystem.

Not exactly true afaik, as the info in the IDL could still be standardized and could also include info as to the de/serializer used on the Rust end. The code generator consuming that IDL then would have to use a matching de/serializer on the JS end.

would it be possible to shank all other types that type A depends on without having to do that manually?

Yes, but B would have to be annotated somehow. As I said currently shank uses Borsh annotations as a hint to include it. Just adding ShankType could also work. However the piece still missing would be to implement a beet-like de/serializer that matches whatever those programs use.

An alternative is to just include any type used, but that would lead to huge IDL + generated code for types not even used by accounts or instructions and makes name clashes more likely, so I'd prefer an annotation.