Splitting code inside impl_runtime_apis! macro into separate files

[amiyatulu]

How can I separate custum runtime api into separate file, instead of putting in all in one file.

impl_runtime_apis! {

impl profile_validation_runtime_api::ProfileValidationApi<Block, AccountId> for Runtime {

        fn get_challengers_evidence(profile_user_account: AccountId, offset: u64, limit: u16) -> Vec<ChallengePostId> {
            ProfileValidation::get_challengers_evidence(profile_user_account, offset, limit)
        }
 }
}

[bkchr]

That is currently not possible.

[gupnik]

Yeah, I wanted to explore this after Construct Runtime v2. Current impl_runtime_apis seems too restrictive and probably deserves a v2. @kianenigma Thoughts?

[bkchr]

If it would be supported, I would have implemented it years ago. If we remove support for the native runtime entirely, we can improve the situation and switch to an attribute macro.

[kianenigma]

@gupnik you should follow the work regarding the deprecation of the native runtime and see how it can simplify CRV macros.

1. In the short term, we can still have a new CRV macro that is easier on the eye.
2. Once complexities of the native runtime or gone, if we can have it be split, even better

But it is important that step 1 and 2 are compatible, so it is good to keep an eye on this.

@bkchr are you sure the existing macro is impossible to break up? It might be a good reason for @gupnik to already start exploring.

[bkchr]

I just realized that if we use a static constructor that registers the different runtime api in the native runtime it could work.

[pkhry]

I just realized that if we use a static constructor that registers the different runtime api in the native runtime it could work.

I'm not quite following this, can you elaborate on your suggestion?

[kianenigma]

fwiw, the end goal of this feature is to

1. enable teams to upgrade to latest "default/standard" set of runtime apis, without needing to manually alter this macro
2. Split it into smaller files for better organization.

It might be worth looking into what happens out of the box when you just have two impl_runtime_apis! {} in a file. Which duplicate symbols are generated, if any?

[pkhry]

fwiw, the end goal of this feature is to

1. enable teams to upgrade to latest "default/standard" set of runtime apis, without needing to manually alter this macro
2. Split it into smaller files for better organization.

It might be worth looking into what happens out of the box when you just have two impl_runtime_apis! {} in a file. Which duplicate symbols are generated, if any?

ok, some notes:

If we call impl_runtime_apis multiple times within context of a single file we will see that those symbols will get duplicated:

• RuntimeApiVersion -> const val
• RuntimeApi - declaration, empty struct
• base runtime api traits will be implemented on the RuntimeApi
• interface functions that are used to call into the runtime in wasm.
• in addition to duplicate items, we will also encounter coherence issues where multiple applicable trait impls are present for an item.

possible solutions to having two macro invocations:

• using statics or thread-locals in proc-macro: Not reliable since it's not guaranteed that macro will be run at all due to the incremental compilation and there's no guarantee on ordering between macro calls, esp if they are in different files.
  • using fs to track state: Same issue of we don't know whether or not all macro invocations will be recompiled or not and there's no guarantee on ordering.
  • Introducing new macros that can be more granular in things that will get generated per macro block Can work, but would require loading generated code in memory and parsing trait impls to generate wasm shims.

[kianenigma]

Thanks for the great exploration here!

I am on the edge of my knowledge here, so take what I say with a grain of salt + trust your opinion from the code and or a comment from someone who knows the code better like Basti more.

Introducing new macros that can be more granular in things that will get generated per macro block Can work, but would require loading generated code in memory and parsing trait impls to generate wasm shims.

I like this either this, or moving towards making these ident names configurable so that you can call the macro multiple times. It seems simple, and no antipatterns are in it.

Also it might be worth noting that if anything is generated by this macro as #[cfg(feature = "std") is only needed for the legacy native runtime, which we can already count on removing.

This gives me hope that we can easily rename/dedupe these Rust idents, and have zero impact on the final wasm.

[pkhry]

i think the issue can be postponed until later/closed for now. as we need to have cross macro communication in impl_runtime_apis! due to generating code for native runtime and the need to define RuntimeApi struct to be used for all subsequent macro invocations.

native runtime shims need to know about all of the apis and methods for its codegen to function as it generates a large switch statement with all the <ApiTraitName>_<method_name> match statement