Ideas for a new set of example/template pallets

[kianenigma]

          good start, although I hope we can also revise both this and `example/ basic` in a more fundamental manner -- I think a lot of the existing features that are put into these example pallets are somewhat nonsensical and could be improved.

### I can imagine set of 4 example pallets:
- [ ] 1. `template`: a shell pallet, it is your starting point.
- [ ] 2. `basic`: one that has a small set of features, akin to a smart contract
- [x] 3. `kitchensink`: one that has ALL of the features. https://github.com/paritytech/substrate/pull/14052
- [x] 4. `dev-mode`: meant to demonstrate the difference between the latter and `dev-mode`.

As for nodes

- [ ] `node-manual-seal` template
- [ ] `node-fast-block` template
- [ ] `node-grandpa-aura` template

I can actually work on this myself as a part of doc-revamp.

Originally posted by @kianenigma in https://github.com/paritytech/substrate/pull/13944#pullrequestreview-1391416198

[ggwpez]

Yes and in one of those we should clearly demonstrate all V2 benchmarking syntax.
Currently that is the basic example pallet, but it is a bit too simple for good benchmarking demonstration.
Maybe this could go into the kitchensink example. The idea is to have a one-stop copy&paste source for all the syntax.

[kianenigma]

Another issue I see is that we don't seem to have good examples for manual and instant seal. Would be good to have a template that is devoid of all the consensus noise and just uses these consensus methods.

Update: https://github.com/paritytech/substrate-contracts-node is a good example.

[kianenigma]

All in all, I think it is not a good idea that our default templates that we provide are so noisy from top to bottom:

• The example pallet is way too noisy.
• The examples runtime similar. It has aura and grandpa, which might not be relevant at first.
• The node is consequently complex.

ideally, we would have templates that cover a wider range of things, and then we can use them in a cli tool that @ggwpez and @gupnik are working on:

• a pallet skeleton
• which can optionally have:
  • custom runtime API + RPC ()
  • boilerplate for a signed extension
  • boilerplate for validate-unsigned
  • boilerplate for benchmarking
  • boilerplate for ...

  • as a cherry on top, would be great if this CLI can also check your /std, /runtime-benchmarks etc similar to https://github.com/hack-ink/subalfred

Then, for the runtime, we should have templates that have:

• aura
• babe
• babe + grandpa
• aura + grandpa
• none

each runtime would have its own corresponding node. Note that none can be used with manual seal.

Once we have all of these templates in place, the CLI can just pull from these and build a substrate based project for you in a lego-like manner.

[brunopgalvao]

A good, well-documented OCW example would be highly appreciated as well.

[ggwpez]

All in all, I think it is not a good idea that our default templates that we provide are so noisy from top to bottom:

Yes. It is not really helpful in that state. I think we could have one example for each specific feature in FRAME. Like one for dev_mode, default_config, benchmarking (cant think of more right now), and one verbosely documented pallet. Probably the basic one? Just to explain the most important things about a pallet in great detail.

I cant say much about the runtime templates regarding the consensus. Maybe start with the one that is most often used by para-chains?
But we should also have one to demonstrate the default features, and in the future the new construct_runtime syntax.

[kianenigma]

A good, well-documented OCW example would be highly appreciated as well.

This was recently done by @bernardoaraujor, I think he pushed his changes upstream to substrate as well, so the existing template is much better.

[bernardoaraujor]

I haven't pushed any changes to upstream substrate yet.

Re-writing frame/examples/offchain-worker has been on my todo list for a while now. I'll try to give it some higher prio so I can provide you guys something on the next few weeks.

[kianenigma]

Yes. It is not really helpful in that state. I think we could have one example for each specific feature in FRAME. Like one for dev_mode, default_config, benchmarking (cant think of more right now), and one verbosely documented pallet. Probably the basic one? Just to explain the most important things about a pallet in great detail.

Yeah I like this and I am kinda doing it as a part of https://github.com/paritytech/substrate/pull/13454.

[kianenigma]

Thinking a bit more about the CLI, I think it would be great for us imagine what questions the end-user would be asked when going through that CLI, and then based on that, we can build as many templates as we need. Indeed, I think these templates can be the backing repository of the CLI.

What I would imagine is:

(questions about the overall node)

• welcome to substrate-node-builder-cli!

• what consensus mechanism you want to use?

  • babe + grandpa
  • aura + grandpa
  • manual seal / instant seal

• if you want to have any asset/balances system pre-configured, which one do you want to have?

  • balances
  • assets
  • both

• if you want to have any staking system pre-configured, which one do you want to have (this will be hard to nail, as the pallets for staking system are quite messy)

• if you want to have any governance system pre-configured, which one do you want to have?

  • Gov1
  • Gov2
  • Sudo will do!

• if you want to have any nft system pre-configured, which one do you want to have?

• do you want to have an identity system pre-configured?

• do you want to have a proxy system pre-configured?

• do you want to have a multisig system pre-configured?

• do you want to have a multisig system pre-configured?

All of this should be enough to build the node, its chain spec, and its service.rs with the consensus code in it.

Then, finally we ask:

• how many custom pallets do you want to have in your runtime?
• Then for each custom pallet, we ask a set of questions that I have not fleshed out yet.

---

This is more or less the direction that I see for this CLI, and one use-case of having more numerous example/templates.

If this CLI is actually going to surgically assemble a pallet, it can very well use @sam0x17's docify under the hood to move items around.

[ggwpez]

About the CLI:

I remember that @shawntabrizi had the idea to use some TOML/YAML config files to eventually generate a runtime. Not sure if the idea was to parse the config files in a proc macro, or to stuff the config files into a CLI.
Just to recap what we are doing now: Prompting the user, auto generating Rust code, checking that it compiles.
I think this can massively help new runtime and pallet developers to build on Substrate. My only concern would be that we need to version it again. Whatever we output from the CLI will only work for a specific Substrate version. So we should be aware of that.

[sam0x17]

As we re-construct these example pallets (or do this CLI thing), I think it is a good opportunity to, for each item we add to the pallet, go through the (rust) docs for that item and make sure that the behavior we are using is actually documented. For example, the rust docs on #[pallet::weight] don't really explain how to do anything, and there are a number of completely undocumented (AFAIK) ways of passing things to it.

My point is the example pallets should be such that they could have been constructed from first principles using only the rust docs as a guide, and this is a good opportunity to take a step towards that

[ggwpez]

Another idea: A pallet that implements all the different Hooks. And explains eg. the difference between on_idle, on_initialize and the soon to come on_poll.

[shawntabrizi]

When I suggested auto-generation of the runtime, I was thinking web app or CLI.

Cosmos has this: https://github.com/ignite/cli

I haven't had time to use it yet, but seems to be a popular way for people to get started in their ecosystem and scaffold out projects.

[sam0x17]

TOML would be very railsy so I'm definitely for it

[brunopgalvao]

Just want to add here, I do not think there is enough education around how to properly work with the fungible traits - it can be overwhelming for a newcomer (and at times confusing).

It would be nice to have a playground or have examples that showcases how to use the many different fungible traits.

[kianenigma]

Just want to add here, I do not think there is enough education around how to properly work with the fungible traits - it can be overwhelming for a newcomer (and at times confusing).

It would be nice to have a playground or have examples that showcases how to use the many different fungible traits.

paritytech/polkadot-sdk#225 might help with this. @liamaharon based on the above, would be great if you try and improve the documentation of these APIs, probably add more examples and so on as well.

[liamaharon]

Just want to add here, I do not think there is enough education around how to properly work with the fungible traits - it can be overwhelming for a newcomer (and at times confusing). It would be nice to have a playground or have examples that showcases how to use the many different fungible traits.

paritytech/polkadot-sdk#225 might help with this. @liamaharon based on the above, would be great if you try and improve the documentation of these APIs, probably add more examples and so on as well.

Sure. I'll create a new issue to track improving the docs, and work on those as I go with the tests. I've tried to include descriptive inline comments in all of the tests so they should be good candidates to include as examples with the help of docify.

[liamaharon]

1. template: a shell pallet, it is your starting point.

This is already implemented here: substrate/bin/node-template/pallets/template/src/lib.rs. I'm unsure why it's seperate from the rest of the example pallets, maybe it should be moved there?

---

Do we really need both a 'basic' pallet and a 'kitchensink' pallet?

• I understand the kitchensink would be very close to the basic one, but just with a few extra features
• I'm not sure when as a developer I'd want to reach for 'basic' rather than 'kitchensink', since I'd be using it as a template / example to write my own pallet from it makes sense to just have all the features I might want on-hand
• Having both pallets is additional complexity for developers (need to decide which to reach for) and additional maintenance burden on the FRAME team
• I feel it will be difficult to differentiate between what should be in the 'basic' and what should be in the 'kitchensink'.

---

'template' and 'basic' have similar meanings, I could imagine people being confused about the difference just from looking at the names. Maybe a more descriptive name for the template pallet could be 'starter' or 'skeleton'?

[sam0x17]

I'm unsure why it's seperate from the rest of the example pallets

Side note / random nit: I would really like to see all the frame examples in a proper/real "examples" directory for the frame crate, when that comes into existence, since cargo has a facility built in for examples and we are essentially dodging it with our current structure by virtue of frame not being a real crate.

[AlexD10S]

Trying to get a solution into this StackExchange Question: How to customize pallet_contracts? I found this very old tutorial by Shawn: Extending Substrate Runtime Modules where it shows how to create a pallet that wraps the pallet_contracts adding some functionality.

I believe it makes sense to add an example of a “wrapper pallet” which adds some functionality on top of an existing pallet (basic_pallet for example).