[Infra Refactor] Some way to retrieve and display code snippets on a GitHub repository

[CrackTheCode016]

Highlight the main issue(s) below:

Often it is useful to display code snippets directly from FRAME or Polkadot-sdk. Often, these may change, leading to this repository becoming outdated.

What is the proposed solution?

Because this is Javascript, it is not going to be the prettiest solution ever.

A CI job that:

1. Retrieves the relevant code snippets from polkadot-sdk
2. Saves them in a .rs file (or individual JSON file representing that snippet)

3. Use the CodeBlock component to display these code snippets

   The end result, in an MDX file would look like:

   ...
   <CodeBlock language="rust">{RustCode}</CodeBlock>
   ...

   Where RustCode is the example file, which the CI job generates

[liamaharon]

Are you thinking the entire file or snippets from it?

[CrackTheCode016]

Are you thinking the entire file or snippets from it?

@liamaharon Snippets makes the most sense. It would be nice to, for example, show the usage of frame_support::storage::types or a StorageMap, and the CI takes care of ensuring this is always in sync with the 'latest example' directly from the polkadot sdk

Essentially this is how docify works, but for Javascript or web-driven environments.

[bkchr]

@sam0x17 should be able to make this possible for you with https://docs.rs/docify/latest/docify/

[CrackTheCode016]

@bkchr thank you; looks like this is pretty much what I need: https://docs.rs/docify/latest/docify/macro.compile_markdown.html

I will test this docify templating out, and if all goes well this could probably make its way into some regular CI job that runs once a month or something and updates all snippets, or just make it part of the deploy process to docify stuff in compilation

[sam0x17]

If you run into any issues let me know. I have been meaning to add a git commit feature that lets you reference a particular git commit hash for more exotic embedding situations, and also open up the ability to embed things that don't have #[docify::export] on them. Have just been waiting for someone at parity to have not having these as a blocker and I'll add it so if this is that scenario let me know

[CrackTheCode016]

@sam0x17 So far it is working pretty well. I wrote a minimal program that exposes a Downloader trait, which can be configured with a hash, repo, etc. It downloads whatever file needed, grabs the snippet via docify, then it can delete the file afterwards. It essentially emulates just the file structure needed for the snippets to be rendered in the markdown file,

This means that essentially, we are first having to parse the markdown comment, fetch the file, then docify. Not sure if this approach is the best but it seems working for now

[sam0x17]

That is fine with me but if you've already put in that effort might be really helpful to others if you refactor it a bit into a PR on docify itself by adding logic that looks for a git url/hash to docify::embed and then uses your Downloader stuff 😉

[CrackTheCode016]

Once I get it cleaned up and working well, I'd be happy to do so :)

[CrackTheCode016]

A thought for taking this to the next level: If somehow, very barebones runtime environment can be constructed in something like Replit, then these snippets can be embedded and executable in page.

Docify --> Replit w/ tiny runtime or TestExternalities environment--> Runnable Rust in-browser

[sam0x17]

Yeah I think the ultimate dream for this would be to leverage the fact that we are WASM based anyway and directly browser embed the wasm with some kind of repl like you said