[Content] Make compilable docs

[gnunicorn]

Content request

We want to make sure that all code examples and tutorials in the docs can be (automatically) compiled, so we can easily test that they still build. Little is worth than having people copy paste a tutorial or example and find the code not building at all because we broke that without noticing.

Refs https://github.com/paritytech/substrate/issues/10165

Are you willing to help with this request?

Maybe (please specify)

[nuke-web3]

I have a plan in mind for this. The gist:

1. All examples are based on the node template, at a specific release version. This is latest tag in the template for a known working version with all examples in the docs.
2. Generate changes on the template for your example in a separate branch per unique example, keep important commits in mind (the "steps" you want to highlight).
3. Generate patch files for each step of note to review
   • For indv. patch files: git diff <older_commit>..<newer_commit> <optional filename> > patches/<step+file name>.patch
   • For whole commit delta in one patch: git diff .. -o patches`
   • For the whole tuto using all commits, use the above and it will generate a patch file for each commit, and you can apply the entire set withone command: git am patches/*
4. Save all patches in patches/example_name/* in the template root dir so that *these can easily be applied by anyone, and are in sync if the version of the template at any release tag, including latest**
5. Display the contents of the patch files in the docs from latest tag, not hard coded rust blocks
   • We could (and I think should) use the raw patch files throughout with some explanation for the "uninitiated", or (less ideal all around) parse out the important bits to feel more "natural" manually.`
6. CI will be used to apply patches to the template for each example, and confirm the node builds&runs.
7. On upgrades if changes to patch files are needed to fix CI, we tweak manually as needed, or for major things rework the commits and gen new patch files.
   • Note any changes that require docs wording changes and make then now.
   • Consider anything that should be noted in the template and Substrate release changlog that might have been missed or would be nice to add.
   • ...Can you apply patches to patches? like a diff of the base node template version patches mapped onto the example patches :pray: (TBD)
8. CI passes? Update latest tag, all docs known to work with examples.
9. Party :tada:

Example of a git diff > patch file look (use diff codebock to get this FYI):

diff --git a/shell.nix b/shell.nix
index bc289fc..9f3f563 100644
--- a/shell.nix
+++ b/shell.nix
@@ -6,7 +6,6 @@ let
     });
   pinned = builtins.fetchGit {
     # Descriptive name to make the store path easier to identify
-    name = "nixos-unstable-2020-04-26";
     url = "https://github.com/nixos/nixpkgs/";
     # Commit hash for nixos-unstable as of 2020-04-26
     # `git ls-remote https://github.com/nixos/nixpkgs nixos-unstable`
@@ -14,20 +13,23 @@ let
     rev = "1fe6ed37fd9beb92afe90671c0c2a662a03463dd";
   };
   nixpkgs = import pinned { overlays = [ mozillaOverlay ]; };
-  rust-nightly = with nixpkgs; ((rustChannelOf { date = "2021-03-01"; channel = "nightly"; }).rust.override {
+  toolchain = with nixpkgs; (rustChannelOf { date = "2021-03-01"; channel = "nightly"; });
+  rust-wasm = toolchain.rust.override {
     targets = [ "wasm32-unknown-unknown" ];
-  });
+  };
 in
 with nixpkgs; pkgs.mkShell {
   buildInputs = [
     clang
     pkg-config
-    rust-nightly
+    rust-wasm
   ] ++ stdenv.lib.optionals stdenv.isDarwin [
     darwin.apple_sdk.frameworks.Security
   ];

   LIBCLANG_PATH = "${llvmPackages.libclang}/lib";
   PROTOC = "${protobuf}/bin/protoc";
+  RUST_SRC_PATH = "${toolchain.rust-src}/lib/rustlib/src/rust/library/";
   ROCKSDB_LIB_DIR = "${rocksdb}/lib";
+
 }

[lsgunnlsgunn]

I'm all for doing includes for code inside of docs, but this seems a bit heavyweight process-wise )at least to me).

[nuke-web3]

I will get a PoC made for the add nicks tutorials to demonstrate the flow here :soon:

[nuke-web3]

https://github.com/paritytech/substrate-template-generator

This is a very rough README and NOTES.md describing the the tool

Basically if a full substrate node exists, you can use this tool to "fork" it into a new independent repo in a one liner setting the config you want: https://github.com/paritytech/substrate-template-generator/tree/main/config

With this template generated using any upstream commit, we can then with scripts/CI apply git patches for the devhub examples on top of and test them. As it's a full node, we can extend tests to full integration test as well, not just mock runtime and unit testing :rocket:

So it is the foundation for the outlined workflow (above)