Closed mockersf closed 2 years ago
I like this proposal a lot: I think it's substantially superior to all three of the other options listed.
Integrating Awesome Bevy directly into the website would be lovely; I think it deserves both a nicer UI and better visibility at this point. I'm not sure if it should be in the same repo or another.
I agree that awesome-bevy should be built into the main bevy website.
I really like having awesome-bevy in another repo for "separation of concerns" reasons. We can treat it like a database to be queried and we don't conflate "core website changes" with "new community plugins/games/etc", which will likely continue to increase in frequency (and would therefore create a bunch of noise in the bevy-website repo). The biggest downside is ensuring the website is updated when awesome-bevy is updated. Using a single repo would trivially ensure that awesome-bevy has valid content and that every change is automatically deployed.
But that seems like a solvable problem:
I'm not yet convinced that the addition of mdbook is a net win. Zola-everywhere has a number of benefits:
zola serve
is the only command required to work on all of bevy-website
The biggest missing piece that mdbook provides but bevy-website is missing is rust code-block validation. But that actually feels relatively straightforward to me:
rustdoc --test content/learn/book/getting-started/setup/_index.md
works after a few tweaks:
#
or %
. The "zola header" with metadata is generally the first thing in book md. This can be worked around by just inserting # FILE_PATH_HERE
at the top of each file (when validating the file). Duplicate h1s don't break rustdoc so the only downside here is off-by-one errors for line numbers. And adding the file path makes debugging easier (see example below)rs
language type in code blocks wont work. It needs to be rust
(which is still compatible with zola's syntax highlighting).The command above properly validates the "hello world" codeblock in the getting started guide (first run is valid content, second run introduces a bug):
I'm now realizing the file path is already part of the error message. So we would probably just use # bevy_website
as the auto-inserted header.
There is also the matter of letting rustdoc know where to find dependencies (such as the bevy
crate). This is possible with -L
and --extern
flags. So far I've gotten it to respect --extern bevy=path_to_bevy_rlib
, but you also need to pull in each dependency, so thats a bit nasty. However it looks like mdbook also just farms out validation to rustdoc and it also looks like mdBook doesn't handle dependency resolution. That issue references skeptic which appears to be exactly what we need.
It looks like @alice-i-cecile's "Understanding Bevy" book and @jamadazi's "Bevy Cheatbook" actually both avoid using the rustdoc validation, in favor of just embedding content from rust crates validated using normal cargo build
.
I think we should support both patterns. By default, we should favor directly-embedded code as this keeps the code close to the context and avoids the "line number file filter" errors introduced by something like `{{ import ../my_crate/foo.rs lines 0-10 }}``. But embedding content from external crates avoids code duplication for large examples introduced across many code snippets.
Therefore I propose:
It looks like @alice-i-cecile's "Understanding Bevy" book and @jamadazi's "Bevy Cheatbook" actually both avoid using the rustdoc validation, in favor of just embedding content from rust crates validated using normal cargo build.
This was courtesy of @mockersf's setup in my case <3
If we're extending Zola to do this, we should try to upstream it as well. Less maintenance burden in the long-run, and I'm sure it's a feature others will appreciate.
Lol well (2) was really easy:
The idea was not to snipe you into making it 😄
So you feel strongly about keeping everything with Zola.
It is possible, but will require more elbow grease in the CI to build it 👍
Cargo.toml
files and run cargo test
Are you ok with that plan?
It looks like @alice-i-cecile's "Understanding Bevy" book and @jamadazi's "Bevy Cheatbook" actually both avoid using the rustdoc validation, in favor of just embedding content from rust crates validated using normal cargo build.
I like this pattern as it provides a complete context on how to run a snippet, and is nice to split a tutorial into step by step
The idea was not to snipe you into making it
No worries! I definitely didn't interpret it that way. I just had enough Zola context to know it should be doable and enough motivation to compare that type of implementation to what mdBook does.
for full crate examples
- validation is easy, just have to find all Cargo.toml files and run cargo test
- include would need to be able to filter by line or code block as mdbook, should be doable
Yeah I agree with all of that. Just gotta be a bit careful about "accidental offset errors" when someone makes a change to the code (or even just runs cargo fmt) and moves the desired line numbers to a position outside of the range referenced. Code blocks using this approach will require extra diligence on our part (which is why inline should be the default).
for snippets
- skeptic parses the specified md files, extract code blocks, put them in a test file and run the tests. Its functions for doing it are private without issuing cargo commands, but we can build an empty crate that would just run skeptic
Yeah before committing to skeptic someone needs to do a quick proof of concept (in the context of the Bevy Book) to make sure the workflow is ideal. I'm a little afraid the templates will be overly limiting, whereas rustdoc is pretty flexible. Solving the rustdoc dependency problem in a clean way will be challenging, but probably not impossible (and maybe solved by someone already).
Also both skeptic and rustdoc will rely on us solving the "ignored line" problem. Most inline code blocks will require some "context" that isn't rendered, but helps the example compile. rustdoc normally supports filtering out "hashtag lines" like # use bevy::prelude::*
. But we'll need to do that ourselves because rustdoc isn't generating html for us. Skeptic will also compile "hashtag lines", but we still need to filter them out during rendering (which i have confirmed zola does not do).
Doing hashtag filtering would require a pre-pass of some kind:
We could also just rely on skeptic templates for "hidden content". But unfortunately using the skeptic template syntax breaks syntax highlighting in zola. Ex: starting a code block with ```rust,skt-foo
breaks syntax highlighting, presumably because it is looking for a language named rust,skt-foo
.
If we can't find some way to solve the "hidden code for inline blocks" problem cleanly (or we're blocked by Zola upstreaming a feature we need), we might need to just rely on "external code blocks" completely when code validation is required.
Pulling in @Keats (creator of Zola) for input here as they have contributed to the Bevy Book in the past / seem interested in helping us out :smile:
Yeah I agree with all of that. Just gotta be a bit careful about "accidental offset errors" when someone makes a change to the code (or even just runs cargo fmt) and moves the desired line numbers to a position outside of the range referenced. Code blocks using this approach will require extra diligence on our part (which is why inline should be the default).
Better than filtering by line number, there is filtering by code block delimited by anchors (see the part that starts with To avoid breaking your book when modifying included files
in https://rust-lang.github.io/mdBook/format/mdbook.html#including-portions-of-a-file). I really like anchors for that.
We could also just rely on skeptic templates for "hidden content". But unfortunately using the skeptic template syntax breaks syntax highlighting in zola. Ex: starting a code block with ```rust,skt-foo breaks syntax highlighting, presumably because it is looking for a language named rust,skt-foo.
I used skeptic in the past and read it's source recently to check if we could use it, I think it's the best way forward with Zola. We will need to change Zola language resolution anyway because we may want to have code blocks annotated with no-run
, should-panic
, and so on. I think it's a small enough change that it may be accepted: instead of takin everything of the same line as `` as the language, split on the first
,`
Better than filtering by line number, there is filtering by code block delimited by anchors (see the part that starts with To avoid breaking your book when modifying included files in https://rust-lang.github.io/mdBook/format/mdbook.html#including-portions-of-a-file). I really like anchors for that.
Ooh yeah I'm sold. Implementing this should be possible will zola/tera templates using a combination of the split/starting_with functions (ex: split the file by newlines, iterate over lines and find those starting with // ANCHOR:
split on that, check if name matches requested anchor, if no continue, if yes try to find ANCHOR_END, combine range of lines back into single string (stripping out all anchor comment lines), and finally pipe string through markdown renderer. I won't pretend that writing these shortcodes will be fun, but it should be possible.
I think it's a small enough change that it may be accepted: instead of takin everything of the same line as ``` as the language, split on the first ,
Yup this doesn't seem particularly contentious and seems like a very scoped investment / quick win.
We'll also want to see if skeptic handles zola's "markdown header metadata" syntax properly, otherwise we'll need to add a preprocessor to strip it out.
Oooh so you guys want to implement mdbook anchor syntax in Zola? :eyes: :eyes: :eyes:
So, this should allow the official book to work kinda like Cheatbook, but in Zola instead of mdbook? :eyes:
Yup this doesn't seem particularly contentious and seems like a very scoped investment / quick win.
Zola actually already more or less support that, but right now it ignores tokens it doesn't know and consider the last token to be the language. if skeptic doesn't care about the ordering it should work already 👍 https://github.com/getzola/zola/blob/master/components/rendering/src/markdown/fence.rs
it also has a hl_lines
(for highlight) that accepts range that I didn't see in the documentation, will have to try it
Anything you need from Zola?
Short term I don't think so. Soon we should know whether the zola and skeptic code block language formats are compatible with each other. If they aren't, then we'll need to sort out the best way to resolve that (ex: we might request that zola defaults to the first language in a comma separated list instead of the last language, or we might do the reverse for skeptic). Not sure if theres a spec for that / what sort of precedent the wider markdown ecosystem has set.
tried integrating with skeptic, should work well but build issue with native libs: https://github.com/bevyengine/bevy-website/pull/148
and other version including code blocks from a validated crate: #150
Ooh you implemented anchors. Very nice :)
yup I wanted to have the two options for comparison
I think all of the main questions in this are now resolved, and we're sticking with Zola. Any remaining nits should go in their own issues, so we can address them properly.
Website would be made of several parts:
Awesome Bevy part would pull from another repo files describing all the community contributions and display them integrated in the website (https://github.com/bevyengine/awesome-bevy/pull/71)
Work needed for that:
Downsides:
Other options:
What do you think?