search

executablebooks / meta

A community dedicated to supporting tools for technical and scientific communication and interactive computing
https://executablebooks.org
129 stars 164 forks source link

Discuss shared theme infrastructure between EBP and other themes #279

Open choldgraf opened 3 years ago

choldgraf commented 3 years ago

Background

Currently there are several themes that have slight variations on the same structure:

A lot of these themes share the same basic structure - a topbar, left sidebar, middle content, right sidebar, and bottom nav/footer. They also share many similar UI elements (eg., "click to expand subsections" in the sidebar). As a result there is a lot of duplication of similar work and UI/UX happening across these themes.

Proposal

Can we open a discussion for opportunities to unify pieces and use shared infrastructure? For example, we could unify on a single basic theme that defines the "topbar + 3 column layout" structure that we all use. This would be minimalistic and unopinionated, and designed to be sub-classed and styled by other themes. It would have an HTML structure that was templatizable, customizable, and extendable. (it could be one of the pre-existing themes, or a new basic theme that we'd depend on).

One idea for this theme structure could be this proposal from @chrisjsewell for a sphinx-book-theme layout. Perhaps we could generalize this to provide the right foundation for all of these themes, and thus have a good starting foundation for each.

If enough folks were on board with this, there are a few ways that we could collaborate towards making it happen (in increasing order of complexity):

Things to include in a shared theme

General layout Topbar, with 3-column layout beneath, w/ footer below. Basically this proposal from @chrisjsewell for a sphinx-book-theme layout

Here's a screenshot for reference:

image

Modular HTML components Components as standalone HTML templates that can be placed in different spots in page with jinja includes

Things to do

Questions

pradyunsg commented 3 years ago

Agree that this would be useful in theory

I agree!

a new basic theme that we'd depend on

This is what I think would be best, ideally without depending on Bootstrap or anything else (Furo demonstrates that this is not just possible, but also achievable). 🙈

This would be minimalistic and unopinionated, and designed to be sub-classed and styled by other themes.

One question I have here: do we want this to be like Sphinx's basic theme, where it only provides the assets + bare-bones styling, or do we want the styling to be a bit more fleshed out and usable (but overriddable)?

One option is to split the CSS for this theme into 2 parts: "scaffold.css" and "theme.css", where the "theme.css" can be what derivatives override/provide, while the scaffold is in the base theme! :)

choldgraf commented 3 years ago

Good point about the bootstrap - I think that kind of question is also important to ask: would a shared project need to be done without any JS/CSS framework? Or would something like Bootstrap be on the table of options? To me the benefit of bootstrap is that it lets people who aren't web developers make quick progress. OTOH if the base theme itself is designed well enough, perhaps this is not necessary? I'd be curious if @drdrij has any experience there.

pradyunsg commented 3 years ago

I spent some more time thinking about this, and I feel like it'd be good to add a few more "components" to this base theme, even if they're not included by default:

choldgraf commented 3 years ago

Announcements is a cool idea - you don't think that's something that could be done via an extension so it's theme agnostic? Eg you could add a node above the title node in the doctree of each page or something.

I started collecting a list of shared features that seem broadly useful across all themes in the top comment

jstac commented 3 years ago

Given the biomedical focus, a CZI EOSS grant might tie in well with the shift of @gregcaporaso's team's QIIME material to JB. That work is partly in scope of existing commitments but perhaps development of common theme infrastructure could take place concurrently with development of a theme specific to their publications --- analogous to QE's theme mentioned above.

pradyunsg commented 3 years ago

Announcements is a cool idea - you don't think that's something that could be done via an extension so it's theme agnostic?

Nope, I don't think this can be theme agnostic.

The announcement is supposed to be above the top bar, which is really not possible to do unless you know what the skeleton of the theme is -- which makes it theme specific.

chrisjsewell commented 3 years ago

One idea for this theme structure could be this proposal from @chrisjsewell for a sphinx-book-theme layout.

ah yeh cool, I was just about to moan that I had already proposed this lol.

would a shared project need to be done without any JS/CSS framework? Or would something like Bootstrap be on the table of options? To me the benefit of bootstrap is that it lets people who aren't web developers make quick progress.

Probably no, we should not depend on Bootstrap, at least not how we do now. For example, to have the left side-bar appear from the left (not top) as we very definitely should, the pydata/sbt theme template has to be completely re-done, not using bootstrap's grid framework. Also, there is the issue with the incompatible sphinx and Bootstrap container CSS classes

gregcaporaso commented 3 years ago

Given the biomedical focus, a CZI EOSS grant might tie in well with the shift of @gregcaporaso's team's QIIME material to JB.

@jstac, I'd be happy to do whatever I can to support this. I am planning an unrelated EOSS round 4 proposal so I'm not sure if it would be a detriment to this one if I were to serve as a co-investigator on another one. I also probably wouldn't need to be a co-investigator to support the project though - like you mentioned, this work could be considered in-scope for the Sloan award.

jorisvandenbossche commented 3 years ago

Thanks Chris for starting this conversation! I also fully agree that it would be nice to have a closer collaboration and shared foundations.

Regarding bootstrap, the main reason that we use it is because when I started the pydata-sphinx-theme (then pandas-sphinx-theme), that seemed the easiest way to get something working relatively quickly for somebody with no web dev experience like me ;) I still think that was a valid choice, but I assume that once a theme has been developed and is working without bootstrap (like @pradyunsg has done with Furo), it shouldn't be harder to maintain and further develop it (the value of bootstrap is probably mainly in the bootstrapping phase .. ;-)). So if there would be an effort (funding) to rebase pydata-sphinx-theme/sphinx-book-theme on something without bootstrap, I think that would be fine for me.

(and it's true that there are regularly compatibility issues with bootstrap's styling and sphinx / docutils output)

jorisvandenbossche commented 3 years ago

This [basic theme] would be minimalistic and unopinionated, and designed to be sub-classed and styled by other themes.

One aspect that's not fully clear to me is whether this would actually be "minimalistic". Would it rather be the superset of the different themes, or the common subset of the features of the different themes?

Concrete example: for pydata-sphinx-theme, the navbar with navigation is quite essential. But the other themes don't use that.

choldgraf commented 3 years ago

I've added a screenshot of @chrisjsewell's layout proposal above so it's easier to reference. @jorisvandenbossche I think one approach is that we could treat the PST navbar items like any other template. The theme could have configuration that would add templates to these sections.

So for the pydata theme it could be something like:

theme_topleft_templates: ["logo.html", "navbar_top.html", ...]
theme_topright_templates: ["social_buttons.html", ...]
theme_sidebar_templates: ["navbar_inpage.html"]

and for the Sphinx Book Theme it could be:

theme_topleft_templates: []
theme_topright_templates: ["social_buttons.html", "launch_buttons.html", ...]
theme_sidebar_templates: ["logo.html", "search.html", "navbar_inpage.html"]

something like that

pradyunsg commented 3 years ago

So... since @choldgraf posted this here, I've started scaffolding the project and HTML for this, to figure out what this should look like. (I'll make the repository public sometime next week, because right now everything is super rough and unsettled so taking direct inputs on the code won't be particularly useful for anyone).

At this point, I'm probably halfway through the whole discovery process for this, and I'll probably treat the rest of this discussion as a "collect everyone's inputs and try to build something that accounts for all of them".

Agree that this would be useful in theory, and then wait for somebody to try building it themselves

In other words, I got nerd sniped and this might be what we're doing now. Minus the "long wait" implied. 🙈

Oh, and I have a few fun questions for everyone here:

(my thoughts are in the nested bullet because... I'm on mobile in bed, and don't have a second editor to cut-paste them into a separate comment)

The theme could have configuration that would add templates to these sections.

I'm using theme templates/blocks-in-templates that can be overridden, since it won't expose this mechanism to the end user unless the theme opts-in to do so. (edit: I mean, the scaffolding shouldn't add user-facing theme options)

FWIW, my opinion on this is that the base theme should only provide the scaffolding and CSS for that scaffolding, and it should only provide theme-author-facing knobs. Let the theme authors control all of the user-facing knobs. OTOH, these knobs should 100% still be configurable for the theme authors (via the templates and setting Jinja2 variables, is what I can come up with). This isn’t set in stone though, because I don’t know what everyone else thinks. ;)

(PS: Furo intentionally doesn't have too many user-facing knobs, and I'd like to be able to switch over to this for it!)

Also, I've got a "theme-debug.css" file, which is what I'm using for debugging how the elements work (eg: for sidebar hover and shitz), but the default style would only be the scaffolding parts - with everything like fonts, typography, colors etc being the responsibility/stylised by the theme authors.

Also, perhaps the most bikeshedding bit: naming! (🌈) I'm thinking sphinx-basic-ng (as in next-gen) but am super open to bikeshedding this. Just, if you want to suggest names, you should probably also answer at least 3 of the questions above. 🙃

pradyunsg commented 3 years ago

Would it rather be the superset of the different themes, or the common subset of the features of the different themes?

My understanding was it'd be a subset.

Making it a superset means that bugfixes related to those parts would be tricky (similar to how SBT needs to wait for a pydata-sphinx-theme release to then make a bugfix release bumping the minimum version). I'm wary of it being a superset also because then it'll become a maintenance nightmare, due to the potential combinations that each of the design choices.

We could still have bits that we all agree are "easy to get wrong, difficult to get right", to make sure that we don't end up making the same mistakes.

pradyunsg commented 3 years ago

Yet-another question is: do we want the content and sidebar widths to exposed as customisable to the various themes?

It's significant complexity to allow this to be customisable, and I don't see any way to make this possible for theme authors without also exposing this to the end users as well.

I feel very strongly about not providing this knob to end users though, because there will be the occasional maintainer who'll want to have their documentation to have 100% width on wide screens, to the determent of all their documentation consumers.

But then again, I do want to hear what everyone else has to say. :)

choldgraf commented 3 years ago

That is a lotta text @pradyunsg :-)

bollwyvl commented 3 years ago

Late to the party, and haven't had time to fully grok this, but a huge win here (and good sugar for grantwriting) is not only going to "strict" HTML5 (whatever that means, but we can fully test it automatically), but further maximizing use of the HTML5 semantic elements which support readers using assistive technologies. Bootstrap 4, 5, 3000, whatever, no horse in the race, but having really a solid, positive accessibility stance is really good for everyone, including the search engines that help us find stuff.

So, concretely:

We're looking to do some of these things in other places in the pydata ecosystem, such as JupyterLab and an attendant nbconvert theme (e.g. labhtml5), but docs are great place to start.

:bellhop_bell: @isabela-pf @tonyfast (can y'all reach out to the numpy folk we were chatting with the other week?)

choldgraf commented 3 years ago

@bollwyvl great point that accessibility and semantics are also important "out of the box defaults" to have in a base theme. I think that would be a very compelling funding case for CZI.

bollwyvl commented 3 years ago

and semantics

To be sure, I'm not talking "Big S" semantic (e.g. RDFa or JSON-LD), much as I would love to see those promoted: just using broadly-supported HTML5 elements that have well-defined structural meanings, when possible.

pradyunsg commented 3 years ago

less div/span soup, but more intentional main / article / section, structure

I tried doing this with Furo, and this is certainly do-able.

The missing bits are things that are generated by Sphinx (it puts out <div id="..." class="section"> for example). That also applies to with figure and figcaption. I don't think the theme should be the one fixing this, and this should be fixed upstream.

That said, if we're willing to be hacky, we can definitely parse and rewrite the output from Sphinx. :)

One of the thing I'm noticing while writing this is that the themes might want to make fairly different choices around the "skeleton" of the various things:

For now I'm just going to make a single choice on these, but I'll file issues for each of them once I get the public git repository online. There's still the open question on how to share the SASS / SCSS across themes, but we'll get to that later. :)

pradyunsg commented 3 years ago

Alrighty! https://github.com/pradyunsg/sphinx-basic-ng/ is up and public.

I've got the basic responsive breakpoints settled down I think, and the barebones skeleton is working.

https://user-images.githubusercontent.com/3275593/111079958-3c17a000-84f4-11eb-9d95-63ecf9d3dbb4.mov

There's 2 "big skeleton" things missing:

One thing I'll note is that, well, the colors make the differences between the various layouts more obvious, but some of the breakpoint changes will look much less prominent once we remove these debugging colors. :)

PS: I notice now that I should've probably added a background for the "next/prev buttons". :)

pradyunsg commented 3 years ago

Oh, and if you want to try it out from a git clone of the repository, install [nox][https://nox.readthedocs.io/] with pip in a virtual environment and run:

nox -s docs-live -- ./tests/barebones/
choldgraf commented 3 years ago

Nice! Thanks for sharing - a couple quick thoughts:'

chrisjsewell commented 3 years ago

Nice one cheers! yeh I will try to give it a test run in the next few days. I would jump in there now and say, will you consider (or were you already) to move this to https://github.com/executablebooks/ at some point, so that all the participants here can be maintainers 😬

I think each section should have a list of templates associated with it, similar to html_sidebar. Then in the base theme, it would iterate through that list and insert templates. Does that make sense?

Yes this is definitely what I had in mind in my proposal, although obviously it depends how this works in practice.

The next question is then "what templates live in this base theme, vs. being in a child theme? Did you have an idea for this one already?

Indeed, we should start making a demo child theme sooner rather than later, to understand how this will work

Then obviously we should make sure we are implementing all the things we have learnt from building the current themes (looking through the open/closed issues/PRs), a few things off the top of my head that may or may not apply to the skeleton or just the children:

chrisjsewell commented 3 years ago

Oh and I also want to cc in to this discussion @mgeier who (as I mention in my original proposal) has been working on https://insipid-sphinx-theme.readthedocs.io

choldgraf commented 3 years ago

I just opened up https://github.com/pradyunsg/sphinx-basic-ng/issues/7 with some more thoughts on the "components" idea, maybe we can iterate on thoughts there?

And thanks for pinging @mgeier - my bad I totally should have pinged you at the outset! I've added insipid to the top comment list as well 👍

choldgraf commented 3 years ago

I have a PR that implements some of the "component" ideas that I had - would love feedback from folks:

https://github.com/pradyunsg/sphinx-basic-ng/pull/8

We can either keep iterating on that PR to change/add/remove what folks wish, or just use it as inspiration and iterate in other PRs, I will not be offended either way 🙃.

pradyunsg commented 3 years ago

will you consider (or were you already) to move this to https://github.com/executablebooks/ at some point

Yup yup. Let's do this once we've settled down on the issue tracker discussions though.

I'd actually tried creating the repository on executablebooks, before realising that I'm not even in the organisation. That also made it easier for me to decide where to put this right now (because I was split on that from a "how should we structure the initial collaboration effort for designing this" perspective).

I've found that when there's a bunch of folks (with each bringing their own opinions and preferences) collaborating on a single thing with GitHub as the primary communication channel, it helps (communication-wise) to not spread out the merge permissions in the initial design process and to have a single person be accountable for making sure that everything that someone said is accounted for.

I guess I've basically unilaterally decided to be responsible for making design choices while considering all the opinions/inputs here. Although if folks would rather we spread this responsibility out or have someone else be responsible for this, just say so. :)

what templates live in this base theme, vs. being in a child theme? Did you have an idea for this one already?

Yessss.

I'm hoping to make this look super barebones but still have all the structural bits in place (like the basic sphinx theme) and provide a bunch of components to use in downstream themes. Basically, no one would want to use this as-is (because it'd generate stupidly empty pages), but it would have all the HTML and CSS to make things usable should someone (like a theme author) decide to add bits of HTML and CSS and... et viola, it all works and looks pretty.

To elaborate a bit on what I had in mind when structuring this:

I think the header should be split into left_header and right_header template sections.

Actually, I had a bit of a different idea for this:

This is much easier to maintain upstream thanks to fewer points to interact with downstream themes and it is easier to adapt for RTL languages as well.

RTL support is also why I don't want to add "left" / "right" phrasing in various places overall, but instead alternatives like "primary" / "secondary".

I think each section should have a list of templates associated with it, similar to html_sidebar. Then in the base theme, it would iterate through that list and insert templates. Does that make sense?

It does make a lot of sense. I definitely want to make it easier for downstream themes to do this though, since it is a super nice way to allow extensibility for the end user.

I'm pretty much a strong -1 for doing it by default though, because that forces any/all derivative theme to provide those knobs because you can't remove a parent theme's theme_options. (yes, this is partly because I don't want to be providing any such knobs in Furo and [as yet unnamed and unreleased theme that I pulled this skeleton out of])

I think a Jinja macro, a helper function and good documentation should be enough to make this straightforward for downstream themes to do.

To generalise what I'm trying to say here: if it's touching theme_options, it should be something that the downstream themes opt-in to. We can make some of these things easier to do downstream (especially if it's tricky to get right or used in multiple places), but they should still be opt-in IMO.

very clear/complete documentation

Yup yup. Definitely want to do this. :)

CSS/SCSS

Let's talk about this in pradyunsg/sphinx-basic-ng#3.

Translations

100%. I actually also want to work on RTL support in this, because that's definitely one of those things that's really tricky to do.

I don't think we're at a point where we can talk much about this yet TBH, but hey, this is definitely on the radar.

I have a PR that implements some of the "component" ideas that I had - would love feedback from folks:

I've left some feedback (basically saying "I don't want to use theme_options, and please elaborate on why the current approach is insufficient for this"), but I've elaborated a lot more here on why that idea doesn't fit how I was modelling this in my head. :)

Let's continue the discussion on that issue tracker though, likely in pradyunsg/sphinx-basic-ng#2.

we should start making a demo child theme sooner rather than later, to understand how this will work

Yup, yup. The way I was thinking of doing this was to create a bunch of sample documentation sections in the tests/ directory that do all the things that we might want to (barebones does nothing, we'd have something that does roughly what SBT does, one for Furo, etc).

And then we can add proper tests for those (once we've settled on the overall approach/design).

And, it helps that I took this out of a half-done skeleton that was basically "Furo's cousin, for documentation that doesn't fit well in a single sidebar".

choldgraf commented 3 years ago

A few thoughts below..

before realising that I'm not even in the organisation LOL

Maybe we should create an @executablebooks/theme team for this organization, and add folks that are interested to it? We do not have a principled process for adding new people to the EBP members list, what responsibilities come with it, etc. Perhaps starting off with this team will be a step in that direction.

Although if folks would rather we spread this responsibility out or have someone else be responsible for this, just say so.

I'm fine with the current approach - the most important thing is that everybody's perspective is considered and given fair weight 👍 agreed that it is more efficient to have one or few people making decisions initially, so long as the feedback process is also given strong attention.

positions the various "components" (eg: sidebars, content, header, footer etc) and those components contain the "elements"

In my mind, component is a much more specific thing than a general section of the page (I think component and element are roughly equal in my mind). I think this is because of web-components which tend to be scoped to a similar complexity on the page.

because you can't remove a parent theme's theme_options

I'm gonna take discussion of this over to https://github.com/pradyunsg/sphinx-basic-ng/issues/7

making a demo child theme sooner rather than later,

I think there should be at least 2 design cases here:

  1. Furo/Book theme - left sidebar w/ navigation, content is rest of screen to right
  2. PyData theme - topbar has more complex elements (left + right elements for example), similar furo/book theme structure beneath

I think it'll be hard to learn where the design choices are clunky until we try to recreate the current themes with this foundation.

choldgraf commented 3 years ago

If there's 2 tags (i.e. elements) in the header component, they would be left-right aligned

Do you think you could bring that discussion to https://github.com/pradyunsg/sphinx-basic-ng/issues/6 ? It is really hard for me to follow all of these nested topic discussions in a single github issue 😬