RFC: Cascadable ("nested") themes for form styling

[sminnee]

UPDATED 16 JUNE - theme identifiers updated

PR #5471 starts introducing bootstrap HTML for form fields. The issue that @chillu raises is that this will break everyone else's forms that rely on the old HTML. We need to find a way of providing a new set of styles for the bootstrap forms, that can be optionally applied.

One approach to this would be to have a CMS theme that contains all the theme styles, and apply that. However, that presents some difficulties:

• Anyone else who wanted to use bootstrap would need to repeat our work
• The CMS styles would need to be stored outside of the module, in the project's themes folder.

  API changes

• Allow the SSViewer::set_theme() to accept multiple, ordered arguments.
• Allow the following strings to be used to reference packages:
  • <vendor>/<package>: Reference a composer package. The theme will be stored in the root of the package. Eg: "silverstripe/simple".
  • <vendor>/<package>:<theme>: Reference a "named theme" in the package, stored as a direct subfolder of the "themes/" folder within the package. Eg: "silverstripe/framework:bootstrapforms".
  • /<path>: Reference a theme contained within the subfolder of the SilverStripe project. Eg: "/mysite"

In addition, we have the old, deprecated syntax: <theme>. It will reference a "named theme" in the project, that is a theme at "BASE_PATH/themes/". Instead of of this, use the syntax "/themes/". However, a better bet is to move your templates to /mysite, if they are project-specific, or to a composer package that you reference by its fully-qualified name, if they are part of a shared theme.

Default themes

In SS3, templates provided by the mysite folder and each module are treated separately from the theme engine. With the change proposed in this RFC, these would be treated as themes in their own right, e.g. '/mysite' or 'silverstripe/blog'.

This would mean there are a lot more themes to join together.

• SSViewer::default_themes() to return the default theme cascade (/mysite, then every module)
• SSViewer::set_themes() to replace the theme cascade in its entirety.
• SSViewer::add_themes() to prepend additional themes onto the current stack. If add_themes() is passed a theme already in the cascade, then the given theme is pulled to a higher precedence place in the list.

For example, SSViewer::add_themes(["/mysite", "silverstripe/simple"]) will set the site to use the simple theme but ensure that templates in /mysite override the theme.

More sophisticated controls are being left out of the initial implementation until we see more use-cases. However, this approach will make it easier to provide options that tweak the priorities of different themes in ways that would have been awkward in SS3.

Theme bundles

The current behaviour that the theme "simple_forum" will be automatically included if you have selected the "simple" theme will be dropped. Instead developers will need to use: SSViewer::add_theme(['simple','simple_forum'])

Limitations

We're not, in this change, going to write the code that handles storing themes in the /vendor folder or looking for them in arbitrary locations. So, we referenced themes must be of type silverstripe-theme or silverstripe-module, and sub-themes must existing in packages of type silverstripe-module. We will rely on our existing rules for package folder location for now, and solve that in the future.

The code will need to look for the theme '/' in either '/themes/' (look here first) or '/' (look here seocnd). The potential for name conflicts is an accepted limitation for now.

Use

With theme cascade in place, themes can be created that cater to specific needs. For example, we can create a silverstripe/framework:themes/bootstrapforms theme that will provide form styling appropriate for bootstrap. This can be included into the CMS without disrupting other use-cases.

Alternatives

Possible ideas:

• We could hard-code module-packaged themes to be stored at themes/<subthemename>. This would make it easier to build a manifest of them. It would also make theme references shorter. However, it wouldn't be clear how to reference the content of mysite/templates as a theme.
• We could let module provide yaml config that provided a mapping of longer theme identifiers to shorter theme names. This would be another way of building a manifest of themes, but would make the manifest generation reliant on config generation, which may be a circular dependency.

Ideas I don't like:

• Rather than allowing arbitrary theme folders, we could either allow any modules to have a theme subfolder, or we could specify multiple THEME_DIR values. This strikes me as a bit inflexible, and not substantially beneficial. It may be a simpler starting point, if we wanted to start with a smaller change.
• A construct other than themes for form styles. However, I don't think that we need multiple abstractions for "a set of templates and associated resources" — it's better to make themes more flexible.

  Future ideas

I've deliberately left these out of the RFC so we can focus on implementing the first step. These ideas may happen in the future, but I don't want them to prevent us from starting.

• Some kind of registry of theme name => theme location mapping, e.g. in the config yaml
• Theme inheritance
• Theme clustering (so that the forum add-ons for the simple theme can be automatically added)
• Yaml configs for themes.

  Tasks

• [x] Theme lookup changed from single theme to list of themes
• [x] Themes can be looked up in four locations listed
• [x] API for setting theme list added
• [x] API for getting "default" theme list added
• [x] Tests
• [ ] Documentation
• [x] Fix broken template references

[tazzydemon]

I'm guessing none of this is to be retrofitted to 3.4 (which is what I am working on)? I'm looking forward to shifting to 4 asap because it is so fast. 3.4 is definitely not. Yes consistency. I thought just moving the framework one was the easiest and most logical but as usual it broke something else!!

[tractorcow]

Sorry, I'd love to back-port some logic to 3.x, but the way it's currently set up it'll be nigh impossible to move templates without regressions.

[tazzydemon]

As you have suggested, as long as I know and it’s consistent.

[pitchandtone]

While we've just started using this new system, there's two things that come to mind: 1) "inherit from other themes, and define codeless page types" are probably two features not one, I suspect the latter is more interesting to devs 2) This seems to introduce a bunch of headaches around the inheritiance idea, which from my perspective has questionable value to actual devs and only solves the issue with breaking people's current form styling.

Can someone explain why cascading themes is a good idea?

[pitchandtone]

For the new form templates, can't we just be brave go to the new template and provide an additional module that could be included to override this if someone needed to stay with the old templates.

[sminnee]

The primary use case of the cascaded themes so far is for adding "form-only" themes into the mix - you can define a theme that provides HTML, css, and JavaScript for say a bootstrap boilerplate, and then add site-specific templates on top of that.

The other thing is that we've always had a cascade, it's just been a stupidly hard-coded one of modules -> theme dir -> project dir.

[sminnee]

It would be good if you could elaborate on he problems that you're having.

[pitchandtone]

Why would the old system not be suitable to the "form-only" issue?

[sminnee]

We're not going back to the old system on the basis of a non-specific complaint. What makes be new system so difficult for you?

[sminnee]

OK, so a bit more context:

Template cascade: it's always been there, it's just less of a hack now

In SS2-3 we support templates and client resources provided by the project, by modules, by themes, and by sub-themes (e.g. "blackcandy_blog")

The "cascade" is simply the order in which these resources are searched, the first matching templating being used. In SS2-3 this order changed a few times, but it was hard-coded to:

• Project
• Theme
• Sub-themes
• Modules (in alphabetical order)

This has some issues:

• The handling of themes, modules, and templates were quite different. There were two parallel paths for similar sets of code. It was messy.
• The specific order that we had baked into the concrete worked for most people, but not all. It wasn't something that should be 1-setting-for-everyone, as the range of use-cases were quite diverse. Some people always used themes, some people never did.

So in response we unified the template engine's handling of modules and themes, and referenced everything as either:

• A composer package, or
• A subdirectory of a composer package (e.g. silverstripe/admin:/themes/cms-froms)

Internally a prioritised list of all such resources is maintained, and that defines template override rules. There's a default "ThemeManifest" that mimics the prioritisation rules of SS3, but that can be removed and you can change the order however your use-case dictates.

We had one specific issue which we decided to park until this feature was implemented, and that was the handling of bootstrap forms in the CMS. We recognised that having. Sure, we could remove the legacy forms and replace them with bootstrap forms, but what if someone wanted to use foundation, or another library that didn't work nicely with our default bootstrap forms? Being able to have a template-pack that took care of form styling for the CSS framework of your choice seemed like a nice thing to support. However it was more "the straw that broke the camel's back" than the reason we did this.

In the end we never implemented the idea of theme inheritance, instead just letting people specify and ordered list of themes.

Template namespacing

There's another change we made to template handling around this time, and that was the handling of namespaced templates. Template names generally match classnames, and with the introduction of namespaces, this meant our templates were namespaced too. In the end we decided on:

• Use namespace in the template names too
• Use a folder convention similar to PSR-0, where namespaces are turned into subfolders
• Get rid of the "manifest scan" approach to finding templates in whatever directory they're found, and enforce the use of a namespace-based folder structure.

Optionally, you can use an "Includes" subfolder in this mix, but frankly I think it's a hangover from the old approach to managing template folders and should be deprecated. Others were less sure, but it's possible that this is a source of confusion.

---

So, we've made a bunch of changes for a bunch of reasons, and although I'm happy to admit that we might be able to improve the developer experience, we're going to do that by building on what we have an not reverting it.

[pitchandtone]

Firstly I'm not asking to revert anything, I just didn't see any justification for these changes and I'm trying to understand the bigger picture that you're working with. The previous comment helps this greatly, so thank you.

So to summarise:

Two issues => Two solutions: 1) Different folder structure => Namespaced theme, no template manifest 2) Hardcoded order => Customisable order

What I'm still not clear on is why the form issue was parked until this work was done, I don't see why the hardcoded order wouldn't have sufficed for this?

[sminnee]

In retrospect I guess we could have done form styles as a module. It would have been harder to get precise about the ordering of those templates as vs any in framework. And it wouldn't have been possible to enable/disable them for different UIs. So if we had separate form styles for CMS from the front-end site, we'd have had to have bundled them with an admin theme.

The other thing would have been that "an admin theme" would have been more awkward - it would have had to have been it's own composer package checked out to the themes folder.

None of these problems are insurmountable they just added up to a messier picture.

[pitchandtone]

Thanks for the synopsis @sminnee makes more sense now. I'll raise specific issues if we find them.

[sminnee]

No worries — there's a lot of new stuff SS4 that's appeared over the last couple of years! ;-)

[patricknelson]

Deleted my comments regarding twig/blade blocks due to not being very relevant here, sorry about that. Posted here here instead: #7498