Allow theme = ["base", "my-theme" ] (aka theme composition and inheritance)

bep commented 6 years ago

I know this has been requested before, in the form of some extends = some-other-theme.

This isn't "theme inheritance" in its purest form (which would be "theme1 extends theme2 extends theme3"), more like theme composition. Which is much simpler to understand/implement, but with most of the added benefits.

This relates to my work on https://github.com/bep/html5up-to-hugo -- which, with the current Hugo, becomes less elegant than it could be.

The simple rule is that the themes' files will form a big overlay/union file system from left to right.

So:

theme = ["base", "my-theme" ]

All files in base will be used unless there is a file with the same name in my-theme.
This will work with layouts, data, i18n, static (and in the future: content etc.)

What do you say?

bep commented 6 years ago

Note that this is a "proposal", I'm not done with my part of the thinking here -- maybe this belongs in the theme itself etc...

RickCogley commented 6 years ago

One thing I see on the forum a lot is, "how can I make page X look different from the rest of the site". Could this facilitate styling a page like that, kind of overriding the theme applied to most of the site?

brunoamaral commented 6 years ago

From @bep's description, yes it does. You could for example have your own version of single.html or of a shortcode.

I can see how this is useful. It's the same mechanic that Wordpress uses in creating 'child-themes', but I am not a fan. I remember losing track of whether I was editing the right files.

It's a personal perspective. Others will make it useful and may help produce more themes for Hugo. I don't have the knowledge to comment on the implementation. So I will abstain from voting and may even end up using it anyway.

kaushalmodi commented 6 years ago

+1 if I can have my personal "theme" of partial and shortcode collection, and can keep that theme as the secondary theme.

Thinking of something like:

theme = ["actual-theme", "partial-shortcode-collection-theme" ]

I've wanted this feature for a while. I had tried symlinking my commonly used partials/shortcodes from a common git repo, but symlinks don't work.

RickCogley commented 6 years ago

@kaushalmodi +1 for that. That's a cool idea.

Jos512 commented 6 years ago

Child themes are a great idea. In theory a feature like this should make it easier to get started with Hugo for beginners. And they probably also make it a lot easier for people to update the original, parent theme.

bep commented 6 years ago

@RickCogley not sure about your example, but @kaushalmodi 's example is a primary use case, and I think this becomes super-powerful when we get proper dependency management, i.e:

theme = ["https://github.com/spf13/hyde/v2", "https://github.com/kaushalmodi/shortcode-pack/v1" ]

When Russ Cox is done with his brilliant thinking, I'm going to steal his thoughts: https://research.swtch.com/vgo

To me, this is mostly about DRY. On the project side, we can already today extend the theme by adding a single.html or overriding a partial etc. But that is not an option in a theme. So we get themes like hydex and hydey -- only slightly different ports of the same theme. If you want to create a new hyde theme with some blue colour, this should be possible without too much duplication.

Thinking about it, I think I will try to get this to work both on theme and in project level (themable themes).

So in theme.toml for my new Hyde theme hyde32:

# This theme is based on SPF13's port of Hyde, but with prettier colours.
theme = ["hyde"]

And then in my config.toml:

theme = ["hyde32", "partial-shortcode-collection-theme"]

The ordered set of file collections will then be:

hyde, hyde32, partial-shortcode-collection-theme

This may be too complex if the graph gets too big. Will think.

In its first iteration, the end user is responsible for pulling the above 3 themes into /themes -- but the ultimate goal here is to do:

# Shallow clone, no themes (only referenced in config.toml)
git clone https://github.com/bep/my-site
cd my-site
# hugo will download dependencies if not cached.
hugo

regisphilibert commented 6 years ago

I think it will be very useful. One use case coming to mind is with output formats. If you want to add an "api" (json output) layer to your content, all you'd have to do is add a special "api" theme and reference it. Rather than singlehandedly adding the json templates in your layouts/_default or other.

regisphilibert commented 6 years ago

Also it will allow theme developer to release several "extension" for their theme which not everyone will need. Like a "photo gallery" extension or a "Dentist" extension for a theme whose base is made for broader "health" structure.

budparr commented 6 years ago

This would be very useful. My use case is similar to @kaushalmodi in that I keep a set of really basic files (_default layouts, etc.) to reuse on all projects and then layouts/themes for specific types of projects on top of that.

bep commented 6 years ago

A development tip: Features like this are hard to get going unless you have a solid and failing test up and running:

const (
        themeStandalone = `
            name = "Theme Standalone"
`
        themeCyclic = `
            name = "Theme Cyclic"
            theme = "theme3"
`
        theme1 = `
            name = "Theme #1"
`

        theme2 = `
            name = "Theme #2"
            theme = "theme1"
`

        theme3 = `
            name = "Theme #3"
            theme = ["theme2", "themeStandalone", "themeCyclic"]
`

        theme4 = `
            name = "Theme #4"
            theme = "theme3"
`

        site1 = `
            theme = "theme4"
`
        site2 = `
            theme = ["theme2", "themeStandalone"]
`
    )

With a test that fails to build site1 and site2 -- then the fun starts! It is a threshold of boring work to get there.

regisphilibert commented 6 years ago

On more use case is "shortcode" bundles. Your theme layout files would only contains shortcode files and people could use them by just adding your theme to their "theme set" and easily upgrade with future updates.

I am not very intelligible when excited.

Just saw @kaushalmodi's comment above :/

bep commented 6 years ago

I just figured out that this would also be really cool for bundling of shortcodes!

:-)

regisphilibert commented 6 years ago

Or bundling of output formats templates ! 🎉
You could even design an "Algolia" search theme "addon" along with script, css, template, output format template etc...

So much possibilities!

mwyvr commented 6 years ago

Yep, all those things that we copy from theme to theme, often without changing a single line. Shortcodes, output formats. Heck even "base" for many.

Note if it makes dealing with static asset customization (like a site font file or landing image

bep commented 6 years ago

Just a quick note that this is still pretty much priority.

In Hugo 0.38 themes got config.toml with namespaced .Params etc. and this will, of course, extend to these composites. Which will make something really powerful.

regisphilibert commented 6 years ago

Can't wait for @kaushalmodi to make a mini-theme out of his debug functionalities! (debugprint and so forth...)

kaushalmodi commented 6 years ago

@regisphilibert Thanks! By now, I have couple of "mini-themes" I'd like to roll out once this feature is out:

Debugging (debugprint.html partial, Kind/Type/etc. debug info, etc.) [DONE: hugo-debugprint]
ATOM, jf2 as custom output formats [DONE: hugo-atom-feed, hugo-jf2]
Search based on fuse.js [DONE: hugo-search-fuse-js]
Better versions of internal shortcodes and templates: figure, opengraph, twitter_cards, ..

bep commented 6 years ago

@kaushalmodi @regisphilibert coming back to this issue after some months, I was a little bit confused about my own writing above.

Given:

theme = ["base", "my-theme" ]

On duplicates, what theme would you say should win? base or my-theme?

bmackinney commented 6 years ago

👍 my-theme

kaushalmodi commented 6 years ago

@bep Of course my-theme.

I can visualize that setup as this pseudo code:

theme my-theme extends base-theme
  function single.html
    """
    Only single.html from my-theme overriding that from base-theme
    """

kaushalmodi commented 6 years ago

If my-theme is overriding unwanted layouts and config params, user should trim them out.

bep commented 6 years ago

I guess the theme names masked my real question, how about this:

theme = ["theme1", "theme2" ]

Who should win? Is it now obvious that "theme1" is the base theme? I obviously thought so back in February, just wanted to make sure ...

vassudanagunta commented 6 years ago

Be consistent with #4436

kaushalmodi commented 6 years ago

how about this:
theme = ["theme1", "theme2" ]

I think either direction (left to right, or right to left precedence) would work; just needs to be documented.

But I would suggest left to right increasing precedence order.. here's a reason:

A user could have this initially:

theme = ["some-theme"]

Then they choose to steal certain layouts from a different theme. So they then have:

theme = ["some-theme", "other-theme"]

i.e. they just need to append that to the right. So the delta of change only happen towards the right of that line.

Later, they tweak some portion of that "other-theme" and override just the RSS layout, so it will look like..

theme = ["some-theme", "other-theme", "my-rss"]

What I am getting at is that the initial theme = ["some-theme" part stays constant throughout. If the precendence is reversed, the base theme position will keep moving around.

regisphilibert commented 6 years ago

theme-2 👍

It is my understanding that in any system, hardware of software, the layer on top will hide/override the layers below. As a left to right reader, it therefore makes sense to me that theme-2 is the layer on top.

Ultimately of course and that goes without saying, project's layouts should remain the utlimate rulers of this ecosystem :)

bep commented 6 years ago

One last question. Since I wrote my test on this, a theme can now have config.toml (i.e. site configuration). I also has a theme.toml, which is information about the theme -- we use this info to build the theme site etc.

So,

With this new feature both the project and the themes can be ... themed.

The question is:

Where to put the themes' theme configuration. theme.toml or config.toml? (and please don't suggest that we merge those files into one ...)

regisphilibert commented 6 years ago

With this new feature both the project and the themes can be ... themes.

Wait a minute, why?

bep commented 6 years ago

Wait a minute, why?

I added a missing "d" to the above, which may clear things up.

The obvious motivation behind this is to create a new hyde port without having to copy-and-paste everything (which will be bad, according to EU and DRY).

So I create my hyde-z theme and configure it to be based on some other theme: theme = hyde-x

bep commented 6 years ago

theme = ["some-theme", "other-theme", "my-rss"]

Yes, that makes sense. Another obvious use case would be shortcode collections, so:

theme = ["my-theme", "cool-shortcodes"]

In the above, on duplicate shortcodes, the right will win. If the shortcode exists in the project, that will of course win (as it is today).

regisphilibert commented 6 years ago

I see two ways to do this. The old way: Add your layouts files in layouts directory and set your theme as hyde-x: themes = 'hyde-x' The new way: Create a /themes/hyde-z directory alongside themes/hyde-x and set theme both in your config.toml: themes = ['hyde-x', 'hyde-z']

In both case, your project is not a theme, so I am still confused about your previous sentence :/

kaushalmodi commented 6 years ago

Where to put the themes' theme configuration. theme.toml or config.toml? (and please don't suggest that we merge those files into one ...)

I honestly think of theme.toml less of a functional config and more of a meta-data config.

I think the theme should go in config.toml. Now.. with that, I believe that if the theme's config.toml has:

# theme-x's config.toml
theme = [ "_this_", "cool-shortcodes"]

and user's site config.toml has:

# user's site config.toml
theme = [ "theme-x", "my-shortcodes"]

Looks like we will need a special placeholder like "_this_" in theme config? Otherwise, how will "theme-x" refer just to its own stuff without including the extensions?

Does that effectively become:

# user's site config.toml
theme = [ "theme-x._this_", "cool-shortcodes", "my-shortcodes"]

regisphilibert commented 6 years ago

I understand now. I always assumed the user would add themes’ port himself in his config.toml.

I don’t think a theme should be able to set itself as a layer of another theme. This is prone to inheritance being unknown to the user.

The port should let the user know he/she needs both themes in his themes directory and reference them both in his project’s config.toml this order. This would ensure maximum control to the user at very little cost.

bep commented 6 years ago

@regisphilibert your project is not a theme, but it can be themed ... Which in my English means that you can configure it to use a theme.

@kaushalmodi

If project is:

theme = [ "my-theme", "my-shortcodes"]

And theme is:

theme = [ "theme-x", "even-cooler-shortcodes"]

The final graph will be:

[ "theme-x", "even-cooler-shortcodes", "my-shortcodes"]

The above assumes that "theme-x" is shallow and it does not include the content of my-theme as I didn't find a way to put it in there.

bep commented 6 years ago

@regisphilibert maybe ... I'm not totally convinced either way. We have lots of very similar themes as it is today. People create a new theme just to change some minor thing. And since you currently would have to do manual clones, It would not be "unknown to the user".

kaushalmodi commented 6 years ago

I'm on a move, so cannot articulate properly right now.. may be do something like:

# theme config
theme_extensions = ["a", "b"]

If user uses that theme without modifying theme-extensions, they inherit the theme's settings as they do now.

If user wants to add their own extensions, they need to do:

# user site config
theme_extensions = ["a", "b","c"]

bep commented 6 years ago

In both case, your project is not a theme,

But it can be themeD?

bep commented 6 years ago

... that said, I see enough questions in the above conclude what we can wait with the "inheritance" part, and start with the simpler setup where a project only can use themes (i.e. composition).

kaushalmodi commented 6 years ago

Though, don't you think that having an extra theme_extensions for list config param will make this easier and leave the theme as a single string value.

bep commented 6 years ago

Though, don't you think that having an extra theme_extensions

What is the difference between a theme and a theme_extension? Why create one more term if they are the same? "Themes can be composed" is simpler to explain to people than "a project can have a theme. You can also add theme_extensions, which is the same as a theme, but just named differently."

kaushalmodi commented 6 years ago

It will blend into the theme extension inheritance that I talked about in my comment earlier.

With just one theme param, it will get difficult if users want to remove one or more of the extensions from the original theme.

Having the extensions, using the same example as earlier, user can set the extensions param as follows:

[] # empty
a
a,b,c
c
if they don't set, they get the extensions the theme defined (a,b).

bep commented 6 years ago

@kaushalmodi you have a point, but if we start with "theme in project only", this can wait. It is a simpler model.

If we somehow need to control order at some point, we could allow the theme as a more complex structure:

[themes]
[themes.my-theme]
weight = 1

Which then could override the default right-to-left order you get.

kaushalmodi commented 6 years ago

@bep

My suggestion for theme_extensions was to deal with the "deep/shallow" theme reference control.

Quoting you above:

If project is:

theme = [ "my-theme", "my-shortcodes"]

And theme is:

theme = [ "theme-x", "even-cooler-shortcodes"]

What are "my-theme" and "theme-x" there?

Let me use sort of a real example to convey my point.

Let's say my "hugo-bare-min-theme" contains the "debugprint" extension that contains the related HTML and CSS partials.

So how should that be specified in the hugo-bare-min-theme's config.toml? Like this?

# theme config.toml
theme = ["hugo-bare-min-theme", "debugprint"]

And how will a user refer to the hugo-bare-min-theme, including the extensions that I want to ship with that theme by default? By just doing this?:

# user site config.toml
theme = ["hugo-bare-min-theme"]

Isn't that confusing that inside the theme config, the theme name is shallow reference, and in site config, the same is deep reference (i.e. auto includes the theme's default extensions?)
What if the user wants to disable one or more of the theme's default extension? Are you planning to support that?

This is how above will look with my proposal:

theme config

# theme config
# no need to specify theme..
theme_extensions = ["debugprint"]

user config, if wants to use the theme's default extensions

# user site config
theme = "hugo-bare-min-theme"

user config, if does NOT want to use the theme's default extensions

# user site config
theme = "hugo-bare-min-theme"
theme_extensions = []

user config, if does NOT want to use the theme's default extensions, and add his own extensions

# user site config
theme = "hugo-bare-min-theme"
theme_extensions = ["my-extension"]

user config, if wants to use the theme's default extensions + add own extensions

This follows the existing behavior that if user wants to change a param that's defined in the theme's config, they need to define it entirely.

# user site config
theme = "hugo-bare-min-theme"
# need to copy the below param from the theme first and then add more extensions
theme_extensions = ["debugprint", "my-extension"]

I think this looks very clean in the config.

The meaning of the current "theme" param does not change.
If someone doesn't want to change any of the theme's extensions, they define the theme as usual.
theme_extensions is needed to be defined only by people who want to add/remove from a theme's default set of extensions, or club multiple themes.

bep commented 6 years ago

What if the user wants to disable one or more of the theme's default extension? Are you planning to support that?

No (or maybe, but certainly not in the first iteration).

You started the talk about extensions and extensions in the scope as something optional, not me.

We may leave the "allow theme to set themes" out in the first round, but my general view on this is that:

If a theme theme-a says that it depends on theme-b then theme = theme-a ∪ theme-b. You cannot remove theme-b from the theme and expect it to work. This brings in a relevant discussion about transitive dependencies in Go programs (see #4754). It becomes hard when you want to version them etc., but you cannot ignore the problem. And you cannot remove dependencies and expect the program to compile.

But we will have to start simple here. If Hugo gets really popular we can maybe adopt vgo's model and get versioned imports and hugo get etc.

That said, I have been thinking a little about this when sleeping. And I think we're going to reverse the order of precedence in the theme slice compared to what we talked about above.

It may be slightly less intuitive, but it will be consistent with #4436 (/cc @vassudanagunta) and it will make it more clear when you add "extension type themes" to the mix.

So in the project's config.toml:

theme = ["cool-shortcodes", "the-theme", "my-base-theme"]

The theme's config.toml:

theme = ["my-cool-shortcodes", "cool-shortcodes"]

The precedence is from left to right.

I have put cool-shortcodes first, because I want my shortcodes to be used and not the collection (my-cool-shortcodes) provided by the-theme.
cool-shortcodes is also included in the theme, but will not be included twice.
my-base-theme is to the far right with the lowest precedence.
This also works mentally when you think about the project as the ultimate winner (far left, before the = sign).

Also note that we currently do not have a "global identifier" for the themes, so the theme name whatever you name it when you clone it on disk. So you can certainly exclude a theme by putting some empty theme in the named folder. This is another big question that we need to keep out of this discussion.

regisphilibert commented 6 years ago

So, I'm confused now, as of this commit: Can themes declare their own "dependencies"?

regisphilibert commented 6 years ago

Here comes another question, sorry.

If I understand correctly, theme's config.toml will bear its own theme declaration, we'll call it theme.theme. So will, as always, the project, we'll call it project.theme.

Now, If I understand the current state of inheritance/overriding of config.toml, the project's overrides the theme's.

So no matter what theme.theme is set to, will it not be systematically overridden by project.theme? This is undeniably what the user will expect anyway regarding any setting present in the theme's config.toml.

bep commented 6 years ago

Can themes declare their own "dependencies"?

We may wait with that until a later release, but I think that will eventually happen.

So no matter what theme.theme is set to, will it not be systematically overridden by project.theme?

project.theme = something
theme.theme = something_totally_different

There is no systematic override in the above?

regisphilibert commented 6 years ago

There is no systematic override in the above?

That is what I thought. Unless the slices will be merged somehow? Every project needing a theme will set a project.theme, thus overriding the theme.theme of its needed theme (which was used to declare some sort of theme dependencies)

bep commented 6 years ago

That is what I thought. Unless the slices will be merged somehow?

The slices will not be merged, but there are 3 fairly simple rules to be aware of. If you think of the slices from left to right, and continue that thought into the theme.

If both the project and the theme defines myshortcodes as a dependency, it will only be included once (the first one)
If myshortcodes includes someother-myshortcodes which then includes myshortcodes it is of course also only included once.
All other overrides are done on file level. So layouts/shortcodes/myshort.html will, on duplicates, the left-most version win.

bep commented 6 years ago

And note that myshortcodes is supposed to refer to the same theme, so it should not matter where it is defined. You just need it. In practice, it may be harder, with versioning etc., but we need to solve that problem later.

gohugoio / hugo