Add support for installing themes from Python packages

[rfleschenberg]

I think it would be cleaner if pelican themes were just plain Python packages that could be installed using setuptools / pip. This would enable users to rely on pip freeze / pip install -r for repeatable virtualenvs.

I believe it should be possible to do this change while retaining backwards compatibility with existing themes.

[rfleschenberg]

Maybe pelican-themes could detect if a theme is a Python package or not (presence of setup.py) and convert old-style themes to Python packages on the fly. Or maybe Pelican could support two ways of distributing / installing themes for a transitional period?

[ingwinlu]

here is my input on the topic: creating python packages from existing themes is easy, a simple setup.py mentioning the appropriate directories, author,... to build and upload it and an empty __init__.py file is enough.

the pelican core itself would need a utility function using inspect to grab the install path of the module-theme to pass towards the jinja engine, i.e. something like this:

>>> THEME = 'pelican-twitchy'
>>> theme = __import__(THEME)
>>> os.path.dirname(inspect.getsourcefile(theme))
'/home/winlu/gitrepos/pelican_my_themes/pelican-twitchy'

note the __import__ call because dashes are not allowed in module names.

example theme-package: https://github.com/ingwinlu/pelican-twitchy/tree/package

i know that this topic has been discussed in the past in IRC but no official issue has ever been created (to my knowledge) and input from maintainers is needed.

[rfleschenberg]

I am actually not sure if package management for themes is really needed. If you assume that almost everybody will modify the installed theme anyway, it is probably sufficient to just tell users to download a theme, place it on their filesystem and modify the THEME setting accordingly.

However, if you want to install themes by writing to site-packages, then you should do it properly (by using existing package management tools, and without modifying an already-installed package).

[FedericoCeratto]

Many users cannot or don't want to install stuff downloaded from GitHub as root. Maybe the documentation and pelican-themes's manpage should recommend creating local copies as a preferred option (especially given that users might want to tweak the themes).

[MinchinWeb]

I think there's some value in being able to install a theme from PyPI; I think it decreases the learning curve when starting with Pelican for the first time, and is also helpful if you want to get a site up quickly.

If anyone else wants to do this: To post my theme to PyPI, I first created a setup.py (like any other Python package, as @ingwinlu did), but I also created a small function that returns the theme location on the filesystem. Then in my pelicanconf.py, I specify my theme (Seafoam) as follows:

import seafoam

THEME = seafoam.get_path()

The get_path() function is very simple: it just returns the root directory the file is stored in:

from pathlib import Path

def get_path():
    return(str(Path(__file__).parent))

[justinmayer]

While something like Peru was floated as an idea in #1655, I think standard Python packaging is the best way forward. Hopefully the community can come together to implement that for the next major Pelican release. :sparkles:

[kdeldycke]

I just did as @MinchinWeb proposed and also ended up packaging my theme in a standard Python package: https://pypi.python.org/pypi/plumage . Works great so far!

[kdeldycke]

I guess this issue can now be closed: Pelican perfectly support plugins and themes as Python packages.

[justinmayer]

Pelican 4.5+ supports installing plugins via Pip, in which those plugins are automatically detected and registered, and they become available immediately without any further action needed by the end user.

Ergo, the scope of this issue has been narrowed to track the addition of similar support for Pip-installable themes, perhaps as namespace packages.

Unlike plugins, presumably only a single theme can be active at a time (at least until someone decides to take up the torch and implement support for multiple concurrent themes). While selection from multiple installed themes could be done via the existing THEME setting, the absence of that setting would leave the question of how should Pelican choose which theme to use (e.g., alphabetical, last-modified date, ???).

[MinchinWeb]

I would love to see official/standardized support for installing themes as namespace plugins!

Because my theme is (currently) effectively a "standard" plugin, I've made use of the plugin capabilities too, and do things like supply a "default configuration", inject the theme version into the template context (so it can be displayed on the generated site), and adjust the CSS classes of the output HTML. I think this is powerful, and enjoy having the code together. I don't know if this is a good pattern to follow, but if the theme plugin had to be "pure" I'd probably end up writing a "theme companion plugin"...

One thing that would be helpful is a Pelican signal to set the theme. (My theme current works by adjusting the pelican settings directly.) This would make it "official" that the plugin is setting the theme. It would also provide a helpful place for Pelican to check that the signal has only been called once and raise a warning (or error) if it gets called a second time.

As for dealing with multiple attempts to set the theme, how does Pelican current deal with multiple attempts to set a reader for the same content type? Maybe that provides a pattern to follow. If not, I would go with the first loaded (does that mean alphabetically?) rather than make something complicated, and display a warning to the user. Perhaps the "manual override" would be to set the THEME constant in the settings to the theme you want to use (and skip the warning); this too could be helpful if you want to install all the themes and then regenerate the site with each theme, like for a theme sample site.

[avaris]

I'll handle the implementation details but I need theme developers to chime in with their requirements. Currently I'm thinking about a structure similar to namespace plugins.

pelican
  |- themes
       |- theme_name
            |- __init__.py
            |- # other stuff

It will be imported like a regular package. What should go in __init__.py will be the quasi-required interface. I'm thinking about an initialize(settings) method minimum. In there, theme can do any house keeping, define defaults, inject variables, etc. And potentially a method or two to return THEME_STATIC_PATHS and templates folder. __version__ string could be one as well, that is auto-injected to context with THEME_VERSION or similar, or it could be left to the theme to handle that. Rest is up to the theme. They can have their own built-in plugins, connect to signals etc. if they wanted, they will be python packages after all.

Auto-activation of a theme would be doable, but in case of multiple themes installed, I'd rather have this be a fail condition than arbitrarily choosing one. Stop with an error message to force THEME setting or -t option. Even more, we could extract notmyidea (and even simple) to their own packages with those as (optional?) dependencies of pelican. This might complicate auto-activation but they can be special cased, so "installed themes" would be anything except the defaults at least for the internal logic. i.e. fail if there are more than one non-default themes installed.

One thing that would be helpful is a Pelican signal to set the theme. (My theme current works by adjusting the pelican settings directly.) This would make it "official" that the plugin is setting the theme.

You can use initialized signal to configure THEME setting.

As for dealing with multiple attempts to set the theme, how does Pelican current deal with multiple attempts to set a reader for the same content type?

Well, not sure how this relates to themes, since Reader and theme are quite different things. Readers are associated with extensions, pretty much a dict of extension: Reader pairs. If you set multiple to the same extension, last one will replace any previous ones.

[MinchinWeb]

I like what's proposed!

In there, theme can do any house keeping, define defaults, inject variables, etc.

Would you expose the typical signals available to plugins? Or perhaps a subset of those?

And potentially a method or two to return THEME_STATIC_PATHS and templates folder.

This is probably wise. But should a default be assumed? I think most of the time, a default (like a \template and a \static directory within the theme root) would work, and might avoid some boilerplate. I guess the downside is it makes it harder to change that assumption later.... In any case, it might be wise to present a "standard convention" in the related documentation.

__version__ string could be one as well, that is auto-injected to context with THEME_VERSION or similar

Yeah, that should work. Alternately, export it as <theme_name>_VERSION, e.g. SEAFOAM_VERSION.

Auto-activation of a theme would be doable, but in case of multiple themes installed, I'd rather have this be a fail condition than arbitrarily choosing one. Stop with an error message to force THEME setting or -t option.

I like auto-activation, because I think for most people, in most times, this is going to work for them. Good error messages can help solve the rest.

One thing that would be helpful is a Pelican signal to set the theme. (My theme current works by adjusting the pelican settings directly.) This would make it "official" that the plugin is setting the theme.

You can use initialized signal to configure THEME setting.

That is what I'm doing now, but it always seemed kind of icky to mess directly with the configuration variables. I guess I worry about changing something at the wrong time, so it doesn't get picked up for the full site generation run and then Pelican becoming very hard to debug, or a new version of Pelican changing the configuration variable name. I thought a dedicated set_theme (or similarly named) signal might be cleaner and more explicit. Also, with a dedicated signal, if I ever have to debug the theme selection code (inside Pelican), there is an obvious place to look: much of my plugin writing has required peering deep into the Pelican internals to figure out how to make the things happen that I want to happen.

As for dealing with multiple attempts to set the theme, how does Pelican current deal with multiple attempts to set a reader for the same content type?

Well, not sure how this relates to themes, since Reader and theme are quite different things. Readers are associated with extensions, pretty much a dict of extension: Reader pairs. If you set multiple to the same extension, last one will replace any previous ones.

I'd just thought there might be some existing code that could be reused, or at least a pattern. After all, only a single Reader is active (per source format) as Pelican get running.

Hope that's helpful! Let me know if I can answer any other questions.

[avaris]

In there, theme can do any house keeping, define defaults, inject variables, etc.

Would you expose the typical signals available to plugins? Or perhaps a subset of those?

Nothing is hidden, so nothing to "expose". They are effectively like plugins, i.e. functionally more or less identical. Nothing will stop a theme from doing:

from pelican.plugins.signals import finalized

def post_process(pelican):
    # do stuff

def initialize(settings):
    # do stuff
    finalized.connect(post_process)

And potentially a method or two to return THEME_STATIC_PATHS and templates folder.

This is probably wise. But should a default be assumed? I think most of the time, a default (like a \template and a \static directory within the theme root) would work, and might avoid some boilerplate. I guess the downside is it makes it harder to change that assumption later.... In any case, it might be wise to present a "standard convention" in the related documentation.

Sure, a default is probably needed, especially to support current theme implementations (therefore quasi-required). Something like, call these to get values if they are present, otherwise assume default.

__version__ string could be one as well, that is auto-injected to context with THEME_VERSION or similar

Yeah, that should work. Alternately, export it as <theme_name>_VERSION, e.g. SEAFOAM_VERSION.

I'm more inclined to the generic and more predetermined THEME_VERSION, but can be discussed :).

You can use initialized signal to configure THEME setting.

That is what I'm doing now, but it always seemed kind of icky to mess directly with the configuration variables. I guess I worry about changing something at the wrong time, so it doesn't get picked up for the full site generation run and then Pelican becoming very hard to debug, or a new version of Pelican changing the configuration variable name. I thought a dedicated set_theme (or similarly named) signal might be cleaner and more explicit. Also, with a dedicated signal, if I ever have to debug the theme selection code (inside Pelican), there is an obvious place to look: much of my plugin writing has required peering deep into the Pelican internals to figure out how to make the things happen that I want to happen.

Setting theme at runtime is something I'm struggling to find a use case for, but I'm not opposed to it. We can find an appropriate place and a name for such a signal.

Well, not sure how this relates to themes, since Reader and theme are quite different things. Readers are associated with extensions, pretty much a dict of extension: Reader pairs. If you set multiple to the same extension, last one will replace any previous ones.

I'd just thought there might be some existing code that could be reused, or at least a pattern. After all, only a single Reader is active (per source format) as Pelican get running.

Not really. Code for namespace plugin support will be definitely handy. However, I suspect, most of the effort would be extracting theme related logic that is scattered around the codebase to a dedicated place and define a clean interface for it. Depending on the scope of refactor, it could end up as a breaking change. Not for the user, at least that's what I'll be aiming for. But I don't want to guarantee that for the existing plugins, some might require a tweak or two :).

[justinmayer]

@avaris said:

I need [Pelican] theme developers to chime in with their requirements.

To that end, I am copying some folks who have experience with Pelican themes in the hope that some of you will comment here with input regarding what you believe you need from official Pelican support for themes-as-Python-packages.

Would you please take a moment to post your thoughts here?

cc: @alexandrevicenzi @talha131 @iranzo @mosra @jorgesumle @frankcorneliusmartin @arulrajnet @kdeldycke @gfidente @aleylara @pchampin @iKevinY @kplaube

[alexandrevicenzi]

Hi @justinmayer @avaris

With a package, it should be possible to require extra plugins automatically. This is good if you want to implement a feature based on a plugin, this would avoid IFs everywhere checking if a plugin is installed or not.

If you plan to include a theme initialization step, I would like to see options to create custom Jinja filters and helpers that I can use in my theme without any plugins. This saves some time with IFs everywhere or custom macros.

It might be useful to have pre/post processor hooks, to inject something before/after Pelican. This might be useful if you want to implement something like Hugo Shortcodes.