Handling resource compilation in sphinx extensions (SCSS,JS,TS))

[chrisjsewell]

Originally posted by @chrisjsewell in https://github.com/executablebooks/sphinx-panels/pull/32#issuecomment-692871773 (also discussed in executablebooks/sphinx-panels#31)

just to finish my thoughts here, basically as you say each option seems to have an annoying tradeoff. Putting aside JavaScript compilation for now, my ideal goals are:

• Using SCSS as the CSS source, which is linted in the pre-commit

• Having hashed CSS end up in the built documentation

• Having a way of allowing users to configure certain variables (colors/font-size), without having to add their own custom CSS overrides. sphinx allows for these .css_t Jinja templates, but I'm not really a fan of them, because obviously this means now you can't use SCSS and/or lint/compress the CSS. I think the solution I've implemented here works pretty well: dynamic creation of a small additional CSS file, with some custom properties declared in the root namespace.

• Being able to do live development; whereby you can have the documentation in the browser, which updates when your messing around with the any of the Python code, SCSS or RST/Markdown pages

  • This would also be run via tox, so you don't have to set-up development environment manually.
  • If possible it would avoid having to use a nodeJS infrastructure, especially if its just to compile a few SCSS files (although it may be unavoidable if you have more comlex requirements, like pydata-sphinx-theme)

• Not having the SCSS compiler as an install dependency. It's not necessarily a bad option, sphinx after all is a build system; however, the available packages are both not ideal; python-libsass adds a large install overhead, because it has C-bindings, pyScss is lightweight, but not very well maintained, has some bugs, and lacks some key features. Plus it doesn't work great with how sphinx handles theme packages.

• (Having the SCSS and sourcemaps in the built documentation would be nice, but low priority)

[pradyunsg]

Assuming we provide wheels, treating SCSS as a "compilable" artifact should satisfy these. We'd need $compiler-toolchain if we ever go source/git-repo -> installed (instead of wheel -> installed) tho.

[chrisjsewell]

Could you expand on this a bit. What's the physical setup you envisage for the repo/workflow?

[chrisjsewell]

Is there a way to hook into setuptools/distutils etc, to run arbitrary pre-build code (to create the compiled CSS files)? It looks like you can maybe use the cmdclass option in setuptools.setup, but I couldn't find any good documentation on it that wasn't specifically for compiling C extensions.

[akhmerov]

Is there a way to hook into setuptools/distutils etc, to run arbitrary pre-build code (to create the compiled CSS files)?

Yep! Check this out: https://gitlab.kwant-project.org/kwant/mumpy/-/blob/master/setup.py we render a jinja template to generate a cython file, which we cythonize to C.

[chrisjsewell]

Ah brilliant thanks @akhmerov, have you found any drawbacks/gotchas with this approach?

[akhmerov]

The setuptools part never gave any trouble. Obviously your build env needs to have all the tooling, but the rest is smooth sailing. e.g. pip only ever sees the built files

[pradyunsg]

@akhmerov made my entire long-form comment redundant (that I was writing) because I came up with the same approach as mumpy is already using. :)

[chrisjsewell]

perfect ok ta, I can give a go at setting this up for sphinx-panels. If that goes well can then try rolling out to sphinx-book-theme, which is a little trickier, because you also want to replace the CSS file name in theme.conf, etc

[pradyunsg]

Consider making it a separate package that we could depend on at build-time?

[chrisjsewell]

Yeh quite possibly. For simplicity, I'm initially just going to put all the code in sphinx-panels. Then we can look to extract the repo independent stuff into a separate package.

[pradyunsg]

One fun thing to note here -- we would want to check what is available in the ReadTheDocs build environments; I'm not sure they ship with npm and friends, and that'd be a -ve for this approach. :(