Initial setup of the "new starter pack" that uses the extension

[ru-fu]

This PR is not to main, but to a new branch that we can keep in parallel to main for a while until we switch over.

The main changes are:

• Removed all design files that are now moved to the extension.
• Updated the conf.py file to contain what used to be in custom_conf.py.
• Split the content into sp-files (the files that are needed to set up new documentation) and sp-docs (the documentation for the starter pack).
• Updated the docs (only the framework, not the content) to use as many files as possible directly from sp-files.
• Added an action to check the starter pack docs.
• Did a quick update of the init script to work with the new file structure. My plan is to have the init script as the default way of setting up the starter pack in the future.

See the separate commits for more information. There is more cleanup needed - see the tasks in https://warthogs.atlassian.net/browse/DOCPR-407.

To test this PR, download init.sh and run it. This will then get the rest of the files from my branch (I'll update that commit before merging to point to the new branch).

[ru-fu]

I can't get the sync action to work here, but it works fine on another repo. My suspicion is that either it will start working after the merge (because the token permission is limited for PRs from a public fork), or we need to change an admin setting on this repo (which I can neither see nor update). Either way, the current behaviour is already doing what we want - the action will succeed for PRs that don't change the files in question (because it doesn't run) and fail for those that do.

[evilnick]

Haven't dug into the issue but on a 'clean' install I'm getting:

make run
make -f Makefile.sp sp-run
make[1]: Entering directory '/home/evilnick/Canonical/cert/cert-test'
... setting up virtualenv
python3 -m venv .sphinx/venv
. .sphinx/venv/bin/activate; pip install --require-virtualenv \
    --upgrade -r .sphinx/requirements.txt \
            --log .sphinx/venv/pip_install.log
Collecting canonical-sphinx
  Downloading canonical_sphinx-0.1.0-py3-none-any.whl (30 kB)
Collecting sphinxcontrib-jquery
  Using cached sphinxcontrib_jquery-4.1-py2.py3-none-any.whl (121 kB)
Collecting myst-parser
  Using cached myst_parser-3.0.1-py3-none-any.whl (83 kB)
Collecting sphinx-design
  Using cached sphinx_design-0.6.0-py3-none-any.whl (2.2 MB)
Collecting sphinx-tabs
  Using cached sphinx_tabs-3.4.5-py3-none-any.whl (9.9 kB)
Collecting canonical-sphinx-extensions
  Using cached canonical_sphinx_extensions-0.0.21-py3-none-any.whl (34 kB)
Collecting sphinx<8,>=7.1.2
  Using cached sphinx-7.3.7-py3-none-any.whl (3.3 MB)
Collecting sphinx-notfound-page
  Using cached sphinx_notfound_page-1.0.2-py3-none-any.whl (8.1 kB)
Collecting furo
  Using cached furo-2024.5.6-py3-none-any.whl (341 kB)
Collecting pyspelling
  Using cached pyspelling-2.10-py3-none-any.whl (45 kB)
Collecting sphinx-copybutton
  Using cached sphinx_copybutton-0.5.2-py3-none-any.whl (13 kB)
Collecting sphinx-reredirects
  Using cached sphinx_reredirects-0.1.3-py3-none-any.whl (5.3 kB)
Collecting linkify-it-py
  Using cached linkify_it_py-2.0.3-py3-none-any.whl (19 kB)
Collecting sphinxext-opengraph
  Using cached sphinxext_opengraph-0.9.1-py3-none-any.whl (1.0 MB)
Collecting snowballstemmer>=2.0
  Using cached snowballstemmer-2.2.0-py2.py3-none-any.whl (93 kB)
Collecting babel>=2.9
  Using cached Babel-2.15.0-py3-none-any.whl (9.6 MB)
Collecting sphinxcontrib-devhelp
  Using cached sphinxcontrib_devhelp-1.0.6-py3-none-any.whl (83 kB)
Collecting sphinxcontrib-htmlhelp>=2.0.0
  Using cached sphinxcontrib_htmlhelp-2.0.5-py3-none-any.whl (99 kB)
Collecting packaging>=21.0
  Using cached packaging-24.1-py3-none-any.whl (53 kB)
Collecting sphinxcontrib-serializinghtml>=1.1.9
  Using cached sphinxcontrib_serializinghtml-1.1.10-py3-none-any.whl (92 kB)
Collecting alabaster~=0.7.14
  Using cached alabaster-0.7.16-py3-none-any.whl (13 kB)
Collecting Jinja2>=3.0
  Using cached jinja2-3.1.4-py3-none-any.whl (133 kB)
Collecting requests>=2.25.0
  Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Collecting tomli>=2
  Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting sphinxcontrib-qthelp
  Using cached sphinxcontrib_qthelp-1.0.7-py3-none-any.whl (89 kB)
Collecting Pygments>=2.14
  Using cached pygments-2.18.0-py3-none-any.whl (1.2 MB)
Collecting sphinxcontrib-jsmath
  Using cached sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl (5.1 kB)
Collecting docutils<0.22,>=0.18.1
  Using cached docutils-0.21.2-py3-none-any.whl (587 kB)
Collecting sphinxcontrib-applehelp
  Using cached sphinxcontrib_applehelp-1.0.8-py3-none-any.whl (120 kB)
Collecting imagesize>=1.3
  Using cached imagesize-1.4.1-py2.py3-none-any.whl (8.8 kB)
Collecting beautifulsoup4
  Using cached beautifulsoup4-4.12.3-py3-none-any.whl (147 kB)
Collecting sphinx-basic-ng>=1.0.0.beta2
  Using cached sphinx_basic_ng-1.0.0b2-py3-none-any.whl (22 kB)
Collecting uc-micro-py
  Using cached uc_micro_py-1.0.3-py3-none-any.whl (6.2 kB)
Collecting mdit-py-plugins~=0.4
  Using cached mdit_py_plugins-0.4.1-py3-none-any.whl (54 kB)
Collecting markdown-it-py~=3.0
  Using cached markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
Collecting pyyaml
  Using cached PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)
Collecting lxml
  Using cached lxml-5.2.2-cp310-cp310-manylinux_2_28_x86_64.whl (5.0 MB)
Collecting markdown
  Using cached Markdown-3.6-py3-none-any.whl (105 kB)
Collecting soupsieve>=1.8
  Using cached soupsieve-2.5-py3-none-any.whl (36 kB)
Collecting html5lib
  Using cached html5lib-1.1-py2.py3-none-any.whl (112 kB)
Collecting wcmatch>=8.5
  Using cached wcmatch-8.5.2-py3-none-any.whl (39 kB)
Collecting MarkupSafe>=2.0
  Using cached MarkupSafe-2.1.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting mdurl~=0.1
  Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Collecting urllib3<3,>=1.21.1
  Downloading urllib3-2.2.2-py3-none-any.whl (121 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.4/121.4 KB 3.9 MB/s eta 0:00:00
Collecting idna<4,>=2.5
  Using cached idna-3.7-py3-none-any.whl (66 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2024.6.2-py3-none-any.whl (164 kB)
Collecting charset-normalizer<4,>=2
  Using cached charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
Collecting bracex>=2.1.1
  Using cached bracex-2.4-py3-none-any.whl (11 kB)
Collecting six>=1.9
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting webencodings
  Using cached webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Installing collected packages: webencodings, snowballstemmer, urllib3, uc-micro-py, tomli, sphinxcontrib-serializinghtml, sphinxcontrib-qthelp, sphinxcontrib-jsmath, sphinxcontrib-htmlhelp, sphinxcontrib-devhelp, sphinxcontrib-applehelp, soupsieve, six, pyyaml, Pygments, packaging, mdurl, MarkupSafe, markdown, lxml, imagesize, idna, docutils, charset-normalizer, certifi, bracex, babel, alabaster, wcmatch, requests, markdown-it-py, linkify-it-py, Jinja2, html5lib, beautifulsoup4, sphinx, pyspelling, mdit-py-plugins, sphinxext-opengraph, sphinxcontrib-jquery, sphinx-tabs, sphinx-reredirects, sphinx-notfound-page, sphinx-design, sphinx-copybutton, sphinx-basic-ng, myst-parser, canonical-sphinx-extensions, furo, canonical-sphinx
Successfully installed Jinja2-3.1.4 MarkupSafe-2.1.5 Pygments-2.18.0 alabaster-0.7.16 babel-2.15.0 beautifulsoup4-4.12.3 bracex-2.4 canonical-sphinx-0.1.0 canonical-sphinx-extensions-0.0.21 certifi-2024.6.2 charset-normalizer-3.3.2 docutils-0.21.2 furo-2024.5.6 html5lib-1.1 idna-3.7 imagesize-1.4.1 linkify-it-py-2.0.3 lxml-5.2.2 markdown-3.6 markdown-it-py-3.0.0 mdit-py-plugins-0.4.1 mdurl-0.1.2 myst-parser-3.0.1 packaging-24.1 pyspelling-2.10 pyyaml-6.0.1 requests-2.32.3 six-1.16.0 snowballstemmer-2.2.0 soupsieve-2.5 sphinx-7.3.7 sphinx-basic-ng-1.0.0b2 sphinx-copybutton-0.5.2 sphinx-design-0.6.0 sphinx-notfound-page-1.0.2 sphinx-reredirects-0.1.3 sphinx-tabs-3.4.5 sphinxcontrib-applehelp-1.0.8 sphinxcontrib-devhelp-1.0.6 sphinxcontrib-htmlhelp-2.0.5 sphinxcontrib-jquery-4.1 sphinxcontrib-jsmath-1.0.1 sphinxcontrib-qthelp-1.0.7 sphinxcontrib-serializinghtml-1.1.10 sphinxext-opengraph-0.9.1 tomli-2.0.1 uc-micro-py-1.0.3 urllib3-2.2.2 wcmatch-8.5.2 webencodings-0.5.1
. .sphinx/venv/bin/activate; sphinx-autobuild -b dirhtml "." "_build" -c . -d .sphinx/.doctrees
/bin/sh: 1: sphinx-autobuild: not found
make[1]: *** [Makefile.sp:55: sp-run] Error 127
make[1]: Leaving directory '/home/evilnick/Canonical/cert/cert-test'
make: *** [Makefile:27: run] Error 2

[evilnick]

I see autobuild is in the requirements so I have no idea why it failed

[ru-fu]

I see autobuild is in the requirements so I have no idea why it failed

This is very strange, yes. Could you check that in your Makefile.sp you have the changes from this PR (mainly line 57)?

Since it's calling . .sphinx/venv/bin/activate; sphinx-autobuild -b dirhtml "." "_build" -c . -d .sphinx/.doctrees and not ...$(VENVDIR)/bin/sphinx-autobuild.

(This is a change I added after I originally opened the PR, so maybe it's somehow getting lost ...)

[evilnick]

I recreated the whole thing again and now it works :shrug:

[rkratky]

Just realized: one annoying side-effect of the Makefile split is that we lost Make autocomplete (or you'd need to be using the sp- prefix on the command line).

[ru-fu]

Just realized: one annoying side-effect of the Makefile split is that we lost Make autocomplete (or you'd need to be using the sp- prefix on the command line).

Yes, I agree. We discussed that in the LXD team when updating the docs to the latest version of the starter pack and couldn't find a workaround. :(

This isn't related to the move to the extension though. Maybe open an issue to track it?

[ru-fu]

I'll merge this PR now - the new file structure will then be on the use-canonical-sphinx-extension branch.

You can still test is as before, just download init.sh from that branch and run it. If you find bugs, please file an issue here on GitHub. For bigger issues/changes, file a Jira issue under DOCPR-407. For questions or discussion, let's use Mattermost.