Missinng Jupyter Book version compatibility information

[mathieuboudreau]

Jupyter Book has a (very annoying) habit of completely rehauling their project frequently, and so I think it'd be essential to know in the docs 1 - for what version of Jupyter Book these instructions were written for, and 2- what versions do we support on NeuroLibre.

For example, I'm trying to test our T1 book example which is a couple of years old in the submission process, but there appears to be a lot of files missing (eg toc) and differences in folder structres, and it's not clear to me if there's an easy workaround for this or if I'll have to restart the project from scratch using the latest Jupyter Book (which is missing some discontinued features that we were using in the original Jupyter Book that I compiled our T1 Book with)

[pbellec]

Re release cycle of JB: this is a very young project. My understanding is that sphinx+myst is meant as the long term stack.

Re version, totally agree. I will let @emdupre confirm, but I think we will only support the most recent iteration. I don't know what version that is, but need to be added.

Re adapting an older book. It is likely simpler to start from a fresh template, and add the notebooks in there. Re discontinued feature, would be interesting to know what's missing.

[emdupre]

There's only been one overhaul of the JB infrastructure, which moved from Jekyll as a static site generator to Sphinx. This was enabled by the introduction of MyST, but I wouldn't expect to see such a large, breaking change again any time soon -- the move was discussed for a long time before it was made, and it was clear almost from the beginning that Jekyll wasn't a sustainable long-term solution. Sphinx is proving a much better home for a lot of this work.

The current workflow assumes that you're working with the latest version of Jupyter Book, where latest is anything after 0.7.0 (aka the re-write in Sphinx). I'd agree with @pbellec that I would likely use the notebooks to generate a new book entirely, rather than trying to adapt an older book.

Will be curious re: what discontinued features you have in mind.

[mathieuboudreau]

Re release cycle of JB: this is a very young project. My understanding is that sphinx+myst is meant as the long term stack.

Yup absolutely, it just makes it a pain to create documentation for us to use; also if some users start using JN at the start of their project, it may have changed dramatically when they are ready to submit to NeuroLibre (this has happened to Notebook Factory in the middle of a push), so we should think about how to handle that.

Re version, totally agree. I will let @emdupre confirm, but I think we will only support the most recent iteration. I don't know what version that is, but need to be added.

Ok thanks!

Re adapting an older book. It is likely simpler to start from a fresh template, and add the notebooks in there. Re discontinued feature, would be interesting to know what's missing.

I'll see what's missing when I re-do the T1 book, and ask Kiril from the neurolibre team as I know he had to write some custom files to reimplement some lost features last winter.

[mathieuboudreau]

There's only been one overhaul of the JB infrastructure, which moved from Jekyll as a static site generator to Sphinx. This was enabled by the introduction of MyST, but I wouldn't expect to see such a large, breaking change again any time soon -- the move was discussed for a long time before it was made, and it was clear almost from the beginning that Jekyll wasn't a sustainable long-term solution. Sphinx is proving a much better home for a lot of this work.

I think for the major back-end, that's is in fact the case that it was only once. But the development cycle still doesn't ensure a consistency in files/API, as we've had to do some migration more than once between even minor versions. I gave up a while ago updating the JN versions as they came out as I found it took too much time last year, but if this current release is expected to have much better long-term stability then I'll reconsider.

The current workflow assumes that you're working with the latest version of Jupyter Book, where latest is anything after 0.7.0 (aka the re-write in Sphinx). I'd agree with @pbellec that I would likely use the notebooks to generate a new book entirely, rather than trying to adapt an older book.

Ok will do!

Will be curious re: what discontinued features you have in mind.

I'm going to compare my new book with the old current version and report back; I remember there was something important for our team that was lost during one of the releases. Kiril also encountered a few ones, but I think he was able to manually re-introduce them by copying some files/code.

Thank you both!

[ltetrel]

Just to add some information (even if I am not sure it is a concern here), the backend is currently agnostic to jupyter book version. It will just use the user environment which has the correct jupyter book installed (old or latest). I don't know for the front-end. but for the doc we could reduce the jupyter book file hierarchy section to the bare minimum, and instead provide some example repo (with different jupyter-book version) ?

[mathieuboudreau]

Just to add some information (even if I am not sure it is a concern here), the backend is currently agnostic to jupyter book version. It will just use the user environment which has the correct jupyter book installed (old or latest).

Oh - if that's the case, would it work with this one: https://github.com/qMRLab/t1_book? I'll try. If so, I agree maybe the hierarchy should be updated.

[mathieuboudreau]

I just tried and it failed with my repo compiled with the old Jupyter Book.

[ltetrel]

Oh - if that's the case, would it work with this one: https://github.com/qMRLab/t1_book? I'll try. If so, I agree maybe the hierarchy should be updated.

Although it can change in the future, if I am able to trigger the jupyter book build in an init-container, then the jupyter version would be fixed by this container (and not the user env anymore)

[ltetrel]

I just tried and it failed with my repo compiled with the old Jupyter Book.

Hum does the front-end make some integrity check before ? I will check the backend

[agahkarakuzu]

@mathieuboudreau T1 book is not following the folder organization our build expects. It should have printed an informative err message though, I'll check that.

[emdupre]

There is an integrity check, else serving the files will differ by whether the SSG was Jekyll or Sphinx. We only handle Sphinx-built books.

But the development cycle still doesn't ensure a consistency in files/API, as we've had to do some migration more than once between even minor versions. I gave up a while ago updating the JN versions as they came out as I found it took too much time last year, but if this current release is expected to have much better long-term stability then I'll reconsider.

Before JB hits 1.0 I'd expect some turnover, but you shouldn't see extensive migration between minor versions. I have no stake in what version you adopt, it should just be after 0.7 :)

[ltetrel]

Indeed, I checked and there is no notification neither on the k8s cluster or api. The jupyter book build is triggered only if there is a _config.yml in the content folder. Also here is how the book build is triggered:

jupyter-book build --path-output book_dst_path_docker content

I don't know how jupyter book was handling the build cmd, and plan to, so let me know if you think there could be an issue with this.

[agahkarakuzu]

@ltetrel request is not sent unless the repo meets basics listed in the docs. @mathieuboudreau please lmk if it was not obvious in the docs.

[ltetrel]

https://github.com/neurolibre/docs.neurolibre.org/commit/3223ff4e1b32aa8e2456cb3d5c451d8bc281adcf