Closed mmcky closed 4 years ago
Summary of Entry Points to sphinxcontrib-jupyter
:
make ipynb
: generates notebook files only (ignoring all extended options)make jupyter
: respects the conf.py
and is a fully configurable option with all support of execution
, website
generation (download notebooks sets and conversion to html), and coverage.We could also add the following for running those stages of the pipeline (and any dependent ones before them) but I don't think they would be used a great deal except perhaps make jupyter-coverage
for generating coverage reports as an independent process.
make jupyter-execute
make jupyter-html
make jupyter-coverage
I think this is a great idea (and we had some initial .sh
files to do this on the Julia side). Few thoughts:
I think it would be great to have an execution option in between make ipynb
(execute nothing) and make jupyter
(execute everything.) Maybe attaching flags like -n name1 name2 -d dir1 dir2
, where name
s are lecture names, and dir
s are whole directories (e.g., tools_and_techniques/
, or dynamic_programming_squared/
.)
While we're at it, a make rst-check
or something which calls out to an RST linter would be very useful. I could do some playing around, but the important thing would be to make sure that links, tables, math, and code blocks are well-formatted. And enforce QuantEcon styling, like no periods at the ends of sentences, sentences followed by line-breaks, etc.
This was a big pain point while we were doing the Julia rewriting.
make rst-check
, depending on options in the conf.py
. For example, regarding spaces in function calls, etc. More of a long-term goal, though. Perhaps something like this already exists in the Python side? With the Julia, especially since we had a relatively large and rotating set of RAs, this would've also been a huge help.
@mmcky and I discussed this yesterday, but I have a different perspective to you guys. I'd like to keep the interface very simple and clear, so that it's easy for new users to understand.
For example, make ipynb
and make jupyter
might lead to confusion as to what their roles are. We should follow the philosophy that basic operation is clear to an intelligent user who meets the system for the first time and isn't trying to do something fancy, without having to read the manual.
How about, for the moment, we stick to
make jupyter
-- convert rsts to ipynbs but don't executemake website
-- build the whole websitemake pdfs
-- build the pdfs (this is in work on the Python side and we'll try to extend to all adopters)make all
-- build everythingThen we can look at adding make rst-check
later if we do add such functionality.
If we need more fine grained control, then we can add flags, as @arnavs suggested. But only when there's a clear use case and no other easy way to do it.
Thanks @jstac, what you wrote makes sense to me.
Would we then have a make execute
to do the ipynbs with the execution step (but no HTML, PDFs, etc.)?
Good question @arnavs
I guess my idea is to keep the number of make
commands small and for the use of each one in standard cases to be immediately clear from the name. Anything else is done by adding flags to the basic commands --- such as the four I listed.
The make website
command would, I suppose, regenerate ipynb files from those rst files that have changed, execute them and build html. We could add a -- refresh
flag or something similar to override caching.
I'm thinking from the perspective of an average user who writes content, which is what I am. So I want a simple interface. But I don't know everything that you guys need to do. So I suggest we start with those four basic make
commands I listed above and then you give me a scenario where they would be insufficient. Then we can have a debate as to whether the extra functionality is really worth cluttering the user interface. For example, do we need a make execute
when we can just make website
, which will also execute. Yes, it does a few other things, but they are almost instantaneous, so who cares? That's my thinking, but I might be wrong, so feel free to tell me. But let's start with a minimal and uncluttered user interface, and then build incrementally if we're all convinced it's necessary.
A small issue: due to an unfortunate bifurcation and my own mistakes, in the sandpit right now make notebooks
is a synonym for make jupyter
. Could we please preserve this, even thought it shouldn't have happened, so as not to break Tom's workflow?
Roger that @jstac, thanks for clarifying. This makes sense.
(The main reason we were interested in make execute
is because of difficulties assembling the site, e.g. with HTML templates, but it's likely these will evaporate as we iron out the build process.)
I understand, and I don't want to get in the way of you power users :-)
I just have to take the perspective of the average Joe, not the developers, when I think about the interface. So it's great to have this conversation, where we try to balance out competing needs.
I forgot that we need make preview
and make setup
, so the six basic ones I can think of now are
make setup
make jupyter
/ make notebooks
make preview
make website
make pdfs
make all
These can then be specialized with flags. How does that sound to everyone?
thanks @jstac for summarizing the user interface (through the Makefile).
with regards to --refresh
this is really catered for through make clean
which is a standard approach to a new build in sphinx
Right, we should have make clean
too. Thanks @mmcky, good call.
This is implemented at the source
repository level rather than here in the extension.
Updates to jupinx-quickstart
:
website
, preview
, notebooks
to the quickstart Makefilepdf
only once v0.5
is released I think this issue can now be closed with the release and management of jupinx
. Interfaces between jupinx
and Makefile
will need to be maintained (as per this issue) until jupinx
interfaces directly with sphinx-build
.
Currently
make jupyter
will execute everything in theconf.py
file which may includeexecution control
, andhtml generation
. But it would be nice to have the ability to just generate notebooks for editing and writing that doesn't have all the additional steps if the extension is used to generate websites.So I think we need to introduce:
make ipynb
ormake notebook
that specialises on generating a local collection of notebooks. So we will need to think about which options are parsed and which are ignored and make this VERY clear in the documentation. I think this is useful for editing and will be used heavily.make jupyter
will then respect theconf.py
file as it does currently in line with typical sphinx behaviour.I am reluctant to use
make jupyter-notebook
as it is long to type and it would be easy to just usemake jupyter
@AakashGfude when you're free -- lets catchup sometime to discuss this.