awslabs / gluonts

Probabilistic time series modeling in Python
https://ts.gluon.ai
Apache License 2.0
4.55k stars 747 forks source link

Docs: Reduce tutorial runtime. #2083

Open jaheba opened 2 years ago

jaheba commented 2 years ago

Running the tutorial notebooks is slow:

=== trainer_callbacks.md.input finished evaluation in 11.296359777450562 sec
=== hp_tuning_with_optuna.md.input finished evaluation in 420.5497136116028 sec
=== howto_pytorch_lightning.md.input finished evaluation in 88.51285362243652 sec
=== quick_start_tutorial.md.input finished evaluation in 17.239495754241943 sec
=== extended_tutorial.md.input finished evaluation in 171.61477756500244 sec
=== pandasdataframes.md.input finished evaluation in 174.57174253463745 sec

Ideally we reduce the runtime as much as possible for the workflows to run.

However, I can see that we would like to run them for longer when publishing the release docs.

jaheba commented 2 years ago

/cc @rsnirwan @lostella

lostella commented 2 years ago

We could have some environment variable control whether to run the notebooks in "development" or "release" mode, and use that to control stuff like number of epochs, number of trials, or anything. But doing it in such a way that doesn't "pollute" the output notebook is challenging: maybe some pre-processing of the markdown files? Something like

    [...]
    epochs = $1; 10$
    [...]

that gets substituted with epochs = 1 in development mode, and epochs = 10 in release mode.

Just an idea.

jaheba commented 2 years ago

Was thinking something similar. We could have some template running.

But maybe we are just more radical and only run the notebooks on push and not for pull_request.

lostella commented 2 years ago

But maybe we are just more radical and only run the notebooks on push and not for pull_request.

Sounds good to me, but then we better make sure we check the output status of the docs build every now and then. How about putting a badge in the README for it?

lostella commented 2 years ago

Docs build

jaheba commented 2 years ago

We would still run the full docs workflow on push, but not for the individual pull_requests, except when https://github.com/awslabs/gluon-ts/labels/pr%3Adocs-build-notebook is used.

jaheba commented 2 years ago

2082 doesn't run notebook on default for PRs.

We can still add another layer using Jinja2 to compile the tutorials as templates.