Tutorials webpage

[INF800]

Hi, there are multiple examples available in the repository (in notebooks or scripts) but they are not available online as a webpage in documentation.

Can I send some PRs so that these examples are explained thoroughly and displayed in separate webpage like famous "keras examples" page.

Thanks!

[andsteing]

Hi Asapanna

@avital FYI

Thanks for reaching out! We're always very interested in improving on our examples, so let's try to figure out together how to best do this.

Note that we already have a (very simple) page in our RTD: https://flax.readthedocs.io/en/latest/examples.html

When you refer to "Keras examples" I assume you mean this page: https://keras.io/examples/

It seems they have all their examples as Colab notebooks. We believe that this is great format for some easy examples to get started (like our Annotated MNIST), but for larger examples (e.g. our examples/wmt) we think it's actually better to have the code in multiple files that can be edited with text editors / IDEs - both because Colab does not scale very well with large cells, and because for serious ML you'll want to have this in separate modules, and refactoring a Colab into a module is an error-prone process.

Because we love Colabs, we already added some notebooks to examples that allow you to explore the code and data, and even modify it and store it on your Drive - all without any installation required. Check out the MNIST example Colab and the Imagenet example Colab.

Additional to the examples, we also have "How do I ...?" notebooks explaining how to manage parameters and state, ensemble on multiple devices etc.

But this is just me talking about what we already have, so I'd be curious to hear your ideas how to improve it!

Best, Andreas

[INF800]

Note that we already have a (very simple) page in our RTD: https://flax.readthedocs.io/en/latest/examples.html

Sure, yes. I saw the page. But can we have more examples? Especially targeted for beginners so that experienced as well and new researchers / practitioners can start training models with ease. The idea is to have least entry barrier as possible.

When you refer to "Keras examples" I assume you mean this page: https://keras.io/examples/

Yes !!

for larger examples (e.g. our examples/wmt) we think it's actually better to have the code in multiple files that can be edited with text editors / IDEs - both because Colab does not scale very well with large cells, and because for serious ML you'll want to have this in separate modules, and refactoring a Colab into a module is an error-prone process.

I think the examples should NOT be focused on covering as many "use-cases" as possible like keras-examples-page. And we do NOT need to implement larger examples (e.g. our examples/wmt) .

Instead, in my humble opinion, the falx examples page should be centered around these two ideas - (which would help both experienced and beginner researchers/practitioners to pick up flax with ease)

1. Need to implement small examples which cover the Flax API as much as possible. We cover it in breadth but not in depth.
2. Whenever a new feature is out in flax we build a simple model using that feature(s). This would help others to replicate novel ideas by looking at examples.

[andsteing]

Thanks for the additional explanations.

IIUC then you propose to mainly rework our examples page, leaving the full-fledged examples ("depth") as is, but adding more small notebooks to the examples ("breadth"), similar to our "How do I ...?" series, but shorter and more self-contained.

Let's maybe first sketch a short list of such examples in this issue, and then we can discuss which ones to implement and how (e.g. it will be important that the examples are automatically tested so we notice when something becomes stale).

[INF800]

Let's maybe first sketch a short list of such examples in this issue

You read my mind :) I am going through the already implemented examples. Once I go through all, will draft the list.

Please note: I am talking about examples in google colab only. So that we can also convert them to webpages on the go.

These are the categories that I am thinking to cover. Please let me know if there are any categories that we can add/remove.

• Modelling: Modelling (different types) architectures in flax for computer vison and NLP
• Training: how models in different ways. In TPU/GPU/CPU. Using different types of training loops or schedulers as in "how to section". (Adding hyperparameter tuning is relevant here?)
• New flax features: Examples based on new flax features or new research papers
• and ..?

[marcvanzee]

Hi @INF800, any updates on this issue? Note I recently added a HOWTOs tracking issue #2116 where you could also add ideas.

[marcvanzee]

Closing this then!