Modularization: a serious rewrite

[stared]

The goal is version 0.5.0:

• extendable input plugins (kind of right now)
• extendable output plugins
• type hints (Python 3.5+)
• tests

Some things won't be backward compatible.

A project with @bartolo1024. So far this branch does not work - work in progress.

To do:

• [x] write MainLogger
• [x] write PlotLoss
• [x] adapt matplotlib plugin (a big thing...)
• [x] adapt extrema printing
• [x] adapt input callbacks
• [x] adapt TensorBoard plugin
• [x] adapt Neptune plugin
• [x] write tests
• [x] write GitHub actions
• [x] update Jupyter examples (and add Colab links)

[stared]

I made some progress on reimplementing metrics plots.

from livelossplot.plot_losses import PlotLosses
plotlosses = PlotLosses()
plotlosses.logger.groups = {'acccuracy': ['acc', 'val_acc'], 'log-loss': ['loss', 'val_loss']}
plotlosses.update({
    'acc': 0.5,
    'val_acc': 0.4,
    'loss': 1.2,
    'val_loss': 1.1
})
plotlosses.update({
    'acc': 0.55,
    'val_acc': 0.45,
    'loss': 1.1,
    'val_loss': 1.0
})
plotlosses.update({
    'acc': 0.65,
    'val_acc': 0.55,
    'loss': 1.0,
    'val_loss': 0.9
})

Then plotlosses.logger.grouped_log_history() gives

{'acccuracy': {'acc': [LogItem(step=0, value=0.5),
   LogItem(step=1, value=0.55),
   LogItem(step=2, value=0.65)],
  'val_acc': [LogItem(step=0, value=0.4),
   LogItem(step=1, value=0.45),
   LogItem(step=2, value=0.55)]},
 'log-loss': {'loss': [LogItem(step=0, value=1.2),
   LogItem(step=1, value=1.1),
   LogItem(step=2, value=1.0)],
  'val_loss': [LogItem(step=0, value=1.1),
   LogItem(step=1, value=1.0),
   LogItem(step=2, value=0.9)]}}

And plotlosses.send() (in Jupyter Notebook) draws:

Now @Bartolo1024 your turn. :)

[stared]

Instead of NamedTuple we may want to use TypedDict due to easier dict/JSON compatibility.

• https://www.python.org/dev/peps/pep-0589/
• https://stackoverflow.com/questions/53409117/what-are-the-main-differences-of-namedtuple-and-typeddict-in-python-mypy
• https://www.reddit.com/r/Python/comments/ffdl85/using_type_hints_for_an_enduser_package/

However, let's double-check as it may be only Python 3.8+ (not acceptable as of now).

[Bartolo1024]

Instead of NamedTuple we may want to use TypedDict due to easier dict/JSON compatibility.

• https://www.python.org/dev/peps/pep-0589/
• https://stackoverflow.com/questions/53409117/what-are-the-main-differences-of-namedtuple-and-typeddict-in-python-mypy
• https://www.reddit.com/r/Python/comments/ffdl85/using_type_hints_for_an_enduser_package/

However, let's double-check as it may be only Python 3.8+ (not acceptable as of now).

We can use this for python 3.5, but with mypy_extensions in dependencies (as in your links). Currently, it is not so good.

[stared]

@Bartolo1024 Thanks, rebase-merged here.

• I think the next step is maxima print.
• Then tweak inputs/outputs (they almost work, just one or two lines to set groups or so).
• Then making sure it works as a package & updating examples.

For workflow - it is nice that there are tests, but I think conda is not needed. At the same time, we may need Python package from https://github.com/stared/livelossplot/actions/new.

[stared]

These version matrices seem to work well, vide antipatch #88.

[stared]

It seems to be an issue not with our library, but with matplotlib (support 3.6+). https://github.com/matplotlib/matplotlib/issues/12358

I vote for dropping Python 3.5 support, as matplotlib is the main dependency. Bonus: we will get f-strings! :)

[stared]

• requirements.txt vs install_requires: https://packaging.python.org/discussions/install-requires-vs-requirements/ for further information
• matplotlib<3.1;python_version<"3.6" for https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-platform-specific-dependencies

[Bartolo1024]

What do you think about this old keras import: https://github.com/Bartolo1024/livelossplot/blob/serious-rewrite-old-keras-plugin-compatibility/livelossplot/__init__.py ? We can add custom module-finder to python import pipeline, detect old modules, print warning and import it dynamically. I have found good article about importlib if you want to read more about it: https://manikos.github.io/how-pythons-import-machinery-works

[stared]

@Bartolo1024 If it works, it is great.

However, this "deprecated" part (PlotLossesKeras, PlotLossesTensorFlowKeras) should be either un-deprecated (as a proper way to run things) or removed completely.

For supporting previous paths, I suggest adding all things supported in livelossplot 0.4.2.

---

For other parts, IMHO we should think hard about making a less Java-esque API.

Right now the code would work

from livelossplot import PlotLosses
from livelossplot.output_plugins.extrema_print import ExtremaPrint
from livelossplot.output_plugins.matplotlib import Matplotlib

Flat is better than nested.

I find Keras and PyTorch API nice, so it is, by all means, possible to use some nesting. Though, each time I write import matplotlib.pyplot as plt I die a little bit.

So, for simplicity, what do you think about:

• input_plugins -> inputs
• output_plugins -> outputs

Then it is simpler:

from livelossplot import PlotLosses
from livelossplot.outputs import Matplotlib, ExtremaPrint

plotlosses  = PlotLosses(outputs=[Matplotlib(), ExtremaPrint()])

OR

import livelossplot as llp

plotlosses = llp.PlotLosses(outputs=[llp.outputs.Matplotlib(), llp.outputs.ExtremaPrint()])

Let me know what you think about these API variants.

[stared]

Making it work for Python 3.5 was proven to be much more difficult than I have anticipated.

• The current Matplotlib (3.2) does not support that; we need to use Matplotlib 3.0
• There were some issues with Cython (required for NumPy 1.18, https://github.com/numpy/numpy/issues/15151)... so used numpy<1.18;python_version<"3.6"
• But then it turned out that install_requires work in the reverse order (!!!), https://github.com/pypa/setuptools/issues/498

Another stuff - I try to make a version for Python 3.5 (then Matplotlib 3.0), but I get some issues with installing Cython. Or Numpy - I don't know, I DID specify numpy<1.18;python_version<"3.6" and still it tries to install 1.18. Don't know how to solve it.

[Bartolo1024]

from livelossplot import PlotLosses from livelossplot.outputs import Matplotlib, ExtremaPrint

plotlosses = PlotLosses(outputs=[Matplotlib(), ExtremaPrint()])

I like this. Simplified "inputs" and "outputs" are better, than names with "_plugin" suffix.

[stared]

So, after some thinking I went for two syntaxes:

from livelossplot.inputs.keras import PlotLossesCallback
from livelossplot import PlotLossesKeras

I think that I will favor the second one, as:

• in general, it is PlotLosses replacement for a particular case
• it has the same arguments as PlotLosses
• maybe even I will make it inheriting from PlotLosses (so e.g. to be able to train a bit by Keras/Ignite/etc, and then access or modify everything exactly the same way as for raw PlotPlosses)

Now I focus on updating examples. How is your work going?

[Bartolo1024]

I have auto group generation and auto metric to name, but don't know if it is pure. https://github.com/stared/livelossplot/pull/92

loss
        Validation loss          (min:    0.105, max:    2.000, cur:    0.105)
        Training loss            (min:    0.091, max:    0.500, cur:    0.091)
acc
        Validation Accuracy      (min:    0.754, max:    1.000, cur:    0.975)
        Training Accuracy        (min:    0.719, max:    0.990, cur:    0.948)

[Bartolo1024]

After all can you:

• run the yapf code formatter and yapf . --verbose --recursive --in-place
• repace args of yapf --recursive -i . with . --recursive --diff
• and commit everything I would have not like to do this because it would be problematic for review and merge.