Using pytest as an interface for joli

[tonyfast]

It is interesting to think about joli as aggregating best practices or opinions for notebook formatting. The varied use cases will make it difficult for a one size fits all solution. The choices for notebook constraints should be configurable.

Notebooks have a lot of potential value as test objects when used with pytest extensions like nbval and importnb. joli as a pytest extension could work together with these other tools. joli could include them as default deps because they contain complementary tests.

I think I could contribute the pytest machinery necessary to turn joli into a pytest extensions. Then folks could configure their notebook constraints in the conftest.py or tox.ini. Ya think this could help?

As future work, joli may have value as nbconverters. It would be nice to blacken notebooks --inplace.

This issue is the result of a post on testing features in the notebook data with pytest.

[betatim]

TL;DR: I think that would be cool! Make it so!

Right now I don't have a set in stone idea of how you'd use joli. My original idea was to make it behave like pep8 or other linters. Basically output in a format that other linters use so we can the nbuild a notebook/lab extension that takes that output and puts red wiggles under dodgy things in your notebook as you edit it.

However using black makes it more like a tool you'd run as a pre-commit hook to reformat everything in your repo?

Maybe the only thing I am sure of: we should make the parts that do work in joli into a library and then build different "frontends" from those building blocks. This means we don't have to decide now what frontend is best and can add more later.

How does importnb fit into this story? Is it something joli should use to do its job or is the idea to encourage people to make notebooks that work better with importnb? I'm confused.com :)

[tonyfast]

TL;DR: I think that would be cool! Make it so!

Right now I don't have a set in stone idea of how you'd use joli. My original idea was to make it behave like pep8 or other linters. Basically output in a format that other linters use so we can the nbuild a notebook/lab extension that takes that output and puts red wiggles under dodgy things in your notebook as you edit it.

Yes! I think some of this could be implemented from either the JS side or the Python side. Right now a simple implementation would be to add some html output to the inspector.

However using black makes it more like a tool you'd run as a pre-commit hook to reformat everything in your repo?

Diffing is a problem. Some of the rules defined in joli will make more readable and reliable diffs for notebooks. Configuring some joli pre commit rules with be great.

Maybe the only thing I am sure of: we should make the parts that do work in joli into a library and then build different "frontends" from those building blocks. This means we don't have to decide now what frontend is best and can add more later.

Build all the front ends! It would be nice for users to have a functional library of programming interfaces into measuring and modifying their notebooks. If all of the linters/formatters/checkers are reusable functions then they could be made into magics, nbconvert, click, pytest, precommit apps.

How does importnb fit into this story? Is it something joli should use to do its job or is the idea to encourage people to make notebooks that work better with importnb? I'm confused.com :)

I think the goals of importnb, nbval, and joli all aim to make better notebooks. The common overlap in them would be that they are/could be pytest extensions. There is likely a place for importnb in writing tests for joli, but I think the value I am considering is how of these notebook pytest exists could complement each other at the same time. There is value in notebook authors including formally testing frameworks in their notebook development.