search

jupyter / nbformat

Reference implementation of the Jupyter Notebook format
http://nbformat.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
265 stars 152 forks source link

Fast validation #220

Open ghazpar opened 3 years ago

ghazpar commented 3 years ago

After trying the fast validation, and observing how much faster it is compared to the standard one, I was wondering what is the purpose of this option? Why is it not the default? Why would anyone prefer the slow validation? Are there any down sides to this fast validation? If so, shouldn't this be documented?

goanpeca commented 3 years ago

? Why is it not the default?

@ghazpar it was at the moment an experimental feature to try to address a problem in the less invasive way possible.

Why would anyone prefer the slow validation? Why would anyone prefer the slow validation?

Good question, I guess the better question is when it will be deprecated, or when will the default change?

Are there any down sides to this fast validation? If so, shouldn't this be documented?

Not really, although fast validation will only increase a part of the process not the whole process, so even if it was enabled by default, we will need to include an extra dependency.

Also there is https://github.com/ijl/orjson, which will eventually be a faster(fastest?) option so we might as well make that the default (eventually)

If so, shouldn't this be documented?

You raise a very good point, yes this should have probably been better documented.

bollwyvl commented 3 years ago

We just need to be very careful with required dependencies at this level of the stack. I dislike, for example, that we end up with pyrsistent from jsonschema (which doesn't distribute wheels, for whatever reason), but it is the canonical parser, and fastjsonschema doesn't provide an equivalent API (for whatever reason). Orjson+dog have even more complex dependencies, if only to build them for novel platforms.

Ideally, we'd be able to move much of the specialness here to jupyter_core, and have some consistent ways for parsing, writing, and validating JSON with best-effort performance. Better still, would be async flavors of all of these, as the perceptual speed (especially at startup) would be substantially improved.