search

brimdata / zed

A novel data lake based on super-structured data
https://zed.brimdata.io/
BSD 3-Clause "New" or "Revised" License
1.38k stars 67 forks source link

Abstract the warning channel and create a package for it #2600

Closed philrz closed 2 years ago

philrz commented 3 years ago

@mccanne's proposal at today's group discussion:

To help understand what's going on in a flowgraph, we should abstract the warning channel that's in proc context and create a package for it. It doesn't have to be complicated initially, but over time it will do more powerful things, such as make it easier to enable different types of warnings. Think of it as being like a logger, but for the warning channel.

As part of the exercise, we should document the model and settle on the nomenclature for the different classes of error.

Likely an MVP1 target.

philrz commented 3 years ago

@mccanne will be doing the first part of this in his hazmat PR and will break it into two tasks if there's unfinished work after the PR, or close it if it's all done.

mccanne commented 2 years ago

Async warnings don't work very well. This mechanism should be replaced by in-line errors throughout the analytics core and compile-time analysis combined with meta-data introspection can catch most of the typos, e.g., your query returns nothing because there is no field fu in the lake when you meant to type foo.

mccanne commented 2 years ago

With new model of explicit missing and embedded errors, warnings are no longer needed. We will make the UX never when the compile-time analysis can raise problems immediately when it consults the meta-data for optimizations.