You mentioned that for the time being you don't want to send anything to Soda Cloud, not the samples and not the failed rows.
First question that came up for me is this: What is your initial target workflow for dealing with the alerts that come out of Soda Core? From the brief demo I understood that you have a container that runs a scan. Running the scans produces 3 levels of output information:
If the scan command line fails or not. This can be based on the return code of the soda scan ... CLI command.
Check results and diagnostic information. This can be extracted and digested using the Python API to run a scan.
Samples. Samples are table based diagnostics information like eg failed rows. Using the Python API, you can configure your own sampler storage implementation that stores the samples anywhere you want. Or you can just disable samples if that is what you want.
Given that you don't want to connect to Soda Cloud, do you have all the capabilities in Soda Core to build your alert notifications, analysis diagnostics and resolution workflows? I didn't get that from the conversation so would be great to learn if there is something missing for you.
Related, it was mentioned that later you might connect to Soda Cloud. But I didn't get the question as to what you wanted to verify in that case. Can you elaborate?
jmarien
commented
2 years ago
You mentioned that for the time being you don't want to send anything to Soda Cloud, not the samples and not the failed rows.
First question that came up for me is this: What is your initial target workflow for dealing with the alerts that come out of Soda Core? From the brief demo I understood that you have a container that runs a scan. Running the scans produces 3 levels of output information:
soda scan ...CLI command.Given that you don't want to connect to Soda Cloud, do you have all the capabilities in Soda Core to build your alert notifications, analysis diagnostics and resolution workflows? I didn't get that from the conversation so would be great to learn if there is something missing for you.
Related, it was mentioned that later you might connect to Soda Cloud. But I didn't get the question as to what you wanted to verify in that case. Can you elaborate?