DataRecce / recce

The dbt data-validation toolkit for teams that care about building better data
https://datarecce.io
Apache License 2.0
252 stars 6 forks source link

Add option to include in summary detail information about checks #472

Open oleg-savko opened 4 days ago

oleg-savko commented 4 days ago

Add option to include in summary detail information about checks.

For CICD it will be very usefull to add option to provide detail information for checks in summary, and not only mark Checks of Data Mismatch Detected

For example, for Row count diff, add option to add table with rows count in base / current env, can give significantly impact for review out of box.


Any way, can you suggest is it way to automate and add this type of row count diff table in cicd (extract from anywhere, or maybe exists api) ?

DaveFlynn commented 3 days ago

Hi Oleg, The main issue with supporting more detailed summaries is the size limit for GitHub comments. We found that is was very easy to hit this limit, especially on PRs with many models.

The compromise was to list the models with checks that had mis-matched data, because that would provide the information necessary to continue investigation into the impact via the Recce UI.

If you attach the Recce State to the PR, you could then download this file and run Recce in Review Mode to view the results in Recce.

Recce Cloud removes the friction of having to handle the state file by enabling you to open Recce in your compute resources, like GitHub Codespaces.

We'd still like to discuss your use-case and see if we can provide a better summary for you. Would you be able to drop us an email at product@datarecce.io ?

Thanks,

Dave