search

FRosner / drunken-data-quality

Spark package for checking data quality
Apache License 2.0
222 stars 69 forks source link

about reporter #147

Closed rovernerd closed 6 years ago

rovernerd commented 6 years ago

Hi FRosner, Our team is interested in your project,but we have a problem. In our condition,we need to save the result into mysql or other RDBMS.How to get the result ? Can you offer me some suggestions? Thank you~

FRosner commented 6 years ago

Hi @rovernerd!

Thank you for your question. If you'd like to discuss things like this, I can highly recommend the gitter channel of this project: https://gitter.im/FRosner/drunken-data-quality

To answer your question I need to know what you are trying to save into RDBMS? The constraint results I assume?

You can achieve this in two ways:

  1. Write a JDBC reporter. There is already an issue open (https://github.com/FRosner/drunken-data-quality/issues/67). To do this you have to implement the Reporter trait. There are some examples of this already like the ConsoleReporter or the EmailReporter. Feel free to ask in gitter if you have troubles to understand how the reporters work.
  2. You manually take the constraint results and extract them to your RDBMS. An example of how to use the constraint results for unit tests can be found in the README.