search

agile-lab-dev / DataQuality

DataQuality for BigData
GNU Lesser General Public License v3.0
138 stars 50 forks source link

Are you interested in integrating DataQuality with DataSphere Studio? #6

Open wushengyeyouya opened 4 years ago

wushengyeyouya commented 4 years ago

DataQuality is a very great project in the field of data quality, and I think a good way to enhance the influence of our two projects is to integrate DataQuality with DataSphere Studio. What is DataSphere Studio? DataSphere Studio is a one-stop data application development and management portal open-sourced by WeBank. It meets the requirements of the entire process of data application development from data exchange, desensitization and cleaning, analysis and mining, quality inspection, visual display, regular scheduling to data output. Github address: https://github.com/WeBankFinTech/DataSphereStudio Are you interested？

agile-lab commented 4 years ago

Hi, yes it could be interesting for us. In which way do you want to integrate DataQuality project ?

wushengyeyouya commented 4 years ago

DSS uses a pluggable integrated framework design to allow us to integrate new functional components simply and quickly.

image

As we can see, AppJoint is the core concept that DSS can simply and quickly integrate various upper-layer systems.

If we implement a DataQuality AppJoint, DataQuality can be integrated with DataSphere Studio, somewhat similar like the Spark Interceptor for Zeppelin Interceptor.

For more information, please see: https://github.com/WeBankFinTech/DataSphereStudio/blob/master/docs/en_US/ch4/The%20Guide%20for%20Third-party%20Systems%20accessing%20DSS.md

emakhov commented 4 years ago

Hi @wushengyeyouya. We've reviewed DataSphere Studio and very excited about integrating Data Quality in there. Please, contact us by email (egor.makhov@agilelab.it, paolo.platter@agilelab.it), so we can discuss this development in detail.