sfu-db / dataprep

Open-source low code data preparation library in python. Collect, clean and visualization your data in python with a few lines of code.
http://dataprep.ai
MIT License
2.06k stars 206 forks source link

eda: create_db_report #740

Open jinglinpeng opened 3 years ago

jinglinpeng commented 3 years ago

Summary

Implement a function 'create_db_report' to elaborate details of a database schema through interactive visualizations of table statistics and schema diagrams.. Ref: https://schemaspy.org/sample/index.html

Design-level Explanation Actions

1- Integrating schemaspy to leverage the front-end features already implemented 2- Connecting Python backend to Java frontend using JSON objects for data transfer 3- Implementing object oriented design in Java to store the database structure

Design-level Explanation

def create_db_report( create_engine)

The API design for this function is simple where the user is required to pass an sqlalchemy create_engine connector object. The create_engine connector supports connections to the following databases (documentation hyperlinks included):

PostgreSQL MySQL and MariaDB SQLite Oracle Microsoft SQL Server

Current version of create_db_report support the first three dialects.

Implementation-level Explanation

Rational and Alternatives

Prior Art

Future Possibilities

Implementation-level Actions

1- Debugging currently implemented front end pages 2- Implement support for orphan tables and anomaly detection 3- Refactor backend code and add support for remaining dialects 4- Reform correctness testing and performance testing

Additional Tasks