Data scientists are expected to orient business users to the datasets by analyzing in a way that is accessible and easy to understand. This process is the first step to building trust in the analysis.
PyDataPeek is a package that enables data scientists to efficiently generate a visual summary of a dataset. This package includes functions that show the size of the dataset, a visual summary of missing data, a sample of the dataset showing the data types as well as exploratory visualizations for quantitative and qualitative data.
This package is also useful for business users who have to interact with data and want to begin exploring the data without using too much code or having to open a potentially large dataset on Excel.
All functions take in csv or Excel files as inputs to generate user-friendly summaries of the ingested dataset.
Several Python packages are available that support exploratory data analysis but none are specific to the targeted use cases here - a simple and technologically friendly way of summarizing data.
pandas
functionality to manipulate dataframes. Our package functionality overlaps with some functions such as pd.describe
which computes summary statistics for dataframes. The package differs in that it aims to offer summary statistics dependent on data type, including long form text data. This project is under development! Upon it's first release, it can be installed with the following instructions.
pip install pydatapeek --extra-index-url=https://test.pypi.org/simple/
Note that selenium
and chromedriver
are needed to save Altair charts. More information on installation can be found here.
We detail an example usage of our function below. The sample data file that we have used below can be found here. The following example assumes that the file has been downloaded as example.csv
in the root of your working directory.
from PyDataPeek import PyDataPeek as pdp
Input
pdp.missing_data_overview('example.csv')
Output
A .png
file in your working directory named 0_heatmap.png
:
Input
pdp.sample_data('example.csv`)
Output | sample_record | data_type | summary | |
---|---|---|---|---|
Unnamed: 0 | 1 | int64 | unique values: 4 | |
A | 1.0 | float64 | median value: 1.0 | |
B | 2013-01-02 | object | average length of string: 10.0 | |
C | 0.5049417 | float64 | median value: 0.34510475 | |
D | 3 | int64 | unique values: 1 | |
E | train | object | average length of string: 4.2 | |
F | foo | object | average length of string: 3.0 | |
movies | "Young couple on the road, minding their own business ..." | object | average length of string: 1336.2 |
...
Input
pdp.explore_w_histograms('example.csv', columns_list=['C'])
Output
A .png
file in your working directory named C_chart.png
:
Input
pdp.word_bubble('example.csv', column="movies")
Output
A .png
file in your working directory named 0_wordcloud.png
:
The official documentation is hosted on Read the Docs: https://PyDataPeek.readthedocs.io/en/latest/
This package was created with Cookiecutter and the UBC-MDS/cookiecutter-ubc-mds project template, modified from the pyOpenSci/cookiecutter-pyopensci project template and the audreyr/cookiecutter-pypackage.