Research useful tools for data exploration.

[bryanpaget]

While on vacation I read about some interesting tools for data analysis. I'd like to write up a proposal for these tools and we can discuss if we want to add them to the AAW. If we don't want to add/support these tools, I can still write something up on how to install and use these tools on the AAW.

For instance, we don't even need to install anything but our documentation can include something along the lines of:

Install PyGWalker

mamba install -c conda-forge pygwalker ydata-profiling ipywidgets

Run PyGWalker

import pandas as pd
import pygwalker as pyg
from ydata_profiling.utils.cache import cache_file

file_name = cache_file(
    "pokemon.csv",
    "https://raw.githubusercontent.com/bryanpaget/html/main/pokemon.csv"
)

pokemon_df = pd.read_csv(file_name)
pyg.walk(pokemon_df)

And that is all that is required to get PyGWalker running on the AAW.

Tools

• [ ] 1. https://github.com/gristlabs/grist-core
• [ ] 2. https://playground.tensorflow.org/
• [x] 3. https://docs.profiling.ydata.ai/
• [x] 4. https://www.trymito.io/
• [ ] 5. https://github.com/nicolaskruchten/pivottable
• [x] 6. https://github.com/koaning/drawdata
• [x] 7. https://github.com/Kanaries/pygwalker
• [x] 8. https://visualpython.ai/
• [ ] 9. https://www.datascienceprophet.com/autoviz-auto-visualization-using-python/
• [x] 10. https://github.com/man-group/dtale
• [x] 11. https://pypi.org/project/sweetviz/

PyGWalker

More information on PyGWalker.

• https://medium.com/@najib_n/in-the-modern-landscape-snowflake-has-emerged-as-an-increasingly-pivotal-tool-within-organizations-c4dcaa5c736b
• https://www.youtube.com/watch?app=desktop&v=L4AO_NPCxcY
• https://tableaupoor.streamlit.app/

[bryanpaget]

I'm working on a Gist here: https://gist.github.com/bryanpaget/4a4503431ada221c4a12407681f13efe

• ydata-profiling is amazing. It gives the user a nice overview of the data and the dataset comparison feature is something I wish I had years ago when I did data science.
• PyGWalker seems to be pretty good at making charts on the fly inside of JupyterLab without needing to install any extra packages outside of what we already have access to on the AAW.
• Grist might require some extra work to get up and running but I'm wondering if Grist can be launched using Kubeflow or ArgoWorkflows.

[bryanpaget]

The work for this task can be found here:

• https://github.com/bryanpaget/jupyter

PR:

• https://github.com/StatCan/aaw/pull/1903

I worked on 7 of the 11 tools.

• Grist does not run from a jupyter-notebook, will require more work to make a demo. Possibly a kubeflow deployment.
• The Tensorflow Playground could be added as a link in the documentation on machine learning models as it's just an interactive toy example of a neural net.
• PivotTable is JS which is outside of the scope of the AAW for the time-being.
• AutoViz seems to be broken.