Closed simkimsia closed 4 months ago
Is it sensible to apply your setup to a jupyter notebook that needs to become an interactive dashboard? Or it's more of a round peg to square hole?
I'd say it's sensible! I wouldn't necessarily recommend putting all of the code -- ingestion, ETL with Dagster, and postprocessing -- in the DAG, but once you have the data transformed and loaded into whatever database or table format you want it in, using a notebook for the dashboard/visualization piece makes a lot of sense. there are numerous open source notebook -> dashboard app frameworks, or tools like Hex, that make this pretty easy
you could also put the Dagster logic in a notebook, but personally I'd prefer to keep that in Python scripts. regardless, just importing whatever you need in your notebook and running it is fine. as mentioned, when I'm experimenting I'll open up a notebook, run from eda import *
that gives me the con
to the database with all the tables + sets up plotly, and go from there
Thanks for the response @lostmygithubaccount
I'm going to give it a try
You ok I update you my progress here?
I'll be closing the question
sure thing! feel free to ask questions here or in the Ibis project Zulip (chat)
sure thing! feel free to ask questions here or in the Ibis project Zulip (chat)
i joined. Let me know if ok i post my progress here and there
feel free to post in either or both!
Crossposting from https://ibis-project.zulipchat.com/#narrow/stream/405263-general/topic/use.20ibis-analytics.20to.20convert.20jupyter.20notebook
In the learning project, I'm doing with a finance analyst, i easily converted the jupyter notebook to be streamlit compatibel script.
i had to swap out the flask/dash related commands. Apparently, not compatible with streamlit but that's easeily done with the help of chat GPT 4. haha :happy:
First time using streamlit and i think it's awesome.
While waiting for feedback from Danny, the finance analyst, I will switch back to replicating Cody's blogpost.
Exciting times!
Here's the latest streamlit-ready python script from the learning project: NBA-Dashboard/NBA.py at main · simkimsia/NBA-Dashboard (github.com)
I saw this setup in the original blogpost.
There's a public jupyter notebook that I am helping the maintainer bring to life as an interactive dashboard. Met him via Chip Huyen's MLOps discord. Same discord where I learn the terms: ibis and dagster.
https://github.com/dgilmore33/NBA-Dashboard
I am thinking of applying your setup (not all of it, just the essential parts) to the jupyter notebook.
I did notice that you mentioned the use of notebook in EDA
in the post.
Is it sensible to apply your setup to a jupyter notebook that needs to become an interactive dashboard? Or it's more of a round peg to square hole?
ChatGPT 4 response
This is what I got when I posed the same question to ChatGPT 4 including the link to the post.
Let me know if there's any obvious misunderstanding from my question, or any obvious flaws from ChatGPT's response.