Closed tura-david closed 3 years ago
@tura-david Can you add the install_jupyter.sh
script from ch1/pandas-intro over? It could help users to install jupyter if they don't already.
Could you please also add a requirements.txt
installing on the pip packages needed for this episode?
@tura-david Can you add the
install_jupyter.sh
script from ch1/pandas-intro over? It could help users to install jupyter if they don't already.
Good point! I'll get it added.
Could you please also add a
requirements.txt
installing on the pip packages needed for this episode?
Just double checked and the only import I'm using is pyspark
which I discuss installing in the first section of the intro. As such, I'm not going to make a separate requirements.txt
unless there is another library being used.
@tura-david I think the scope for the intro
notebook is perfect.
The scope for the RDDDataFrames
notebook could add things like:
Could you please also add a
requirements.txt
installing on the pip packages needed for this episode?Just double checked and the only import I'm using is
pyspark
which I discuss installing in the first section of the intro. As such, I'm not going to make a separaterequirements.txt
unless there is another library being used.
You already have jupyter and you probably going to install other ones added later (for bigquery stuff). It makes it easier mainly for us who switch between developing different lessons
Could you please also add a
requirements.txt
installing on the pip packages needed for this episode?Just double checked and the only import I'm using is
pyspark
which I discuss installing in the first section of the intro. As such, I'm not going to make a separaterequirements.txt
unless there is another library being used.You already have jupyter and you probably going to install other ones added later (for bigquery stuff). It makes it easier mainly for us who switch between developing different lessons
I already added the install_jupyter.sh
; how about I add a 'install pyspark' line like the install pandas already included? That way we have the necessary modules for easy switching like you say but follow the same model as the pandas intro?
Could you please also add a
requirements.txt
installing on the pip packages needed for this episode?Just double checked and the only import I'm using is
pyspark
which I discuss installing in the first section of the intro. As such, I'm not going to make a separaterequirements.txt
unless there is another library being used.You already have jupyter and you probably going to install other ones added later (for bigquery stuff). It makes it easier mainly for us who switch between developing different lessons
I already added the
install_jupyter.sh
; how about I add a 'install pyspark' line like the install pandas already included? That way we have the necessary modules for easy switching like you say but follow the same model as the pandas intro?
Sounds good.
Overall you do a really good job of what Spark is and how to use it, but I feel like the "why" is missing for the introduction to Spark. I feel like I can't really answer the question "why am I learning Spark?" Also just made some typo edits.
Good point; this will be a separate PR by @parvister
Wrote the first two Spark intro notebooks (took me long enough, eh?). @jendevelops can you give them a review to see if they make any sense?