Adding Dask-SQL Weather notebook

[randerzander]

cc @taureandyernv

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:01Z ----------------------------------------------------------------

Is this meant only as an example, or is case sensitivity on by default?

randerzander commented on 2022-02-04T17:48:28Z ----------------------------------------------------------------

It's on by default. Will make that clearer.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:02Z ----------------------------------------------------------------

Could this be wrapped into an IPython extension and installed as part of Dask SQL so someone with dask_sql in their environment could do %load_ext dask_sql to enable it?

randerzander commented on 2022-02-04T17:56:10Z ----------------------------------------------------------------

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:03Z ----------------------------------------------------------------

Can I create a table from a filepath? This is the only place dask_cudf is used in the notebook. It might be nice to move fully off of dask_cudf from a user perspective

randerzander commented on 2022-02-04T17:57:01Z ----------------------------------------------------------------

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:04Z ----------------------------------------------------------------

This is a large row count. Might be nice to add a sentence highlighting this notebook is processing billions of records

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:05Z ----------------------------------------------------------------

Ah okay, I see you can load from a filepath with a flag parameter. What do you think of switching the ordering here? First (above), use the CREATE OR REPLACE TABLE and load from a file, giving you a chance to mention the gpu= parameter. Then, here, use dask_cudf and comment on how you can also create tables from dataframes.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:05Z ----------------------------------------------------------------

Line #9.        # globally defined var from previous cell

This is not yet defined in the previous cell. Looks like it's now dynamic in the SQL statement in the following cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:06Z ----------------------------------------------------------------

Line #43.        replace=True,

Might be valuable to add a sentence explaining this parameter (and perhaps the row_udf one)

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:07Z ----------------------------------------------------------------

Switched from IPython magic to using the Context just for this cell. If intentional, perhaps worth mentioning it? If I want to mix dataframe logic, is this the way I do it (vs using IPython magic)

randerzander commented on 2022-02-04T18:23:02Z ----------------------------------------------------------------

This is explained in the markdown cell above. Maybe it wasn't clear enough?

beckernick commented on 2022-02-04T20:47:13Z ----------------------------------------------------------------

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:08Z ----------------------------------------------------------------

Normalizing what? Heading makes me think this is something separate, rather than what you described above. Perhaps this could just be a regular text markdown cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:09Z ----------------------------------------------------------------

Line #3.    rain_df = _

This IPython specific behavior to get the prior result relies on running the cell in order, which we can't enforce. Can we make this explicit?

_randerzander commented on 2022-02-04T18:27:11Z_ ----------------------------------------------------------------

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #4.    px.line(

Does this not work on the GPU like Plotly Dash (I'm not well versed in Plotly)?

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #1.    px.line(_.to_pandas(), x="yr", y="stations", title="Station Count").show()

Same as above. Can't enforce users running in the same order. I see how this can be challenging with the %%sql magic we're not explicitly assigning to a variable, which makes it appealing to keep this as is.

If we keep as is, we could add a note that these rely on running in the correct order

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:11Z ----------------------------------------------------------------

Line #3.    CREATE OR REPLACE TABLE yearly_stations AS

When this command finishes, is a user able to interact with yearly_stations like a normal variable? E.g., is there some eval magic or something happening? Or is it only internal to the Context

_randerzander commented on 2022-02-04T18:31:05Z_ ----------------------------------------------------------------

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

_beckernick commented on 2022-02-04T20:48:37Z_ ----------------------------------------------------------------

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

randerzander commented 2 years ago

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:02Z ----------------------------------------------------------------

Could this be wrapped into an IPython extension and installed as part of Dask SQL so someone with dask_sql in their environment could do %load_ext dask_sql to enable it?

randerzander commented on 2022-02-04T17:56:10Z ----------------------------------------------------------------

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:03Z ----------------------------------------------------------------

Can I create a table from a filepath? This is the only place dask_cudf is used in the notebook. It might be nice to move fully off of dask_cudf from a user perspective

randerzander commented on 2022-02-04T17:57:01Z ----------------------------------------------------------------

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:04Z ----------------------------------------------------------------

This is a large row count. Might be nice to add a sentence highlighting this notebook is processing billions of records

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:05Z ----------------------------------------------------------------

Ah okay, I see you can load from a filepath with a flag parameter. What do you think of switching the ordering here? First (above), use the CREATE OR REPLACE TABLE and load from a file, giving you a chance to mention the gpu= parameter. Then, here, use dask_cudf and comment on how you can also create tables from dataframes.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:05Z ----------------------------------------------------------------

Line #9.        # globally defined var from previous cell

This is not yet defined in the previous cell. Looks like it's now dynamic in the SQL statement in the following cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:06Z ----------------------------------------------------------------

Line #43.        replace=True,

Might be valuable to add a sentence explaining this parameter (and perhaps the row_udf one)

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:07Z ----------------------------------------------------------------

Switched from IPython magic to using the Context just for this cell. If intentional, perhaps worth mentioning it? If I want to mix dataframe logic, is this the way I do it (vs using IPython magic)

randerzander commented on 2022-02-04T18:23:02Z ----------------------------------------------------------------

This is explained in the markdown cell above. Maybe it wasn't clear enough?

beckernick commented on 2022-02-04T20:47:13Z ----------------------------------------------------------------

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:08Z ----------------------------------------------------------------

Normalizing what? Heading makes me think this is something separate, rather than what you described above. Perhaps this could just be a regular text markdown cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:09Z ----------------------------------------------------------------

Line #3.    rain_df = _

This IPython specific behavior to get the prior result relies on running the cell in order, which we can't enforce. Can we make this explicit?

_randerzander commented on 2022-02-04T18:27:11Z_ ----------------------------------------------------------------

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #4.    px.line(

Does this not work on the GPU like Plotly Dash (I'm not well versed in Plotly)?

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #1.    px.line(_.to_pandas(), x="yr", y="stations", title="Station Count").show()

Same as above. Can't enforce users running in the same order. I see how this can be challenging with the %%sql magic we're not explicitly assigning to a variable, which makes it appealing to keep this as is.

If we keep as is, we could add a note that these rely on running in the correct order

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:11Z ----------------------------------------------------------------

Line #3.    CREATE OR REPLACE TABLE yearly_stations AS

When this command finishes, is a user able to interact with yearly_stations like a normal variable? E.g., is there some eval magic or something happening? Or is it only internal to the Context

_randerzander commented on 2022-02-04T18:31:05Z_ ----------------------------------------------------------------

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

_beckernick commented on 2022-02-04T20:48:37Z_ ----------------------------------------------------------------

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

randerzander commented 2 years ago

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:03Z ----------------------------------------------------------------

Can I create a table from a filepath? This is the only place dask_cudf is used in the notebook. It might be nice to move fully off of dask_cudf from a user perspective

randerzander commented on 2022-02-04T17:57:01Z ----------------------------------------------------------------

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:04Z ----------------------------------------------------------------

This is a large row count. Might be nice to add a sentence highlighting this notebook is processing billions of records

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:05Z ----------------------------------------------------------------

Ah okay, I see you can load from a filepath with a flag parameter. What do you think of switching the ordering here? First (above), use the CREATE OR REPLACE TABLE and load from a file, giving you a chance to mention the gpu= parameter. Then, here, use dask_cudf and comment on how you can also create tables from dataframes.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:05Z ----------------------------------------------------------------

Line #9.        # globally defined var from previous cell

This is not yet defined in the previous cell. Looks like it's now dynamic in the SQL statement in the following cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:06Z ----------------------------------------------------------------

Line #43.        replace=True,

Might be valuable to add a sentence explaining this parameter (and perhaps the row_udf one)

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:07Z ----------------------------------------------------------------

Switched from IPython magic to using the Context just for this cell. If intentional, perhaps worth mentioning it? If I want to mix dataframe logic, is this the way I do it (vs using IPython magic)

randerzander commented on 2022-02-04T18:23:02Z ----------------------------------------------------------------

This is explained in the markdown cell above. Maybe it wasn't clear enough?

beckernick commented on 2022-02-04T20:47:13Z ----------------------------------------------------------------

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:08Z ----------------------------------------------------------------

Normalizing what? Heading makes me think this is something separate, rather than what you described above. Perhaps this could just be a regular text markdown cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:09Z ----------------------------------------------------------------

Line #3.    rain_df = _

This IPython specific behavior to get the prior result relies on running the cell in order, which we can't enforce. Can we make this explicit?

_randerzander commented on 2022-02-04T18:27:11Z_ ----------------------------------------------------------------

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #4.    px.line(

Does this not work on the GPU like Plotly Dash (I'm not well versed in Plotly)?

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #1.    px.line(_.to_pandas(), x="yr", y="stations", title="Station Count").show()

Same as above. Can't enforce users running in the same order. I see how this can be challenging with the %%sql magic we're not explicitly assigning to a variable, which makes it appealing to keep this as is.

If we keep as is, we could add a note that these rely on running in the correct order

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:11Z ----------------------------------------------------------------

Line #3.    CREATE OR REPLACE TABLE yearly_stations AS

When this command finishes, is a user able to interact with yearly_stations like a normal variable? E.g., is there some eval magic or something happening? Or is it only internal to the Context

_randerzander commented on 2022-02-04T18:31:05Z_ ----------------------------------------------------------------

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

_beckernick commented on 2022-02-04T20:48:37Z_ ----------------------------------------------------------------

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

randerzander commented 2 years ago

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:04Z ----------------------------------------------------------------

This is a large row count. Might be nice to add a sentence highlighting this notebook is processing billions of records

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:05Z ----------------------------------------------------------------

Ah okay, I see you can load from a filepath with a flag parameter. What do you think of switching the ordering here? First (above), use the CREATE OR REPLACE TABLE and load from a file, giving you a chance to mention the gpu= parameter. Then, here, use dask_cudf and comment on how you can also create tables from dataframes.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:05Z ----------------------------------------------------------------

Line #9.        # globally defined var from previous cell

This is not yet defined in the previous cell. Looks like it's now dynamic in the SQL statement in the following cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:06Z ----------------------------------------------------------------

Line #43.        replace=True,

Might be valuable to add a sentence explaining this parameter (and perhaps the row_udf one)

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:07Z ----------------------------------------------------------------

Switched from IPython magic to using the Context just for this cell. If intentional, perhaps worth mentioning it? If I want to mix dataframe logic, is this the way I do it (vs using IPython magic)

randerzander commented on 2022-02-04T18:23:02Z ----------------------------------------------------------------

This is explained in the markdown cell above. Maybe it wasn't clear enough?

beckernick commented on 2022-02-04T20:47:13Z ----------------------------------------------------------------

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:08Z ----------------------------------------------------------------

Normalizing what? Heading makes me think this is something separate, rather than what you described above. Perhaps this could just be a regular text markdown cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:09Z ----------------------------------------------------------------

Line #3.    rain_df = _

This IPython specific behavior to get the prior result relies on running the cell in order, which we can't enforce. Can we make this explicit?

_randerzander commented on 2022-02-04T18:27:11Z_ ----------------------------------------------------------------

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #4.    px.line(

Does this not work on the GPU like Plotly Dash (I'm not well versed in Plotly)?

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #1.    px.line(_.to_pandas(), x="yr", y="stations", title="Station Count").show()

Same as above. Can't enforce users running in the same order. I see how this can be challenging with the %%sql magic we're not explicitly assigning to a variable, which makes it appealing to keep this as is.

If we keep as is, we could add a note that these rely on running in the correct order

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:11Z ----------------------------------------------------------------

Line #3.    CREATE OR REPLACE TABLE yearly_stations AS

When this command finishes, is a user able to interact with yearly_stations like a normal variable? E.g., is there some eval magic or something happening? Or is it only internal to the Context

_randerzander commented on 2022-02-04T18:31:05Z_ ----------------------------------------------------------------

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

_beckernick commented on 2022-02-04T20:48:37Z_ ----------------------------------------------------------------

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

randerzander commented 2 years ago

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:05Z ----------------------------------------------------------------

Ah okay, I see you can load from a filepath with a flag parameter. What do you think of switching the ordering here? First (above), use the CREATE OR REPLACE TABLE and load from a file, giving you a chance to mention the gpu= parameter. Then, here, use dask_cudf and comment on how you can also create tables from dataframes.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:05Z ----------------------------------------------------------------

Line #9.        # globally defined var from previous cell

This is not yet defined in the previous cell. Looks like it's now dynamic in the SQL statement in the following cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:06Z ----------------------------------------------------------------

Line #43.        replace=True,

Might be valuable to add a sentence explaining this parameter (and perhaps the row_udf one)

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:07Z ----------------------------------------------------------------

Switched from IPython magic to using the Context just for this cell. If intentional, perhaps worth mentioning it? If I want to mix dataframe logic, is this the way I do it (vs using IPython magic)

randerzander commented on 2022-02-04T18:23:02Z ----------------------------------------------------------------

This is explained in the markdown cell above. Maybe it wasn't clear enough?

beckernick commented on 2022-02-04T20:47:13Z ----------------------------------------------------------------

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:08Z ----------------------------------------------------------------

Normalizing what? Heading makes me think this is something separate, rather than what you described above. Perhaps this could just be a regular text markdown cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:09Z ----------------------------------------------------------------

Line #3.    rain_df = _

This IPython specific behavior to get the prior result relies on running the cell in order, which we can't enforce. Can we make this explicit?

_randerzander commented on 2022-02-04T18:27:11Z_ ----------------------------------------------------------------

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #4.    px.line(

Does this not work on the GPU like Plotly Dash (I'm not well versed in Plotly)?

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #1.    px.line(_.to_pandas(), x="yr", y="stations", title="Station Count").show()

Same as above. Can't enforce users running in the same order. I see how this can be challenging with the %%sql magic we're not explicitly assigning to a variable, which makes it appealing to keep this as is.

If we keep as is, we could add a note that these rely on running in the correct order

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:11Z ----------------------------------------------------------------

Line #3.    CREATE OR REPLACE TABLE yearly_stations AS

When this command finishes, is a user able to interact with yearly_stations like a normal variable? E.g., is there some eval magic or something happening? Or is it only internal to the Context

_randerzander commented on 2022-02-04T18:31:05Z_ ----------------------------------------------------------------

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

_beckernick commented on 2022-02-04T20:48:37Z_ ----------------------------------------------------------------

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

randerzander commented 2 years ago

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:05Z ----------------------------------------------------------------

Line #9.        # globally defined var from previous cell

This is not yet defined in the previous cell. Looks like it's now dynamic in the SQL statement in the following cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:06Z ----------------------------------------------------------------

Line #43.        replace=True,

Might be valuable to add a sentence explaining this parameter (and perhaps the row_udf one)

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:07Z ----------------------------------------------------------------

Switched from IPython magic to using the Context just for this cell. If intentional, perhaps worth mentioning it? If I want to mix dataframe logic, is this the way I do it (vs using IPython magic)

randerzander commented on 2022-02-04T18:23:02Z ----------------------------------------------------------------

This is explained in the markdown cell above. Maybe it wasn't clear enough?

beckernick commented on 2022-02-04T20:47:13Z ----------------------------------------------------------------

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:08Z ----------------------------------------------------------------

Normalizing what? Heading makes me think this is something separate, rather than what you described above. Perhaps this could just be a regular text markdown cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:09Z ----------------------------------------------------------------

Line #3.    rain_df = _

This IPython specific behavior to get the prior result relies on running the cell in order, which we can't enforce. Can we make this explicit?

_randerzander commented on 2022-02-04T18:27:11Z_ ----------------------------------------------------------------

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #4.    px.line(

Does this not work on the GPU like Plotly Dash (I'm not well versed in Plotly)?

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #1.    px.line(_.to_pandas(), x="yr", y="stations", title="Station Count").show()

Same as above. Can't enforce users running in the same order. I see how this can be challenging with the %%sql magic we're not explicitly assigning to a variable, which makes it appealing to keep this as is.

If we keep as is, we could add a note that these rely on running in the correct order

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:11Z ----------------------------------------------------------------

Line #3.    CREATE OR REPLACE TABLE yearly_stations AS

When this command finishes, is a user able to interact with yearly_stations like a normal variable? E.g., is there some eval magic or something happening? Or is it only internal to the Context

_randerzander commented on 2022-02-04T18:31:05Z_ ----------------------------------------------------------------

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

_beckernick commented on 2022-02-04T20:48:37Z_ ----------------------------------------------------------------

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

randerzander commented 2 years ago

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:06Z ----------------------------------------------------------------

Line #43.        replace=True,

Might be valuable to add a sentence explaining this parameter (and perhaps the row_udf one)

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:07Z ----------------------------------------------------------------

Switched from IPython magic to using the Context just for this cell. If intentional, perhaps worth mentioning it? If I want to mix dataframe logic, is this the way I do it (vs using IPython magic)

randerzander commented on 2022-02-04T18:23:02Z ----------------------------------------------------------------

This is explained in the markdown cell above. Maybe it wasn't clear enough?

beckernick commented on 2022-02-04T20:47:13Z ----------------------------------------------------------------

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:08Z ----------------------------------------------------------------

Normalizing what? Heading makes me think this is something separate, rather than what you described above. Perhaps this could just be a regular text markdown cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:09Z ----------------------------------------------------------------

Line #3.    rain_df = _

This IPython specific behavior to get the prior result relies on running the cell in order, which we can't enforce. Can we make this explicit?

_randerzander commented on 2022-02-04T18:27:11Z_ ----------------------------------------------------------------

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #4.    px.line(

Does this not work on the GPU like Plotly Dash (I'm not well versed in Plotly)?

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #1.    px.line(_.to_pandas(), x="yr", y="stations", title="Station Count").show()

Same as above. Can't enforce users running in the same order. I see how this can be challenging with the %%sql magic we're not explicitly assigning to a variable, which makes it appealing to keep this as is.

If we keep as is, we could add a note that these rely on running in the correct order

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:11Z ----------------------------------------------------------------

Line #3.    CREATE OR REPLACE TABLE yearly_stations AS

When this command finishes, is a user able to interact with yearly_stations like a normal variable? E.g., is there some eval magic or something happening? Or is it only internal to the Context

_randerzander commented on 2022-02-04T18:31:05Z_ ----------------------------------------------------------------

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

_beckernick commented on 2022-02-04T20:48:37Z_ ----------------------------------------------------------------

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

randerzander commented 2 years ago

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:07Z ----------------------------------------------------------------

Switched from IPython magic to using the Context just for this cell. If intentional, perhaps worth mentioning it? If I want to mix dataframe logic, is this the way I do it (vs using IPython magic)

randerzander commented on 2022-02-04T18:23:02Z ----------------------------------------------------------------

This is explained in the markdown cell above. Maybe it wasn't clear enough?

beckernick commented on 2022-02-04T20:47:13Z ----------------------------------------------------------------

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:08Z ----------------------------------------------------------------

Normalizing what? Heading makes me think this is something separate, rather than what you described above. Perhaps this could just be a regular text markdown cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:09Z ----------------------------------------------------------------

Line #3.    rain_df = _

This IPython specific behavior to get the prior result relies on running the cell in order, which we can't enforce. Can we make this explicit?

_randerzander commented on 2022-02-04T18:27:11Z_ ----------------------------------------------------------------

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #4.    px.line(

Does this not work on the GPU like Plotly Dash (I'm not well versed in Plotly)?

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #1.    px.line(_.to_pandas(), x="yr", y="stations", title="Station Count").show()

Same as above. Can't enforce users running in the same order. I see how this can be challenging with the %%sql magic we're not explicitly assigning to a variable, which makes it appealing to keep this as is.

If we keep as is, we could add a note that these rely on running in the correct order

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:11Z ----------------------------------------------------------------

Line #3.    CREATE OR REPLACE TABLE yearly_stations AS

When this command finishes, is a user able to interact with yearly_stations like a normal variable? E.g., is there some eval magic or something happening? Or is it only internal to the Context

_randerzander commented on 2022-02-04T18:31:05Z_ ----------------------------------------------------------------

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

_beckernick commented on 2022-02-04T20:48:37Z_ ----------------------------------------------------------------

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

randerzander commented 2 years ago

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:08Z ----------------------------------------------------------------

Normalizing what? Heading makes me think this is something separate, rather than what you described above. Perhaps this could just be a regular text markdown cell

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:09Z ----------------------------------------------------------------

Line #3.    rain_df = _

This IPython specific behavior to get the prior result relies on running the cell in order, which we can't enforce. Can we make this explicit?

_randerzander commented on 2022-02-04T18:27:11Z_ ----------------------------------------------------------------

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #4.    px.line(

Does this not work on the GPU like Plotly Dash (I'm not well versed in Plotly)?

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #1.    px.line(_.to_pandas(), x="yr", y="stations", title="Station Count").show()

Same as above. Can't enforce users running in the same order. I see how this can be challenging with the %%sql magic we're not explicitly assigning to a variable, which makes it appealing to keep this as is.

If we keep as is, we could add a note that these rely on running in the correct order

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:11Z ----------------------------------------------------------------

Line #3.    CREATE OR REPLACE TABLE yearly_stations AS

When this command finishes, is a user able to interact with yearly_stations like a normal variable? E.g., is there some eval magic or something happening? Or is it only internal to the Context

_randerzander commented on 2022-02-04T18:31:05Z_ ----------------------------------------------------------------

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

_beckernick commented on 2022-02-04T20:48:37Z_ ----------------------------------------------------------------

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

randerzander commented 2 years ago

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:09Z ----------------------------------------------------------------

Line #3.    rain_df = _

This IPython specific behavior to get the prior result relies on running the cell in order, which we can't enforce. Can we make this explicit?

_randerzander commented on 2022-02-04T18:27:11Z_ ----------------------------------------------------------------

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #4.    px.line(

Does this not work on the GPU like Plotly Dash (I'm not well versed in Plotly)?

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #1.    px.line(_.to_pandas(), x="yr", y="stations", title="Station Count").show()

Same as above. Can't enforce users running in the same order. I see how this can be challenging with the %%sql magic we're not explicitly assigning to a variable, which makes it appealing to keep this as is.

If we keep as is, we could add a note that these rely on running in the correct order

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:11Z ----------------------------------------------------------------

Line #3.    CREATE OR REPLACE TABLE yearly_stations AS

When this command finishes, is a user able to interact with yearly_stations like a normal variable? E.g., is there some eval magic or something happening? Or is it only internal to the Context

_randerzander commented on 2022-02-04T18:31:05Z_ ----------------------------------------------------------------

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

_beckernick commented on 2022-02-04T20:48:37Z_ ----------------------------------------------------------------

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

randerzander commented 2 years ago

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #4.    px.line(

Does this not work on the GPU like Plotly Dash (I'm not well versed in Plotly)?

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #1.    px.line(_.to_pandas(), x="yr", y="stations", title="Station Count").show()

Same as above. Can't enforce users running in the same order. I see how this can be challenging with the %%sql magic we're not explicitly assigning to a variable, which makes it appealing to keep this as is.

If we keep as is, we could add a note that these rely on running in the correct order

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:11Z ----------------------------------------------------------------

Line #3.    CREATE OR REPLACE TABLE yearly_stations AS

When this command finishes, is a user able to interact with yearly_stations like a normal variable? E.g., is there some eval magic or something happening? Or is it only internal to the Context

_randerzander commented on 2022-02-04T18:31:05Z_ ----------------------------------------------------------------

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

_beckernick commented on 2022-02-04T20:48:37Z_ ----------------------------------------------------------------

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

randerzander commented 2 years ago

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:10Z ----------------------------------------------------------------

Line #1.    px.line(_.to_pandas(), x="yr", y="stations", title="Station Count").show()

Same as above. Can't enforce users running in the same order. I see how this can be challenging with the %%sql magic we're not explicitly assigning to a variable, which makes it appealing to keep this as is.

If we keep as is, we could add a note that these rely on running in the correct order

review-notebook-app[bot] commented 2 years ago

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:11Z ----------------------------------------------------------------

Line #3.    CREATE OR REPLACE TABLE yearly_stations AS

When this command finishes, is a user able to interact with yearly_stations like a normal variable? E.g., is there some eval magic or something happening? Or is it only internal to the Context

_randerzander commented on 2022-02-04T18:31:05Z_ ----------------------------------------------------------------

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

_beckernick commented on 2022-02-04T20:48:37Z_ ----------------------------------------------------------------

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

randerzander commented 2 years ago

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB

beckernick commented on 2022-02-04T16:46:11Z ----------------------------------------------------------------

Line #3.    CREATE OR REPLACE TABLE yearly_stations AS

When this command finishes, is a user able to interact with yearly_stations like a normal variable? E.g., is there some eval magic or something happening? Or is it only internal to the Context

_randerzander commented on 2022-02-04T18:31:05Z_ ----------------------------------------------------------------

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

_beckernick commented on 2022-02-04T20:48:37Z_ ----------------------------------------------------------------

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

randerzander commented 2 years ago

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

randerzander commented 2 years ago

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

beckernick commented 2 years ago

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[randerzander]

It's on by default. Will make that clearer.

---

View entire conversation on ReviewNB

[randerzander]

https://github.com/dask-contrib/dask-sql/issues/281

And here, I specifically want the timing and "select * limit 5" behavior. I don't think most users want this by default.

---

View entire conversation on ReviewNB

[randerzander]

You can. The intent here is to show ability to create tables from both filepaths, and from DataFrames. Will comment to that effect.

---

View entire conversation on ReviewNB

[randerzander]

This is explained in the markdown cell above. Maybe it wasn't clear enough?

---

View entire conversation on ReviewNB

[randerzander]

The whole notebook depends on tables created in prior cells. Will make a note at the top about running in order only.

---

View entire conversation on ReviewNB

[randerzander]

yearly_stations is a table understood by the Dask-SQL context, not a Python variable.

This is how the notebook works for all tables. Is there something specific about this one that made you ask? Or are you noting that should be clearer broadly?

---

View entire conversation on ReviewNB

[beckernick]

I think it's clear that you can do it, but is this the primary/only way? E.g., if I want to mix DF/SQL, should I be doing this or using the SQL magics (which I realize are broken at the moment, per your prior comment). It sounds like the answer is I should be using it like this.

---

View entire conversation on ReviewNB

[beckernick]

Makes sense. Mostly curious as to whether I could do %%sql ... and then mix in dataframe operations "magically"

---

View entire conversation on ReviewNB