search

dask / community

For general discussion and community planning. Discussion issues welcome.
20 stars 3 forks source link

Regular Online Tutorials #57

Open mrocklin opened 4 years ago

mrocklin commented 4 years ago

As a community service we could offer online tutorials hosted by one of the Dask maintainers.

The tutorial at https://tutorial.dask.org was designed to run in a three-to-four hour period (for SciPy). We could offer this, or some stripped down version of it as an online web video call. Hopefully it's relatively straightforward to deliver, and also a good learning experience for those involved in giving it.

My hope is that as more of us deliver this tutorial we then are able to refine and improve the tutorial over time. Preparing, delivering, and then tuning the tutorial afterwards would, I think, be a high value maintenance activity. I think that we may have the bandwidth to start delivering the tutorial somewhat regularly.

jacobtomlinson commented 4 years ago

We could try using Twitch and/or YouTube streaming for this, that way they can be archived afterwards and watched back later. Many conferences that have moved online are using these services and it seems to work quite well.

I've been experimenting this week with streaming bits of my open source maintenance work on Twitch and it was pretty easy to get started.

TomAugspurger commented 4 years ago

This sounds fun.

One question around infrastructure & format though: do you envision attendees following along live and running the examples, asking questions, etc.?

In the past we've temporarily set up a cluster with jupyterhub & Dask so that attendees could play with a real cluster. Would we attempt the same thing here, or would we just have attendees run things locally? My only concern is around who would pay for the cluster computing resources. I'm not too worried about the technical side of things. It's pretty easy to set up a Jupyterhub & Dask Gateway on a kubernetes cluster these days.

Tom

On Fri, May 22, 2020 at 4:18 AM Jacob Tomlinson notifications@github.com wrote:

We could try using Twitch and/or YouTube streaming for this, that way they can be archived afterwards and watched back later. Many conferences that have moved online are using these services and it seems to work quite well.

I've been experimenting this week with streaming bits of my open source maintenance work on Twitch and it was pretty easy to get started.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-632589937, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKAOIUNTKJ62G3V3XOG4FTRSY7MTANCNFSM4NG6DARQ .

mrocklin commented 4 years ago

It seems like there are two approaches to video tutorials

  1. Broadcast with twitch/youtube
  2. Video conference

I think that they're both useful. It seems like twitch/youtube support more viewers, but don't allow for as much engagement. My guess is that they're better if we're doing new content every time. Video conferencing might be better if we have smaller groups (say 10-20) where the students are going through exercises on their own and being coached by community members. This seems better to me if we do the standard tutorial on a regular basis.

My only concern is around who would pay for the cluster computing resources

I think that the Dask/NumFOCUS account could be used if we're looking at something in the few hundred dollars range.

jcrist commented 4 years ago

In the past we've temporarily set up a cluster with jupyterhub & Dask so that attendees could play with a real cluster. Would we attempt the same thing here, or would we just have attendees run things locally?

I recently gave the tutorial using the binder setup at tutorial.dask.org, and found it good enough to teach people how dask works and how to think about it. Yes, they didn't get a real "distributed" cluster, but they still got to see how dask's api works and learned how to reason about the various concepts.

jacobtomlinson commented 4 years ago

@mrocklin yeah I agree with that assessment. I think Twitch/YouTube would be good for doing something like a talk. Especially if the recording would be useful afterwards. But then video conference would be good for a more workshop style session.

mrocklin commented 4 years ago

If we have human resources, I'm inclined to try both and see what engages people.

If I were to organize this I would suggest the following:

  1. We schedule two online tutorials, one for two weeks from now and one for two weeks following. We get ~suckers~ volunteers to sign up to deliver these.
  2. On alternating weeks we also get a couple of people to sign up to give a Twitch-style talks about some topic like "all the ways to deploy Dask on Kubernetes" or "How to use Dask with Xarray"
TomAugspurger commented 4 years ago

I'll volunteer for the second round :)

On Fri, May 22, 2020 at 11:32 AM Matthew Rocklin notifications@github.com wrote:

If we have human resources, I'm inclined to try both and see what engages people.

If I were to organize this I would suggest the following:

  1. We schedule two online tutorials, one for two weeks from now and one for two weeks following. We get suckers volunteers to sign up to deliver these.
  2. On alternating weeks we also get a couple of people to sign up to give a Twitch-style talks about some topic like "all the ways to deploy Dask on Kubernetes" or "How to use Dask with Xarray"

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-632786283, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKAOITRC2VBAXE2B3HDGVLRS2SIBANCNFSM4NG6DARQ .

jacobtomlinson commented 4 years ago

Happy to be a sucker, although I would like to rotate off Czar duties before that time so that I can spend that time preparing.

mrocklin commented 4 years ago

If your Czar duties are tracking dask/distributed then I'm happy to do extra work to allow you to go be a sucker on this one :) I'm excited to see if this is something that we can start doing regularly.

On Wed, May 27, 2020 at 5:42 AM Jacob Tomlinson notifications@github.com wrote:

Happy to be a sucker, although I would like to rotate off Czar duties before that time so that I can spend that time preparing.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-634633612, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTDZ5E2ZG4X53L5WVM3RTUDB3ANCNFSM4NG6DARQ .

jacobtomlinson commented 4 years ago

Ok sounds good.

two weeks from now

From when you sent that message that would be Friday the 5th of June.

Do you have thoughts on a time? I wonder whether we should try and do it at a time like 2pm, which would be practical for folks in immediately neighbouring timezones. Perhaps trying to alternate the presenter between continents week-to-week to be as inclusive as possible.

martindurant commented 4 years ago

Happy to help out too

mrocklin commented 4 years ago

I think that we'll want at least a weak of lead time to advertise and prep. +1 on switching around time zone availability to support a broad set of attendees.

jacobtomlinson commented 4 years ago

Ok given that time is passing I suggest we go for Friday June 12th at 1pm UTC. That'll be 9am east coast US, 2pm for me and 3pm central Europe, so should have good coverage. I also suggest we pencil in 90 minutes for it.

Then maybe next time someone in North America could run it to get coverage for west coast?

I think we should also use Zoom for this first one. Do we have access to a Zoom pro account for avoiding the time limit and doing recordings?

@mrocklin are you able to coordinate promoting it?

mrocklin commented 4 years ago

I think we should also use Zoom for this first one. Do we have access to a Zoom pro account for avoiding the time limit and doing recordings?

Yes, through NumFOCUS. I've sent you and support@numfocus an e-mail

@mrocklin are you able to coordinate promoting it?

Definitely. The obvious things to do is tweet it. I can see if there are a couple of other channels.

dgerlanc commented 4 years ago

I taught a 4-hour workshop on oreilly.com focused on Dask Dataframes and teaching it again at the end of July. It alternates between lecture/exercise. May be up for doing a volunteer teaching session or integrating these into the docs. Here's a link to the notebooks: https://github.com/dgerlanc/dask-scaling-dataframe.

jsignell commented 4 years ago

Is there an invite that I can share?

mrocklin commented 4 years ago

Not currently. We have some work to do around evangelizing this. @jacobtomlinson what would you prefer? We could do something like Evite to rate limit attendees (probably not a problem) and then send out the address to those people a couple of days before. Alternatively we could just blast out a public address.

I'm ok with either. I might lean slightly towards Evite just because it's different and we might learn something. I'm totally happy to defer to you as the person running this though.

On Wed, Jun 10, 2020 at 12:52 PM Julia Signell notifications@github.com wrote:

Is there an invite that I can share?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-642222304, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTHUHCQPH76KZ6CGDPTRV7P6LANCNFSM4NG6DARQ .

jacobtomlinson commented 4 years ago

Yeah I think having people register would be good. We could use something like Eventbrite?

mrocklin commented 4 years ago

Yes, that's what we used for the Dask Summit and it worked well

(I said Evite before, but I think I meant Eventbrite :/ )

On Wed, Jun 10, 2020 at 2:07 PM Jacob Tomlinson notifications@github.com wrote:

Yeah I think having people register would be good. We could use something like Eventbrite?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-642264736, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTAO7CV7GPB2EBDUKF3RV7Y23ANCNFSM4NG6DARQ .

jacobtomlinson commented 4 years ago

@mrocklin do you have an account and are you able to set this up?

jsignell commented 4 years ago

This is still planned for tomorrow right? I am going to tell my work people about it :)

mrocklin commented 4 years ago

We're planning this for next Friday I think

On Thu, Jun 11, 2020 at 7:08 AM Julia Signell notifications@github.com wrote:

This is still planned for tomorrow right? I am going to tell my work people about it :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-642680072, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTD3FXAPUVN4IG5ZKVTRWDQOVANCNFSM4NG6DARQ .

mrocklin commented 4 years ago

@jacobtomlinson no, I don't currently have an account. I can put this on my TODO list for the day though.

On Thu, Jun 11, 2020 at 7:20 AM Matthew Rocklin mrocklin@gmail.com wrote:

We're planning this for next Friday I think

On Thu, Jun 11, 2020 at 7:08 AM Julia Signell notifications@github.com wrote:

This is still planned for tomorrow right? I am going to tell my work people about it :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-642680072, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTD3FXAPUVN4IG5ZKVTRWDQOVANCNFSM4NG6DARQ .

jacobtomlinson commented 4 years ago

Thanks!

jacobtomlinson commented 4 years ago

I had a bit of a play around with using OBS and Blender to make my video setup for the tutorials a little more professional.

I made a couple of scene transitions, they use the Dask logo and add a nice touch. Although they should be used sparingly (way less then I do in this example).

I also made some little elements that pop in to show more information to the viewers. I can share the Blender files, they are quite easy to edit, but I'm also happy to just bulk render out a load that folks can reuse when we run the tutorial again.

I think this would add a nice professional touch to the tutorials, but keen to hear thoughts from others.

Example video on YouTube

mrocklin commented 4 years ago

Ha! This is great!

Personally I prefer the fade over the wipe. My personal style tends to be understated though. The titles/twitter thing looked great as well.

I have to say I'm reallly enjoying you taking on the whole video evangelism role. I'm looking forward to seeing this progress.

willirath commented 4 years ago

Just watched it. Here’s my immediate thoughts:

jsignell commented 4 years ago

I like the name/twitter as is and they feel very polished. But I agree that the diagonal wipe is probably too much. The stripes looked like construction tape to me which might be where the feeling of alert comes from. I thin k the horizontal wipe could be nice for continuity between tutorials, but I think it needs to slow down a bit in the middle if that's possible.

jcrist commented 4 years ago

Nice work @jacobtomlinson.

ccing @lauralorenz, who does all our livestreams at Prefect, in case she has any OBS general advice.

lauralorenz commented 4 years ago

Hi @jacobtomlinson, as @jcrist mentions I'm managing the livestreams at Prefect and also use OBS. You seem to have a good handle on it based on your example vid and your animations look really sleek IMHO! But I'm happy to swap stories some time :) My general setup on OBS for coding livestreams is

My recent new experiments have been about

Ironically I was never a fan of video tutorials myself, but we have observed there is a subset of users who prefer them, and it has been really useful to pass along video links to new users in particular.

jacobtomlinson commented 4 years ago

Thanks for the info @lauralorenz. I'd definitely take you up on the offer to swap stories at some point, perhaps after we run the first video tutorial this week so that I have some lessons learned from that.

I've been looking for a decent virtual audio mixer so thanks for the pointer to blackhole. I'll check it out.

I haven't had any performance issues so far, but I guess much of the work I'm doing doesn't actually happen on my local laptop. For the tutorial we will be using Binder so everything is offloaded there.

I've been using a chat overlay from restream in my personal streams, mainly to be able to aggregate YouTube, Twitch and Periscope chat into one timeline that everyone can see. But I appreciate there are risks associated with doing that. I think if we were to do something similar for Dask it would be sensible to have a moderator filtering the messages.

mrocklin commented 4 years ago

Feedback from the tutorial:

My role during this tutorial was to monitor the chat window, which was generally pretty fun. However, having an encyclopaedic knowledge of documentation and links to point people to was the only thing that made this really doable. I'd suggest that among the two people presenting, the person handling the chat should probably be the more senior of the two. Also because it's nice to give junior folks the more visible role.

Content

Jacob's flow seemed pretty good. It started on pretty accessible materials and then evolved nicely.

I might drop dask.bag, unless we can tie it into an actual workflow (perhaps JSON -> dask dataframe). Things like foldby didn't feel natural

Scale

Presenter view: yeah, it doesn't matter so much Chat view: this was a good amount of activity, probably couldn't handle twice the flow

Next time

Let's do this again, maybe find a better way to handle chat which would allow us to scale to larger numbers, perhaps by allowing people to upvote.

mrocklin commented 4 years ago

Video is uploading and proessing now. It will be available at https://youtu.be/1h0EbX02WeM

mrocklin commented 4 years ago

Nope, that failed because YouTube now requires verified accounts for longer videos. I've verified (I think) and am now reuploading here: https://youtu.be/_u0OQm9qf_A

jacobtomlinson commented 4 years ago

It was great I'd definitely be keen to do it again.

Some thoughts:

cgentemann commented 4 years ago

Hi! I attended this morning. Jacob and Matthew - great job!!!! I really appreciated the time and preparation you put into the tutorial.

I think it is interesting to re-imagine what a tutorial is, given that I was still in my PJs and no one knew. My commute was 10'. So, if we all aren't expending the effort to travel to a similar location, then there really isn't any need to meet for an extended period of time. I found Jacob walking us through the tutorials to be really interesting. I wish that it was about 50% slower and I wanted him to spend more time describing Dask and how it was being used, or what functions might be interesting to use here, or his thought process about how he uses these tools for different types of problems and how he troubleshoots/finds solutions.

While tutorials traditionally have show code / do code / repeat structure, I didn't find the coding part as useful during the tutorial. I want to try out the code, but I can do that on my own.

I also really liked reading the chat & response. It worked well to have Jacob focus on the tutorial and Matthew on the chat. I think it would have made it a little more interactive to have Jacob pause every so often & have Matthew answer a chat verbally or have a short conversation with the person who posted. Yes, this will slow things down, but this is why we are all attending -- there are lots of tutorials we can do online or by ourselves, I wanted to hear from Jacob/Matthew how they used Dask and have more of a conversation.

It almost seemed like it would be useful to break up the tutorial into much smaller bits and just do them more frequently.

Again, great job. This was really useful and I hope to see more in the future. I'd love to see how this works on twitch or other platforms.

lesteve commented 4 years ago

This looks very nice! For some reason the YouTube video definition is 360p maximum (which makes it hard to read the text of the notebook) but maybe it is just YouTube that takes some time to upload higher resolution videos.

Also I edited the chat log to make it more readable, see https://gist.github.com/lesteve/12042f7df7c3843a9ddde52ccbd0e301 (I never know whether people get notifications from gist comments ...)

mrocklin commented 4 years ago

I think that the plan is for @jacobtomlinson to write up the process that we used into a blogpost. Then hopefully @martindurant + someone else will use this plan to run through the process two weeks from now, hopefully at a time that is more convenient for US folks, especially on the west coast.

Additionally on the Pangeo call today I mentioned that it would be fun to repeat this process for other Dask-adjacent projects. I think that Xarray would be a good first candidate for this. @andersy005 said that he felt comfortable volunteering for this.

andersy005 commented 4 years ago

I think that the plan is for @jacobtomlinson to write up the process that we used into a blogpost.

👍🏽 I am looking forward to this post, and thank you in advance for putting this together!

martindurant commented 4 years ago

@andersy005 the blog post appeared at https://blog.dask.org/2020/08/21/running-tutorials , if you didn't see. If you are interested in running a similar xarray online tutorial, I'd be happy to partner, to help set things up.

andersy005 commented 4 years ago

Thank you, @martindurant! What does your schedule look like? I am thinking we could schedule the first session around Friday, Oct 2? @dcherian, would you be interested in join us?

dcherian commented 4 years ago

Sure.

martindurant commented 4 years ago

Oct 2 is fine with me.

andersy005 commented 4 years ago

For some reason the YouTube video definition is 360p maximum (which makes it hard to read the text of the notebook)

@jacobtomlinson, do you happen to know the exact root cause for the low quality recording? Do you have any recommendations for addressing this issue?

Also,

Zoom worked ok and had some tools for user feedback. However it felt more like a 50 person meeting than a presentation so something like a YouTube livestream may be better in future.

I am curious to know how the subsequent tutorials (after the first one) were delivered. Did you use Zoom again? How was the overall experience? Are there any recordings on YouTube? What are your thoughts on Youtube Livestream? I am thinking of using unlisted Youtube Livestream for the upcoming xarray tutorial since I don't have access to a Zoom pro account.

martindurant commented 4 years ago

Yes, we used zoom, without any special effects - so the quality was totally fine. I believe the low quality before was to do with the use of OBS. You can screen-share and record the speaker and/or gallery (this with a licensed account). I didn't post the recording, being so similar to the previous tutorial. For some reason, the feedback buttons were missing from the chat on my session, that was the only annoyance, but I'm not sure I would have used them.

(I can, of course, provide a zoom room, if that makes life easier)

andersy005 commented 4 years ago

Yes, we used zoom, without any special effects - so the quality was totally fine. 👍

I can, of course, provide a zoom room, if that makes life easier

yes, please :) and thank you! That would be very helpful. I plan to send a reminder email with the video conference information tomorrow morning... I will ping you via Gitter DM for the zoom link/password to use in the reminder email.

mrocklin commented 2 years ago

Bringing this up again. @ncclementi @jsignell @pavithraes and @jacobtomlinson just organized and built a tutorial for SciPy. I'd like for us to consider delivering this repeatedly online. I'll suggest the following format to get started, but there are probably lots of improvements that could be made.

Thoughts?

jacobtomlinson commented 2 years ago

Totally agree with the general sentiment here. I feel like monthly may be too much. When we did this the first time it felt like numbers dropped quickly after we reached saturation.

Perhaps bi-montly or quarterly would be a better place to start. We could always ramp up if demand is higher.

I like the idea of one hour per week for four weeks though.

mrocklin commented 2 years ago

Let's do it once at least! 🙂

I'd be in favor of doing it again the following month and see what happens. I'm unlikely to be the one doing the actual work though, so I'm happy to be overruled 😉

I think that the two person structure worked well (one person presenting, another on chat). Does anyone have the bandwidth to organize this?

On Tue, Jul 12, 2022, 9:24 AM Jacob Tomlinson @.***> wrote:

Totally agree with the general sentiment here. I feel like monthly may be too much. When we did this the first time it felt like numbers dropped quickly after we reached saturation.

Perhaps bi-montly or quarterly would be a better place to start. We could always ramp up if demand is higher.

I like the idea of one hour per week for four weeks though.

— Reply to this email directly, view it on GitHub https://github.com/dask/community/issues/57#issuecomment-1181826188, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTBT2FOZZC7B5V3Y7VDVTV52PANCNFSM4NG6DARQ . You are receiving this because you were mentioned.Message ID: @.***>

martindurant commented 2 years ago

@douglasdavis - no pressure, but thought you might find this idea interesting