[EVENT] deeplearning study group

[marimeireles]

I'm proposing holding space for a study group on the dl course by hugging face (https://huggingface.co/deep-rl-course/unit1/introduction?fw=pt) I need to have it discussed in a c-base circle the circle happens next Tuesday 14.03 I'll be there also to join as a member and we might have cbase as a new location to host our events =)

[marimeireles]

It's not going to be following hugging face's course anymore because it's a very bad course, IMO. Looking for a better one, maybe some traditional uni one or just a book? Still not sure. All of this was delayed because of my covid in the week they hold the meeting. Next meeting 01.04. Will update the issue, thank you!

[marimeireles]

Some updates: I've applied to c-base. Soon should get the reply that membership is okay. Still not sure how to spin up this group. Should we have general objectives for the week? Or for every two weeks? I was looking into Spinning Up as a Deep RL Researcher by openai and they have some good resources and what I consider good guidelines to go about building knowledge to understand DL.

Maybe we can decide on a theme for the week together, learn about it, show demos? Should I post recommendations of what to read and do? Not of specific project but broader themes, so we have something that tie us together. We can present to each other our demos. Or we can always elect someone in the week that will give everyone else a small presentation of what was the week's theme... Not sure, folks.

Will update this again once I have a positive reply by cbase.

@linges @morganrivers Emma, feel free to drop any ideas.

[marimeireles]

Another source that might be interesting: https://course.fast.ai/Lessons/part2.html This popped up in HN today, it's a course directed for people to understand how stable diffusion works. I also found this reading relevant: https://80000hours.org/articles/ml-engineering-career-transition-guide/

[marimeireles]

Here's a rough outline:

• introductory knowledge, estimated time is variable here, depends on how much you know already
  • Build up a solid mathematical background. From probability and statistics, feel comfortable with random variables, Bayes’ theorem, chain rule of probability, expected values, standard deviations, and importance sampling. From multivariate calculus, understand gradients and (optionally, but it’ll help) Taylor series expansions
  • Build up a general knowledge of deep learning. You don’t need to know every single special trick and architecture, but the basics help. Know about standard architectures (MLP, vanilla RNN, LSTM (also see this blog), GRU, conv layers, resnets, attention mechanisms), common regularizers (weight decay, dropout), normalization (batch norm, layer norm, weight norm), and optimizers (SGD, momentum SGD, Adam, others). Know what the reparameterization trick is.
• we start with the fast.ai course, 44h of video class (?) + time for exercises, couldn't find estimates of how long
• implement algorithms, estimated time is around 200h
  • Which algorithms? You should probably start with vanilla policy gradient (also called REINFORCE), DQN, A2C (the synchronous version of A3C), PPO (the variant with the clipped objective), and DDPG, approximately in that order. The simplest versions of all of these can be written in just a few hundred lines of code (ballpark 250-300), and some of them even less (for example, a no-frills version of VPG can be written in about 80 lines). Write single-threaded code before you try writing parallelized versions of these algorithms. (Do try to parallelize at least one.)

And that's it! Once we've done these things we'll probably know RL :) Now I'm a member at c-base and will propose to have a meetup space. Will let you know through here about possible dates.

[marimeireles]

This is ongoing! I'm stuck because cbase has bi-monthly meetings and it's really hard to get a hold on them.

[marimeireles]

This is finally happening yaay! Should I close this? Even though it's an ongoing event.