Redefine learning objectives

[agitter]

Now that we have conducted our first pilot workshops, we would like to revisit the content to decide what is truly essential for beginners and what are our learning objectives. The goals we previously shared with workshop participants were:

• Know what machine learning does and why it can be useful in biological research.
• Be able to read papers that apply (simple) ML to biological problems.
• Understand (some of) the language used by computational biologists and be more comfortable to collaborate with them.
• Be prepared to explore opportunities of applying ML in your own projects.

@cmilica will help with this

[cmilica]

These are the objectives that came very clear to me after the previous conversations and the abstract: (I would suggest breaking them down a bit maybe, and being a little bit more specific) 1) ml4bio workshop prioritizes teaching machine learning literacy, that is, the right way to set up learning problems, how to reason about learning algorithms, and how to assess learned models.

2) demonstrating a software pipeline for certain tasks with the emphasis on learning the best practices in various stages of the workflow.

3) explore novel educational frameworks in order to address these challenges in teaching machine learning to biologists.

4) equipping biologists with the proper mindset when it comes to applying machine learning in their research and the ability to critically analyze machine learning applications in their domain.

5) creating the software and interactive exercises to guide participants through a full cycle of the machine learning workflow while doing proper model training, validation, selection, and testing.

6) mind the gap between theory and practice through the illustration of machine learning applications on real biological tasks.

[agitter]

Notes from our discussion today:

• Focus on best practices: if you are working in this part of the pipeline, here are specific recommendations (e.g. evaluation for different types of datasets)
• Look for real examples of where using an automated process with ML would speed up a lab activity substantially. Image labeling comes to mind. We would have to pre-process the images into tabular data or show a realistic workflow and then adjust it for the workshop. The goal would be to show participants what can be done instead of training them in all the software needed to do this with new data.
• We need to define "literacy" in this context (#49). Look into definitions from related workshops or Rob Nowak's "Machine learning for everyone" lecture. It may relate to understanding the jargon of ML, which would mean we should consult or develop a vocabulary.
• We may be able to learn from feedback on public Coursera-like courses or contact the owners of these courses to see what feedback they receive.

[cmilica]

More concrete objectives: (this is what participants should be able to do after the workshop) 1) Define machine learning

• definitions and examples

2) Identify machine learning concepts

• demonstrate ML literacy (QUESTION: what is literacy? #49 )
• distinguish a model accuracy (QUESTION: how are we doing this?)
• make sense of the language used by computational biologists (NOTE: I would make this more concrete)

3) Extend the ML concepts to a project

• (QUESTION: how exactly are they doing this?)

4) QUESTION: what skills should participants have mastered as the result of this workshop?

• explain ML workflow
  • Step 1 Preprocessing: I would completely get rid of explaining the Preprocessing. That is out of the scope of the workshop. We are assuming that the participants have knowledge/training in conducting an experiment. This should be just briefly mentioned.
  • REQUEST It is very hard for me to follow the presentation after this - can someone spend an hour with me to go through it?

5) Classifiers

• not sure how to word this yet. We should pick only the most exciting and important classifiers in the bio field.

[agitter]

@cmilica we could spend our meeting tomorrow looking over the rest of the presentation if there aren't any more urgent discussions we need to have.

I agree with your point about making the language more concrete.

We have a few slides and a couple activities in the ml4bio software to introduce accuracy and other performance measures. We may want to spend more time on this.

Being able to understand parts of a paper in their biological domain that uses machine learning (#44) can be another objective.

[agitter]

Notes from our discussion today

• Prostate cancer example isn't a case of speeding up lab work. Segmentation or labeling cell images is a simple compelling example and we could find recent papers about that. However, it is hard for us to demonstrate.
• Prostate cancer example doesn't make it clear what the clinical value is of having this classifier. May want to start with a more compelling example (even if we can't use that exact dataset) like subtyping, treatment response prediction, etc. Then return to the accessible, simple dataset.
• Start the workshop with more exciting flashy examples and keep the skepticism and combating over-hype of machine learning later. We need to hook the audience up front.
• We'll revise the flow and slides to focus on going through scientific manuscripts. Build interactive activities around example papers. We may highlight papers that fail to describe their methodology. This could come at the end of day 1.
• Continue to focus on a couple core classifiers. Neural networks and random forests are popular and useful, and logistic regression and decision trees are the special cases that help explain them.
• Remove pre-processing part. For this workshop, we need to assume it is clean so we can keep focused on our learning objectives. We could keep data encoding and definitely keep what the rows and columns in a dataset are (dataset structure).
• Is the training process and what differentiates two classifiers truly beginners level? Or is that a slightly more advanced follow up?
• Simplify the learning part to focus on how we choose among models and classifiers.
• May want to only present train/validation/test splitting for model selection.

[agitter]

One of the earlier stated objectives was

Be prepared to explore opportunities of applying ML in your own projects.

If we want to keep that, we could design more specific activities to teach and assess the applicability of classification for different types of tasks. The goal would be to help participants understand what types of problems or datasets may be perfect for machine learning and which are a poor fit.

[cmilica]

Tiny problem - I don't have push access to gitterlab. If you can give me access so that I can push the learning objectives. And the issues don't support that markdown files, so I can't post it here.

[cmilica]

Should we close this issue?

[agitter]

Should we move the learning objectives somewhere that is visible in the workshop web site before closing? One idea would be to move the current file to be a sub-page under the Extras menu.

[cmilica]

I am not sure why I am just getting to this - but yes, I think moving it to the Extras menu would be good. Should I do it, or will you do it?

[agitter]

I'll do it. I just opened a new pull request.