New lesson on Docker for Reproducible Research

[nuest]

I'm giving a "short course" (about 30 minutes) on Docker for Reproducible Research next week.

I started using your template, you can see the outline of the lesson here: https://github.com/nuest/docker-reproducible-research respectively https://nuest.github.io/docker-reproducible-research/

I'd like to incorporate feedback from the first course. Would this be a welcomed lesson once it's finished?

[adam3smith]

I'm not sure how @Repositorian feels about this, but my sense is that both Docker and the python course you suggested on the other ticket are better fits for Data Carpentry. I see Author Carpentry as focused on tools for the writing process. There's going to be some overlap, of course (e.g. with R Markdown and Jupyter where you combine writing and analysis) but Data Carpentry is such a well established and put together set of lessons that I'd be inclined to focus on things that clearly don't fit there.

[nuest]

@adam3smith Thanks for your take on this. I understand your argument. I would see the course more on the topic of a daily routine of doing research and academic writing, how containers can help with that. To me this is on a higher level than actual data analysis, which this lesson would not cover at all.

I also have to admit that the data carpentry requirements for a new lesson are well beyond anything I can contribute at the moment. Looking forward to more opinions.

[Repositorian]

Hello Daniel / @nuest ! thank you for suggesting such an interesting and timely lesson. My sense is that this lesson could be a very worthwhile contribution to Carpentry writ large, and I do see strong possibilities for the fit with AuthorCarpentry -- particularly because the lesson outline concludes with steps to 'publish' the docker-ized research output. I agree that the lesson roadmap and lesson development process for Data Carpentry is more formal and rigorous.

The other thought I have is that Caltech's certified Carpentry instructor team, including @tmorrell who is our Python lead, may also find @nuest's lesson a worthwhile enhancement for Data or Software Carpentry workshops we offer at Caltech. As long as we offer the required components of DC/SW in our official Carpentry workshops, we have the freedom to also extend with additional content. That process might be the most fruitful way to try @nuest lesson's; gain insights and provide feedback to improve it; and open the door for the lesson to eventually get placed on the formal Carpentry roadmap.

So, on both fronts, I think the Caltech response would be to welcome @nuest to work on the lesson as he suggested; and @tmorrell and I can continue to provide feedback and encouragement.

@nuest - would that approach be satisfactory as a start? What else would you need from our end at this initial stage?

[tmorrell]

@nuest I agree with @Repositorian that this docker lesson is a good fit for the Author Carpentry curriculum. I think modern publishing needs to include sharing and publication of any associated code, and docker is one potential method of making code more usable. I think the current lesson roadmap and learning objectives would serve as a useful intro to Docker for researchers. I'd definitely consider teaching this lesson as part of our instruction program at Caltech.

One benefit of Author Carpentry is that we have the opportunity to be more flexible in supporting different lesson content. If a lesson continues to be taught and maintained-great! If not, we can mark out of date lessons as depreciated.

@nuest, I'd encourage you to continue to develop your lesson in your repository. Once you've got a first draft I'll fork it into the Author Carpentry organization and set you as the maintainer. Let me know if you've got other thoughts or run into any problems with the template.

[nuest]

Yes, thanks @tmorrell and @Repositorian for the positive feedback. I don't need anything from your end at this point. I'll work on the lesson, try it out next week, and update it then online in the repository and let you know in this issue.

[nuest]

So I completed a first draft of a lessong and presented the bulk of the material in a short cours on Monday. It was not a hands-on course, but some attendees did follow the invitation to follow the instructions and try them out themselves.

I plan to run this course again at my home university with a handful of people (not computer scientists) in a real "hands-on" setting to develop it further. Until then I would of course welcome any feedback on the current material.

@benmarwick Hi! Here is a new idea of mine. What do you think about this?

[tmorrell]

@nuest I think this looks like a great first draft.

You might want to select and provide some example docker images as files that students can download before attending a workshop. I know we struggle with wifi stability when we're teaching a large group, and some of the docker images are rather large (The jupyter one was 5.3 GB).

A bit more introduction at the very beginning to docker would be good- including how to start the application on your machine.

I ran into an issue with the jupyter image. I also got the following error:

docker run -it --rm -p 8888:8888 jupyter/scipy-notebook:eb70bcf1a292 docker: Error response from daemon: driver failed programming external connectivity on endpoint objective_bell (5ba0427cea42d3d476f229d489b39932bef1e156f9d91dd5be923ffbe78ab1be): Error starting userland proxy: Bind for 0.0.0.0:8888 failed: port is already allocated. ERRO[0000] error getting events from daemon: context canceled

I had been running a different jupyter notebook earlier, but it wasn't open when I tried to run docker.

It would also be great to add activities at the end of each section that culminate in each learner generating and sharing a docker image. You've got most of the components (particularly the last section), but explicitly including a hands on activity will allow learners to better understand how and why they could use docker in their work.

[nuest]

I've left this issue lying around for quite some time. In the meantime, the idea of educational material for using Docker/Dockerfiles lead to a "Ten Simple Rules" article, which is now reviewed and published: https://doi.org/10.1371/journal.pcbi.1008316

I'm happy to leave this issue open as it is for someone else to pick it up, but I must admit it's unlikely I'll turn the article above into more of a workshop style website.