search

carpentries-incubator / docker-introduction

Reproducible Computational Environments using Containers
https://carpentries-incubator.github.io/docker-introduction/
Other
42 stars 48 forks source link

Working with data within containers #130

Open vlrieg opened 2 years ago

vlrieg commented 2 years ago

One barrier I've had to using Docker in my own work has been my confusion about how to access data from within a Docker container. This is briefly mentioned in the Creating More Complex Container Images episode, but it might be useful to include more on how containers relate to the rest of the system, and/or links to other resources for those more complicated use cases. In a genomics use-case scenario, the data files might be large and so copying them to the container would not make sense.

jcohen02 commented 2 years ago

Thanks for highlighting this @vlrieg. I agree that this is an area that can cause confusion. The "Using scripts and files from outside the container" section in the episode that you linked to does include some limited details about use of the -v command line option to "bind mount" a host system directory into the container, as you've noted.

This removes the need to copy files into the container since the files stored on the host system can be accessed directly witin the container. Looking at this again, I see that it's a little buried in the text and maybe a clear subheading within that section of something like "Accessing files on the host system from within your containers" might be helpful? A little more context could be added within the text too which might be helpful.

More generally, were you hoping for some technical descriptions of how these things work, or just a little more explanation around the existing material to provide some additional information? I guess some further technical detail could be provided in an info box but it would be good to keep a balance between providing enough information/context and offering too much technical explanation that could cause confusion for learners who are very new to the topic.

I don't know if this is something that was discussed at yesterday's Carpentries discussion and whether there is already a plan to update this? If this is something you were going to work on following a discussion at the meeting that's great, however, if this is just a general suggestion, I'd be happy to make some updates to try and address this point.

vlrieg commented 2 years ago

Hi @jcohen02,

Looking at this again, I see that it's a little buried in the text and maybe a clear subheading within that section of something like "Accessing files on the host system from within your containers" might be helpful? A little more context could be added within the text too which might be helpful.

This would be tremendously helpful in my opinion. From the discussion, it seems my use case is a little bit more fringe than would be necessary for the average person who would be taking this lesson, but confusion around this was my main stumbling block personally so I wanted to share that bit of feedback. I agree that too much technical description wouldn't be useful in this lesson, but making that step a little more explicit would be a huge help to someone like me.

As far as I know, no one else has plans to make these updates!

Thanks so much! Val

jcohen02 commented 2 years ago

No problem, I'm happy to take a look at this. I'll make some modifications to the material and provide an update here once I've got something that can be reviewed.

Once the changes are made, if you wouldn't mind giving them a check through, @vlrieg, it would be great to have some feedback on whether they are helpful and address the challenges that you found when you first looked this topic.

sstevens2 commented 2 years ago

At Maintainer Meeting on 2021-11-23 we discussed this issue and think maybe we need an image that illustrates copying files into a container vs mounting a folder to the container and clarification in the text.

sstevens2 commented 2 years ago

Also this is related to #150