2i2c-org / infrastructure

Infrastructure for configuring and deploying our community JupyterHubs.
https://infrastructure.2i2c.org
BSD 3-Clause "New" or "Revised" License
103 stars 63 forks source link

Replicate to Docker on laptops (individual "right to replicate")? #975

Open jameshowison opened 2 years ago

jameshowison commented 2 years ago

Description of problem and opportunity to address it

Context to understand the problem When I teach it is important to me that students end the course (and their degree) with access to the tools that I've introduced them to. Many of our students at the Texas Information School go on to small or non-profit organizations without access to infrastructure or engineers.

Problem or idea I would love them to have access to the same toolset on the Hub but running on their laptops. Ideally that would include their files from the Hub.

Proposed solution I don't know the set up well enough to propose a solution, perhaps some derivative of the Hub configuration files that produces image (or a Dockerfile). Perhaps something like hub2repo producing something that can run on mybinder (and thus on any local Docker install).

What's the value and who would benefit This is another form of the "right to replicate" but at the individual level.

Implementation guide and constraints

No response

Updates and ongoing work

No response

sgibson91 commented 2 years ago

The user environment for the Texas hub is built from this repo using repo2docker and is pushed to a public quay.io registry :) https://github.com/2i2c-org/utexas-image

repo2docker is the tool that builds all of the image that run on mybinder, so this is Binder-compatible

choldgraf commented 2 years ago

How would that work with the postgres database that's being set up with the user pods? Would a person need to set that up manually on their computer or something?

sgibson91 commented 2 years ago

@choldgraf yeah, probably :confused: I actually don't know how that would work.

This also doesn't address pulling a user's files from the hub either - at least, not in a way that isn't the students manually downloading them

damianavila commented 2 years ago

Pinging @yuvipanda šŸ˜‰ Pretty sure it will have some thoughts about this one since he developed the in-cluster solution (re: https://github.com/2i2c-org/infrastructure/issues/968)

jameshowison commented 2 years ago

In the past I've done that sort of thing (multiple images, which might be the same as the sidecar approach) with docker-compose.

btw, I think the way you guys are setting things up is awesome and makes this sort of feature achievable (not without work).

damianavila commented 2 years ago

In the past I've done that sort of thing (multiple images, which might be the same as the sidecar approach) with docker-compose.

Yep, I was thinking in the same direction šŸ˜‰

yuvipanda commented 2 years ago

yeah, I think docker-compose would be exactly the same way to go here as well. https://github.com/2i2c-org/infrastructure/blob/4e13e5ad8db5b8ee4cf496d25cb4b132f3de8d8d/config/clusters/2i2c.cluster.yaml#L507 should have all the info needed to create a docker-compose file. That does mean it won't work on mybinder.org though.

jameshowison commented 2 years ago

I will keep fiddling with the docker-compose to get the sidecar images working as well.

For now, though, I did figure out how to pull the quay.io image and mount a directory from my laptop visible and editable in JupyterLab and Rstudio. Woohoo.

Dockerfile:

FROM quay.io/2i2c/utexas-image:latest 

docker-compose.yml

version: '3'
services:
 datawrangling-admin:
    build:
        dockerfile: Dockerfile
        context: .
    container_name: datawrangling_admin_1
    ports:
      - "8888:8888"
    volumes:
      - .:/home/jovyan/datawrangling

So I guess the only learning there, really, was everything is via port 8888 (as opposed to 8787 in the rocker images) and that the accessible filepath is /home/jovyan

IO suppose I don't need the Dockerfile, I can just specify the quay.io image directly in the docker-compose file

damianavila commented 2 years ago

@choldgraf, maybe we can showcase somehow the above piece as an example of our "working for real" R2R commitment?

choldgraf commented 2 years ago

@damianavila that's a great idea - hmmm, what would be the best way to do this? Blog post?

damianavila commented 2 years ago

Yep, I was thinking of a short blog post? Maybe coauthored with @jameshowison?

jameshowison commented 2 years ago

Can do! I'd write about the importance of easy onramps for students and having them leave with a working infrastructure, especially as they spread out to organizations large and small.

On Tue, Apr 5, 2022 at 7:55 AM Damian Avila @.***> wrote:

Yep, I was thinking of a short blog post? Maybe coauthored with @jameshowison https://github.com/jameshowison?

ā€” Reply to this email directly, view it on GitHub https://github.com/2i2c-org/infrastructure/issues/975#issuecomment-1088669072, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAWOUTUS4MGRMLMRLQZ7TLVDQZ3PANCNFSM5NPSJTWA . You are receiving this because you were mentioned.Message ID: @.***>