Docker Developer Environments experience

nebuk89 commented 3 years ago

Tell us about your request Provide a simple way for developers to share their work in progress code, connect code into a container to edit it/rebuild it and a way to do this within a compose project using one container as my development environment while still running the others. Keep my development container backed up, up to date and provide simple tools to share these with my team so anyone can look at what I am working on in one click

Which service(s) is this request for? Docker Desktop

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? Making it simple to move between versions of the same code and to develop a containerized application on my local machine. Allow me easily share code in containers with my team mates.

Are you currently working around the issue? Sharing branches which requires git magic on my machine to move between them

Additional context

darkvertex commented 3 years ago

👀

blackblather commented 3 years ago

Good afternoon everyone

I was trying to figure out a way using docker containers as dev environments in a "IDE-independent" way.

The kind of solution I found was to create a Dockerfile that installs everything a developer needs to start developing the project (ie: JDK15 + Tensorflow + etc...). Then manually run the container with a bind volume that points to the project root folder. That way any changes made to the code are visible in the container.

The Dockerfile could then be pushed to the project git repo and anyone that want to contribute to the project simply has to build/run the container and the dev env is configured.

To compile the project we could run the container with the -it flags and compile it that way.

The issue I find with this approach is that it makes it difficult to debug. For instance, IntellIj has a built-in way to remotely debug a project in a docker container, but it's only in the paid version, making this approach not "IDE-independent"

Maybe there needs to be an effort from other IDEs to support such workflows. VSCode already started, maybe others could follow their footsteps (I'm looking at you Jetbrains 👀👀👀)

darkvertex commented 3 years ago

(I'm looking at you Jetbrains 👀👀👀)

FYI JetBrains' PyCharm has Docker-powered remote interpreter support:

https://www.jetbrains.com/help/pycharm/using-docker-as-a-remote-interpreter.html

Has to be manually configured, but it's a start i guess.

PavelSosin-320 commented 3 years ago

This proposal is not so new. The VSCode development leader already proposed an IDE independent Cloud development environment architecture. It was implemented by several companies and deployed to dozens of thousands of Web application developers in various areas: from the pharmaceutical to aerospace. The main difference is that the most universal GUI today is a Web Browser that doesn't need any local installation, i.e. doesn't create dependency.

Jitsusama commented 3 years ago

I am very much looking forward to JetBrains (PyCharm/GoLand/IntelliJ/WebStorm) integration for this.

AviMoto commented 3 years ago

Hi Greate Feature, I really want to share with all my Developrts team, But we all cli linux users so with (Jetbrain Ultimate, use most of the IDE's).

We will be more then happy to have this feature support in CLI and Jetbrain environments

sdanzig commented 3 years ago

Throwing in my vote for IntelliJ support.

darkvertex commented 3 years ago

While you wait for IntelliJ to get on the devcontainer bandwagon, for a more IDE-agnostic bootup of dev containers check out this neat project: https://github.com/nikaro/devc

It's a CLI to start a devcontainer without an IDE and supports most of the devcontainer.json spec.

hagemt commented 3 years ago

I know the dev containers project has broad goals, but since this appears to solicit feedback:

Upfront: this setup works okay for development IF your workload is more-or-less the "hub and spoke" kind--where there is one main code base in the service tier and smaller services "behind" in its orbit... Maybe the "main" code base is an existing monolith that you are [or aren't] decomposing, OR maybe: you have NO monolith, and "main" is just your focus. i.e. "main" = code YOU work on in a "leave one out" strategy--where other services change less often than your focus, and can run the whole system based upon tagged mainline images in some kind of registry, which you do in prod anyway.

Provided you've got and editor or IDE where you do most of your development, that can drive the main focus outside a container and interact via mapped ports on localhost with the "satellite" services and DBs, etc.

In most places I've worked we do this:

if you're outside the k8s ecosystem, just write one or more docker-compose.yml files
spec your data tier and other stuff like SQL servers, Redis, or whatever ^^^ in here
persist data in volumes, which optionally map to the host: ./data/redis:/data:rw
run your actual services with env/net or w/e configured in the compose file(s)
provide a Makefile with targets for developers, which spin everything up/down
if part of your workflow is to seed schema or data into DBs, add a target
you can also fit this around k8s (e.g. via minikube) or transition later
the Makefile just maintains stable targets, e.g. clean, dev, sane, stop

It has the following drawbacks:

this doesn't scale to large teams or projects of a certain scope (one main focus)
developers need some operational awareness, or find CLIs or an IDE/GUIs that help them
compose can have wonky interaction with AWS ECR and other registries (has various solutions)
specifically, compose does not auto-pull; to restart everything w/ mainline satellites in orbit:

# You can create a Makefile target to run the commands in this order.
docker-compose stop # to bring down the local stack before cleanup
docker-compose rm -f
docker-compose pull # or, write a script to docker login (then pull)
docker-compose up -d
# Provided everything came up sanely, you've got latest mainline.

That's a summary of my experience, plus some warnings and advice re: growing pains FWIW.

hagemt commented 3 years ago

Sorry if my last comment was long and boring. That's just how I've coped over the years w/o more turn-key support for this in Desktop. The spec sounds like it'll help folks on many projects, and I guess what I'm also interested to hear is: how will the new features integrate with compose, if at all?

vraravam commented 3 years ago

In most places I've worked we do this:

if you're outside the k8s ecosystem, just write one or more docker-compose.yml files

spec your data tier and other stuff like SQL servers, Redis, or whatever ^^^ in here

persist data in volumes, which optionally map to the host: ./data/redis:/data:rw

run your actual services with env/net or w/e configured in the compose file(s)

provide a Makefile with targets for developers, which spin everything up/down

if part of your workflow is to seed schema or data into DBs, add a target

you can also fit this around k8s (e.g. via minikube) or transition later

the Makefile just maintains stable targets, e.g. clean, dev, sane, stop

This is almost exactly what I have also ended up doing. The result is that this works great for cmd-line, (but also needs a lot of knowledge transfer to any new joinee on the tooling + process involved). The drawback, as stated above, is that this completely forgets about the IDE-based experience.

hagemt commented 2 years ago

Tip: if they're using Jetbrains IDEs, you can create .run configs for the make targets, etc.

@vraravam: That way, some can point and click or "fork" the supported versions that you git-commit.

baumannalexj commented 2 years ago

As noted by others above, this is almost frictionless using Docker Compose with PyCharm atm. (shameless plug) I wrote a walkthrough for setting this up:

https://baumannalexj.medium.com/how-to-use-pycharm-to-remotely-debug-your-flask-app-running-in-docker-c66eb9b8aa72

andrewcrook commented 2 years ago

I wouldn’t mind an option to have all my dev tools in a container that could join a dev environment when I need to work on it.

rnett commented 2 years ago

In case anyone mentioning JetBrains IDEs in here hasn't seen it yet: https://www.jetbrains.com/remote-development/

briceburg commented 2 years ago

It would be nice if we could leverage docker desktop features in headless environments (like CI runners).

For instance, now that Docker Desktop for Linux is GA, it would be helpful to have an official cli tool to leveraging the Dev Environments Feature -- one that automatically loads the custom dev environment in $PROJECT_ROOT/.docker/docker-compose.yaml) in the same manner it executes on local dev machines via the Docker Desktop GUI.

This would allow us to use Dev Environments features in [linux] CI agents/runners.

jjanviers commented 1 year ago

Can only start 1 container on Windows Docker Desktop dev environment?

docker / roadmap

Docker Developer Environments experience #201