VS Code Dev Container setup

grll commented 4 years ago

Description

Hi,

I spent some time today setting up vs code with "dev container" to work with cookiecutter-django. At the end the dev experience is pretty good as you can code seemlessly directly from within the "django" container with the python interpreter linting and everything coming from the container.

I will share here my setup but it can definetly be improved and added to cookiecutter-django. I did the following things:

install git, zsh and ohmyzsh inside the container when vs code is attaching to it.
add python, and intellicode extensions when vs code attach to the container.
setup linting, formatting and testing to work out of the box when attaching to the container.
override the docker-compose to add a custom .zshrc volume which export the missing ENV variable.

Rationale

Better DX on VS Code using dev container.

Use case(s) / visualization(s)

Once the project created, I just added a .devcontainer folder at the root of my project with the following files:

A devcontainer.json to set what happens when vs code is attaching to the container:

// devcontainer.json
// For format details, see https://aka.ms/vscode-remote/devcontainer.json or this file's README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.112.0/containers/docker-existing-docker-compose
// If you want to run as a non-root user in the container, see .devcontainer/docker-compose.yml.
{
  "name": "Existing Docker Compose (Extend)",
  // Update the 'dockerComposeFile' list if you have more compose files or use different names.
  // The .devcontainer/docker-compose.yml file contains any overrides you need/want to make.
  "dockerComposeFile": ["../local.yml", "./docker-compose.override.yml"],
  // The 'service' property is the name of the service for the container that VS Code should
  // use. Update this value and .devcontainer/docker-compose.yml to the real service name.
  "service": "django",
  // The optional 'workspaceFolder' property is the path VS Code should open by default when
  // connected. This is typically a file mount in .devcontainer/docker-compose.yml
  "workspaceFolder": "/app",
  // Set *default* container specific settings.json values on container create.
  "settings": {
    "terminal.integrated.shell.linux": "/bin/zsh",
    "[python]": {
      "editor.rulers": [120]
    },
    "editor.formatOnSave": true,
    "python.pythonPath": "/usr/local/bin/python",
    "python.linting.pylintEnabled": true,
    "python.formatting.provider": "black",
    "python.linting.flake8Enabled": true,
    "python.linting.mypyEnabled": true
  },
  // Add the IDs of extensions you want installed when the container is created.
  "extensions": [
    "ms-python.python",
    "visualstudioexptteam.vscodeintellicode",
    "esbenp.prettier-vscode"
  ],
  // Uncomment the next line if you want start specific services in your Docker Compose config.
  // "runServices": [],
  // Uncomment the next line if you want to keep your containers running after VS Code shuts down.
  // "shutdownAction": "none",
  // Uncomment the next line to run commands after the container is created - for example installing git.
  "postCreateCommand": "apt-get update && apt-get install -y git zsh wget && wget https://github.com/robbyrussell/oh-my-zsh/raw/master/tools/install.sh -O - | zsh || true"
  // Uncomment to connect as a non-root user. See https://aka.ms/vscode-remote/containers/non-root.
  // "remoteUser": "vscode"
}

a new .zshrc for adding the missing env variable to the shell:

# If you come from bash you might have to change your $PATH.
# export PATH=$HOME/bin:/usr/local/bin:$PATH

# Path to your oh-my-zsh installation.
export ZSH="/root/.oh-my-zsh"

# Set name of the theme to load --- if set to "random", it will
# load a random theme each time oh-my-zsh is loaded, in which case,
# to know which specific one was loaded, run: echo $RANDOM_THEME
# See https://github.com/ohmyzsh/ohmyzsh/wiki/Themes
ZSH_THEME="robbyrussell"

# Set list of themes to pick from when loading at random
# Setting this variable when ZSH_THEME=random will cause zsh to load
# a theme from this variable instead of looking in ~/.oh-my-zsh/themes/
# If set to an empty array, this variable will have no effect.
# ZSH_THEME_RANDOM_CANDIDATES=( "robbyrussell" "agnoster" )

# Uncomment the following line to use case-sensitive completion.
# CASE_SENSITIVE="true"

# Uncomment the following line to use hyphen-insensitive completion.
# Case-sensitive completion must be off. _ and - will be interchangeable.
# HYPHEN_INSENSITIVE="true"

# Uncomment the following line to disable bi-weekly auto-update checks.
# DISABLE_AUTO_UPDATE="true"

# Uncomment the following line to automatically update without prompting.
# DISABLE_UPDATE_PROMPT="true"

# Uncomment the following line to change how often to auto-update (in days).
# export UPDATE_ZSH_DAYS=13

# Uncomment the following line if pasting URLs and other text is messed up.
# DISABLE_MAGIC_FUNCTIONS=true

# Uncomment the following line to disable colors in ls.
# DISABLE_LS_COLORS="true"

# Uncomment the following line to disable auto-setting terminal title.
# DISABLE_AUTO_TITLE="false"

# Uncomment the following line to enable command auto-correction.
# ENABLE_CORRECTION="true"

# Uncomment the following line to display red dots whilst waiting for completion.
# COMPLETION_WAITING_DOTS="true"

# Uncomment the following line if you want to disable marking untracked files
# under VCS as dirty. This makes repository status check for large repositories
# much, much faster.
# DISABLE_UNTRACKED_FILES_DIRTY="true"

# Uncomment the following line if you want to change the command execution time
# stamp shown in the history command output.
# You can set one of the optional three formats:
# "mm/dd/yyyy"|"dd.mm.yyyy"|"yyyy-mm-dd"
# or set a custom format using the strftime function format specifications,
# see 'man strftime' for details.
# HIST_STAMPS="mm/dd/yyyy"

# Would you like to use another custom folder than $ZSH/custom?
# ZSH_CUSTOM=/path/to/new-custom-folder

# Which plugins would you like to load?
# Standard plugins can be found in ~/.oh-my-zsh/plugins/*
# Custom plugins may be added to ~/.oh-my-zsh/custom/plugins/
# Example format: plugins=(rails git textmate ruby lighthouse)
# Add wisely, as too many plugins slow down shell startup.
plugins=(git)

source $ZSH/oh-my-zsh.sh

# User configuration

# export MANPATH="/usr/local/man:$MANPATH"

# You may need to manually set your language environment
# export LANG=en_US.UTF-8

# Preferred editor for local and remote sessions
# if [[ -n $SSH_CONNECTION ]]; then
#   export EDITOR='vim'
# else
#   export EDITOR='mvim'
# fi

# Compilation flags
# export ARCHFLAGS="-arch x86_64"

# Set personal aliases, overriding those provided by oh-my-zsh libs,
# plugins, and themes. Aliases can be placed here, though oh-my-zsh
# users are encouraged to define aliases within the ZSH_CUSTOM folder.
# For a full list of active aliases, run `alias`.
#
# Example aliases
# alias zshconfig="mate ~/.zshrc"
# alias ohmyzsh="mate ~/.oh-my-zsh"

# env variable for django cookiecutter
export CELERY_BROKER_URL="${REDIS_URL}"

if [ -z "${POSTGRES_USER}" ]; then
    base_postgres_image_default_user='postgres'
    export POSTGRES_USER="${base_postgres_image_default_user}"
fi
export DATABASE_URL="postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}"

and finally the docker-compose.override.yml:

version: "3.0"

services:
  django:
    volumes:
      - ./.devcontainer/.zshrc:/root/.zshrc

Let me know what you think about this setup

demestav commented 4 years ago

@grll Thank you for putting this together. I haven't used devcontainers before, and I was wondering how is the development experience improved.

grll commented 4 years ago

there are several development improvement by using this approach. The fact that the IDE is within the container means that you don't need to work with potentially slow mounted volumes, most importantly all the IDE related experience can now be encapsulated within the container and easily shared with your colleagues on a project bases (prettier, pylint, the python interpreter ...) so you not only share the code for the project but also the complete environment to work with the code

MicBoucinha commented 4 years ago

@grll nice setup!

What is your experience with debugging in your setup?

Have you tried the new debugpy integration in the Python extension? I am currently struggling to make it work. If you have any tips could you share them please?

Thanks!

jfilter commented 3 years ago

If you have problems with the missing DATABASE_URL, add this to devcontainer.json

{
...
  // set ENV variables such as DATABASE_URL
  "postCreateCommand": "echo 'source /entrypoint' >> ~/.bashrc",
}

This ensures that the environment variables from /entrypoint get set every time you start an interactive Bash session. However, you need to remove some shell options, so a session does not close whenever a program exists with a non-zero exit code.

# production/django/entrypoint
# comment this out or remove them
# set -o errexit
# set -o pipefail
# set -o nounset

MBARIMike commented 3 years ago

I appreciate this description!

My configuration was getting a database connection error using zsh. I fixed it with this change to devcontainer.json:

-  "postCreateCommand": "apt-get update && apt-get install -y git zsh wget https://github.com/robbyrussell/oh-my-zsh/raw/master/tools/install.sh -O - | zsh || true"
+  "postCreateCommand": "wget https://github.com/robbyrussell/oh-my-zsh/raw/master/tools/install.sh -O - | zsh || true && echo 'source /entrypoint' >> ~/.zshrc"

and installing git and wget in my debug Dockerfile.

Also, I had a problem using docker-compose.override.yml (can a file be mounted?) so I removed it and the .devcontainer/.zshrc file. I suppose this means I'll need to edit the container's /root/.zshrc with any desired changes.

MBARIMike commented 3 years ago

I finally got things working for debugging View code in VS Code. A few notes:

No need to use debugpy, I changed by debug /start script to start runserver_plus on port 8001
Set up a launch.json configuration to start runserver_plus on port 8000 – this works for starting a View code debug session
Had to comment out the **/compose* line that VS Code added in .dockerignore
Committed two .vscode config files to my repo: launch.json and settings.json

It seems that cookiecutter-django could add some set of these files to provide a VS Code friendly setup.

m-roberts commented 1 year ago

+1 for out-of-the-box optional VS Code setup.

masavini commented 1 year ago

would you accept a pull request for this issue..?

browniebroke commented 1 year ago

would you accept a pull request for this issue..?

Yes sure

browniebroke commented 1 year ago

Any luck?

masavini commented 1 year ago

i have some experience with vscode dev containers, but i've been using cookiecutter for just a couple of weeks, so please be patient if the pull request is too "simple". any feedback and correction is appreciated, i might also write some documentation about it if you want to.

pamelafox commented 1 year ago

I'm following this issue as I've been using Dev Containers a lot recently and would love to see them in cookie-cutter. If it helps, here's a Dev Container for Django+PostgreSQL: https://github.com/pamelafox/django-quiz-app/tree/main/.devcontainer It uses docker-compose.yaml to start postgreSQL inside the container. I imagine you could add redis to that docker-compose.yaml as well. Let me know if I can help!

masavini commented 1 year ago

It uses docker-compose.yaml to start postgreSQL inside the container. I imagine you could add redis to that docker-compose.yaml as well. Let me know if I can help!

I think that's a bit overkill: why do you need postgres or redis inside the dev container? i've never seen such a setup before...

pamelafox commented 1 year ago

Two reasons: 1) I find it painful to setup Postgres on the host machine and to make sure DBs have different names across apps. 2) I can use Github Codespaces to develop the apps (all in browser) which is awesome when teaching workshops.

That said, if you're using the Docker option for cookie-cutter, then running docker-compose -f local.yml build takes care of setting up postgres. I've been using the non-Docker option.

Ah and looking at the original post, their devcontainer.json does reference local.yml.

masavini commented 1 year ago

... then you should try my fork and let me know! :)

you just have to create a new cookiecutter project answering yes to the `use_vscode_devcontainer" question, then open the project folder with vscode and click "open in container".

it takes a few minutes to build the images, but then it should work pretty straightforward...

usr3 commented 1 year ago

@masavini Hi, I tried running your fork in WSL2, and got this error:

Start: Run in container: mkdir -p '/tmp/.X11-unix'

mkdir: cannot create directory ‘/tmp/.X11-unix’: File exists
Exit code 1

masavini commented 1 year ago

@masavini Hi, I tried running your fork in WSL2, and got this error:
Start: Run in container: mkdir -p '/tmp/.X11-unix'

mkdir: cannot create directory ‘/tmp/.X11-unix’: File exists
Exit code 1

then you should comment (or delete) the lines of devcontainer.json where /tmp is added as a bind mount.

    "mounts": [
        {
            "source": "./.history/bash_history",
            "target": "/home/vscode/.bash_history",
            "type": "bind"
        },
        {
            "source": "/tmp",
            "target": "/tmp",
            "type": "bind"
        }
    ],

should become:

    "mounts": [
        {
            "source": "./.history/bash_history",
            "target": "/home/vscode/.bash_history",
            "type": "bind"
        }
    ],

browniebroke commented 1 year ago

@masavini Hi, I tried running your fork in WSL2, and got this error:
Start: Run in container: mkdir -p '/tmp/.X11-unix'

mkdir: cannot create directory ‘/tmp/.X11-unix’: File exists
Exit code 1

Which OS are you running @usr3?

usr3 commented 1 year ago

@browniebroke Windows 11 on host Ubuntu 20.04.5 on WSL2

fraserdominicdavid commented 1 year ago

Hi @masavini, I tried running your fork (or rather, applied the changes from it to my existing project generated from cookiecutter-django), but for some reason pytest doesn't seem to work inside the dev container. Running pytest through docker-compose works fine though. I got the following traceback from running pytest inside the dev container:

Traceback

``` Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/_pytest/main.py", line 269, in wrap_session session.exitstatus = doit(config, session) or 0 ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/main.py", line 322, in _main config.hook.pytest_collection(session=session) File "/usr/local/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__ return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall return outcome.get_result() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result raise ex[1].with_traceback(ex[2]) File "/usr/local/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall res = hook_impl.function(*args) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/main.py", line 333, in pytest_collection session.perform_collect() File "/usr/local/lib/python3.11/site-packages/_pytest/main.py", line 665, in perform_collect self.items.extend(self.genitems(node)) File "/usr/local/lib/python3.11/site-packages/_pytest/main.py", line 832, in genitems rep = collect_one_node(node) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/runner.py", line 547, in collect_one_node rep: CollectReport = ihook.pytest_make_collect_report(collector=collector) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__ return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_callers.py", line 55, in _multicall gen.send(outcome) File "/usr/local/lib/python3.11/site-packages/_pytest/capture.py", line 858, in pytest_make_collect_report out, err = self.read_global_capture() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/capture.py", line 780, in read_global_capture return self._global_capturing.readouterr() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/capture.py", line 684, in readouterr out = self.out.snap() if self.out else "" ^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/capture.py", line 570, in snap self.tmpfile.truncate() FileNotFoundError: [Errno 2] No such file or directory During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/_pytest/main.py", line 289, in wrap_session config.notify_exception(excinfo, config.option) File "/usr/local/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1100, in notify_exception res = self.hook.pytest_internalerror(excrepr=excrepr, excinfo=excinfo) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__ return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall return outcome.get_result() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result raise ex[1].with_traceback(ex[2]) File "/usr/local/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall res = hook_impl.function(*args) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/capture.py", line 888, in pytest_internalerror self.stop_global_capturing() File "/usr/local/lib/python3.11/site-packages/_pytest/capture.py", line 755, in stop_global_capturing self._global_capturing.pop_outerr_to_orig() File "/usr/local/lib/python3.11/site-packages/_pytest/capture.py", line 637, in pop_outerr_to_orig out, err = self.readouterr() ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/capture.py", line 684, in readouterr out = self.out.snap() if self.out else "" ^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/capture.py", line 570, in snap self.tmpfile.truncate() FileNotFoundError: [Errno 2] No such file or directory During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/bin/pytest", line 8, in sys.exit(console_main()) ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/config/__init__.py", line 189, in console_main code = main() ^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/config/__init__.py", line 166, in main ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__ return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_callers.py", line 60, in _multicall return outcome.get_result() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result raise ex[1].with_traceback(ex[2]) File "/usr/local/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall res = hook_impl.function(*args) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/main.py", line 316, in pytest_cmdline_main return wrap_session(config, _main) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/main.py", line 311, in wrap_session config._ensure_unconfigure() File "/usr/local/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1055, in _ensure_unconfigure fin() File "/usr/local/lib/python3.11/site-packages/_pytest/capture.py", line 755, in stop_global_capturing self._global_capturing.pop_outerr_to_orig() File "/usr/local/lib/python3.11/site-packages/_pytest/capture.py", line 637, in pop_outerr_to_orig out, err = self.readouterr() ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/capture.py", line 684, in readouterr out = self.out.snap() if self.out else "" ^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/_pytest/capture.py", line 570, in snap self.tmpfile.truncate() FileNotFoundError: [Errno 2] No such file or directory ```

browniebroke commented 1 year ago

@fraserdominicdavid pytest works fine for me on a vanilla project generated from the fork. Are you sure you haven't missed anything from the diff?

fraserdominicdavid commented 1 year ago

@browniebroke sorry for the late reply. As I recall, I have applied everything from the diff. However, I have fixed my issue now. I just followed what @masavini suggested to fix @usr3's error, that is, deleting the following lines from mounts:

{
    "source": "~/.ssh",
    "target": "/tmp",
    "type": "bind"
},

My machine is on Windows 10, so I'm not sure if this particular issue only happens on Windows machines.

quroom commented 5 months ago

@MBARIMike May I ask you share your setting?

cookiecutter / cookiecutter-django