Add devcontainer.json for development with VS Code in a Docker container

mkoeppe commented 2 years ago

https://code.visualstudio.com/docs/remote/containers

We add two new sections in the developer's guide: https://47f8b984ef964a5aa34147393fbdc32e0dde88ad--sagemath-tobias.netlify.app/developer/portability_testing.html#using-our-pre-built-docker-images-published-on-ghcr-io

We also set up devcontainer configurations for the CoCalc and computop/sage Docker images, as well as downstream distribution packaging of Sage.

https://github.com/3-manifolds/sagedocker/issues/2

Tested devcontainer.json of

portability-ubuntu-jammy-standard: builds well; runs well
develop-docker-computop: builds well; runs well
downstream-docker-cocalc: builds well; runs well (except machines with the issue #32434)
downstream-docker-computop: builds well; runs well
downstream-archlinux-latest: builds well; runs well
downstream-conda-forge-latest: builds well; runs well

Follow-ups:

34363 Installation guide: Document installation on Windows via devcontainer
34286 Cookiecutter for Sage user projects with devcontainer
34403 GitHub Codespaces - may also be able to test it by starting a codespace at https:https://user-images.githubusercontent.com/5037600/216875765-6697bbf8-70f9-4acc-868e-9309db67be92.jpg/new?hide_repo_select=true&ref=public%2Fbuild%2Fdevcontainer&repo=302607352&machine=standardLinux32gb&devcontainer_path=.devcontainer%2Fdevcontainer.json
downstream-* for more distributions (generate with tox from the info in tox.ini, use_sagemath` package)
develop-conda-forge-src-environment-dev (https://doc.sagemath.org/html/en/installation/conda.html#using-conda-to-provide-all-dependencies-for-the-sage-library-experimental, similar to our gitpod configuration)
Restore portability-debian-buster-i386-standard and fix it
Restore develop-docker-cocalc (was: builds well (after increasing disk image space on the Docker daemon); fails to run)
Restore - portability-centos-7-devtoolset-gcc_11-standard (was: builds well; fails to run (exactly the same on WSL))
Restore downstream-docker-sagemath (after #34242)
add github workflows that verify that sage is at least compiling in these containers. That should be relatively straightforward using https://github.com/devcontainers/ci
improve portability-Dockerfile as explained in comment:299

Depends on #33873 Depends on #34352

CC: @tobiasdiez @dimpase @williamstein @culler @saraedum @kwankyu

Component: user interface

Author: Tobias Diez, Matthias Koeppe, Kwankyu Lee

Branch/Commit: 4affef2

Reviewer: Kwankyu Lee, Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/33671

mkoeppe commented 2 years ago

comment:71

like this?

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from bcf4302 to e0ae39f

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

e0ae39f src/doc/en/developer/portability_testing.rst: Update

mkoeppe commented 2 years ago

comment:73

Do you have access to Codespaces? This appears to be a paid service.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from e0ae39f to 8befa1c

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

8befa1c src/doc/en/developer/portability_testing.rst: Add link to [GitHub](../wiki/GitHub) devcontainer doc

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 8befa1c to 16d925d

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

16d925d src/doc/en/developer/portability_testing.rst: Fix rst

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

1712a3e .devcontainer/*/devcontainer.json: Use 'make configure' instead of './bootstrap' to match instructions in README

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 16d925d to 1712a3e

tobiasdiez commented 2 years ago

comment:78

Yes, I have access to codespaces as I've been in the beta program. I think you can also enable them on the sagemath org (with a billing limit of say 1 dollar).

It's partly working. Only archlinux-latest-downstream, conda-forge-latest-downstream and ubuntu-jammy-standard are shown in the dropdown box. Any idea why? Moreover, there is a "Default codespace configuration" which is the default github image. If I read the docs about multiple devcontainer files correctly, I think one needs a ".devcontainer/devcontainer.json" file that serves as the default config (I guess that would be ubuntu-jammy-standard in our case). I would also suggest to prefix the other devcontainer names by their function (portability-testing?) since they are not really meant as a platform for development, right?

mkoeppe commented 2 years ago

comment:79

Replying to @tobiasdiez:

I think one needs a ".devcontainer/devcontainer.json" file that serves as the default config

I have not created this file in the repository because in VS Code users have to edit this file to make changes - and then the worktree would be dirty.

mkoeppe commented 2 years ago

comment:80

Replying to @tobiasdiez:

I would also suggest to prefix the other devcontainer names by their function (portability-testing?) since they are not really meant as a platform for development, right?

Good idea, will do

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 1712a3e to 73110b6

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

73110b6 .devcontainer: Use prefixes 'downstream-', 'portability-'

tobiasdiez commented 2 years ago

comment:82

Replying to @mkoeppe:

Replying to @tobiasdiez:

I think one needs a ".devcontainer/devcontainer.json" file that serves as the default config

I have not created this file in the repository because in VS Code users have to edit this file to make changes - and then the worktree would be dirty.

Since the idea of the dev container is to provide a container for the development, I think this should be the main and default config. If someone temporarily wants to investigate some issues using one of the other images, then its okay to have an uncommited change dangling around. I would also suggest you open an issue at the vscode repo to support multiple devcontainers.

mkoeppe commented 2 years ago

comment:83

Replying to @tobiasdiez:

Since the idea of the dev container is to provide a container for the development, I think this should be the main and default config.

But I would think most people who want to use VS Code will want to do dev locally, not in the container. It's pretty annoying to be asked to reopen it in the devcontainer.

mkoeppe commented 2 years ago

comment:84

Replying to @tobiasdiez:

If someone temporarily wants to investigate some issues using one of the other images, then its okay to have an uncommited change dangling around.

No, I wouldn't like that

tobiasdiez commented 2 years ago

comment:85

Replying to @mkoeppe:

Replying to @tobiasdiez:

Since the idea of the dev container is to provide a container for the development, I think this should be the main and default config.

But I would think most people who want to use VS Code will want to do dev locally, not in the container. It's pretty annoying to be asked to reopen it in the devcontainer.

If you have the remote container extension installed, then its fair to assume you may also want to use it.

By the way, please remove the extension from the recommended list. I think its to situational to recommend to everyone.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 73110b6 to 23d6223

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

23d6223 Revert ".vscode/extensions.json: Recommend extension 'Remote - Containers'"

mkoeppe commented 2 years ago

comment:87

Replying to @tobiasdiez:

If you have the remote container extension installed, then its fair to assume you may also want to use it.

Hardly. What extension is installed is not specific to a project.

tobiasdiez commented 2 years ago

comment:88

Replying to @mkoeppe:

Replying to @tobiasdiez:

If you have the remote container extension installed, then its fair to assume you may also want to use it.

Hardly. What extension is installed is not specific to a project.

Yes it is, you can disable/enable extensions for a specific workspace. https://code.visualstudio.com/docs/editor/extension-marketplace#_disable-an-extension

mkoeppe commented 2 years ago

comment:89

Replying to @tobiasdiez:

you can disable/enable extensions for a specific workspace. https://code.visualstudio.com/docs/editor/extension-marketplace#_disable-an-extension

Thanks, that's good to know

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 23d6223 to a8a902d

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

`93fabf8`	`.devcontainer/devcontainer-archlinux-latest-downstream.json: Install pip, notebook`
`928cd43`	`.devcontainer/devcontainer-conda-forge-latest-downstream.json: New`
`b11ae77`	`.devcontainer/post_create.sh: Simplify using new options of sage-print-system-package-command`
`33dc4eb`	`.devcontainer/devcontainer-archlinux-latest-downstream.json: Use spkg _sagemath`
`c9e52c5`	`.devcontainer: Move devcontainer.json files to separate directories for Codespaces`
`863e24d`	`src/doc/en/developer/portability_testing.rst: Update`
`0fae3d8`	`src/doc/en/developer/portability_testing.rst: Add link to [GitHub](../wiki/GitHub) devcontainer doc`
`98204ae`	`src/doc/en/developer/portability_testing.rst: Fix rst`
`07e8495`	`.devcontainer/*/devcontainer.json: Use 'make configure' instead of './bootstrap' to match instructions in README`
`a8a902d`	`.devcontainer: Use prefixes 'downstream-', 'portability-'`

mkoeppe commented 2 years ago

Changed reviewer from Tobias Diez (-1), ... to none

mkoeppe commented 2 years ago

Changed dependencies from #33851, #33786, #33870, #33873 to #33786, #33870, #33873

mkoeppe commented 2 years ago

Description changed:

--- 
+++ 
@@ -5,6 +5,7 @@

 Users who have access to GitHub Codespaces may also be able to test it by starting a codespace at https://github.com/codespaces/new?hide_repo_select=true&ref=public%2Fbuild%2Fdevcontainer&repo=302607352&machine=standardLinux32gb&devcontainer_path=.devcontainer%2Fdevcontainer.json

-We also set up devcontainer configurations for the CoCalc,  computop/sage, and sagemath/sagemath Docker images.
+We also set up devcontainer configurations for the CoCalc,  computop/sage, and sagemath/sagemath Docker images, as well as downstream distribution packaging of Sage.
+
 - https://github.com/3-manifolds/sagedocker/issues/2

mkoeppe commented 2 years ago

Changed dependencies from #33786, #33870, #33873 to none

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

`5380a51`	`.devcontainer/devcontainer-archlinux-latest-downstream.json: Install pip, notebook`
`ece0fde`	`.devcontainer/devcontainer-conda-forge-latest-downstream.json: New`
`9772b1c`	`.devcontainer/post_create.sh: Simplify using new options of sage-print-system-package-command`
`b44280c`	`.devcontainer/devcontainer-archlinux-latest-downstream.json: Use spkg _sagemath`
`64ad9d0`	`.devcontainer: Move devcontainer.json files to separate directories for Codespaces`
`a813a75`	`src/doc/en/developer/portability_testing.rst: Update`
`60e58d3`	`src/doc/en/developer/portability_testing.rst: Add link to [GitHub](../wiki/GitHub) devcontainer doc`
`6137928`	`src/doc/en/developer/portability_testing.rst: Fix rst`
`0b9fc68`	`.devcontainer/*/devcontainer.json: Use 'make configure' instead of './bootstrap' to match instructions in README`
`1792288`	`.devcontainer: Use prefixes 'downstream-', 'portability-'`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from a8a902d to 1792288

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 1792288 to b828e55

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

b828e55 Remove use of _develop package

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from b828e55 to 425f51f

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`ab5a4f1`	`build/bin/sage-get-system-packages: If system=auto, use sage-guess-package-system`
`798f4d3`	`build/bin/sage-print-system-package-command: If system=auto, use sage-guess-package-system`
`ef249cc`	`build/bin/sage-print-system-package-command: New option --spkg`
`d75da7e`	`tox.ini: Simplify using new options of sage-print-system-package-command`
`425f51f`	`Merge #33873`

mkoeppe commented 2 years ago

Dependencies: #33873

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

e12ca6c .devcontainer: Install git

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 425f51f to e12ca6c

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from e12ca6c to 6a6ce78

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

6a6ce78 src/doc/en/developer/portability_testing.rst: Fix markup

mkoeppe commented 2 years ago

comment:100

Here's a version with fewer dependencies.

tobiasdiez commented 2 years ago

comment:101

Attachment: codespaces.jpg

It's still not working on github's workspaces (see attached screenshot). The default config (which is used if you use the direct link without manual config) just starts github's default (= almost empty) workspace, and thus is not useful for sage development. Moreover, only a few of the added dev containers are actually appearing in the dropdown menu (maybe some parsing issues?).

Codespaces are now included in the educational program of github, so you might be able to test it yourself.

Moreover, some of the container names are a bit confusing (e.g. sagemath-sagemath). Could this be improved? In either case, I would welcome if each one is shortly explained in the docs.

mkoeppe commented 2 years ago

comment:102

I still don't have access to codespaces. So I'd say let's defer codespaces to a follow up (unless you have time to work on it for this ticket).

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 6a6ce78 to 029c9a0

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

029c9a0 .devcontainer/post_create.sh: Install python3 so that python3-venv is available on Debian

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 029c9a0 to 23b4cc5

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

23b4cc5 Merge tag '9.7.beta7' into t/33671/public/build/devcontainer

mkoeppe commented 2 years ago

Description changed:

--- 
+++ 
@@ -9,3 +9,6 @@

 - https://github.com/3-manifolds/sagedocker/issues/2

+See also:
+
+- #34286 Cookiecutter for Sage user projects with devcontainer

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

d798acc Merge branch 'develop' into t/33671/public/build/devcontainer

sagemath / sage

Add devcontainer.json for development with VS Code in a Docker container #33671

34363 Installation guide: Document installation on Windows via devcontainer

34286 Cookiecutter for Sage user projects with devcontainer