Dev container and documentation for GitHub Codespaces

[samruddhikhandale]

Hi 👋

I am Samruddhi, a Software Engineer for GitHub, working with the GitHub Codespaces team, and a maintainer for the devcontainers org 👩‍💻

GitHub Codespaces is a cloud-based development environment that uses your dev container to provide great contributor and maintainer experiences for all users of your project 🪄

🙅‍♀️ No one would ever have to worry about setting up their local machines, adding tools/runtimes for a specific project (which most frequently contradicts with their current environment) and spend hours before even starting their actual work 🙅‍♀️

Would you be willing to accept a dev container configuration and documentation to improve the GitHub Codespaces experience? 💁‍♀️

Thanks, Samruddhi

// cc @craiglpeters

[dprotaso]

Hey thanks for opening the issue, I'm curious to get feedback from our community on whether they would find a .devcontainer config file valuable.

Some quick observations/questions from my usage

.devcontainers.json

1. What's the cost to maintain them?

We currently have our Prow CI image and have various reusable actions in knative/actions. How can we consolidate the configuration and setup of all these similar but unique tooling. For example - it looks like there's a unique way to install the aws cli using devcontainer features vs. how you accomplish that in GitHub actions.

I just don't want an extra thing to update when we bump go versions etc.

2. devfile

https://devfile.io/ seems to be the CNCF standard for this kind of stuff - what's your take here? Do you support this format too?

Codespaces

My experience with codespaces is a bit mixed since a non-active codespaces consume storage quota and it would run out in a week or so. Unsure if this is because of all the go modules that needed to be in the cache when building with ko. I ended up having to make a help issue to figure out what was going on - https://support.github.com/ticket/personal/0/1917393

Given that what options do you have for contributors beyond running VSCode locally with Docker.

[psschwei]

If folks find it useful and it's low effort to maintain, then I don't have an objection to it (even though personally I'm not a fan).

[samruddhikhandale]

Thanks for your response! ✨

1. What's the cost to maintain them?

We could use the Prow CI image (if necessary) to build the dev container. Besides that take advantage of dev container Features to install tooling, else add the tools using a Dockefile.

Dev config is low maintenance, Features/images are pinned to major versions and by default latest versions of toolings can be installed wherever possible (which avoids manual version bumps). I don't expect updates to dev config unless there are project setup changes/updates.

2. devfile

Interesting, I wasn't aware that devfiles are a standard for CNCF projects. I have recently worked with other CNCF projects (like kubernetes, containerd, thanos, and etcd) for contributing dev container configs, (which surprisingly didn't give me the same impression). I would be interested in learning about it, @dprotaso do you mind pointing me to docs?

Unfortunately, Codespaces doesn't support devfiles.

I ended up having to make a help issue to figure out what was going on - https://support.github.com/ticket/personal/0/1917393

Apologies for having storage issues, I could take a look at the ticket and see what I could find.

Given that what options do you have for contributors beyond running VSCode locally with Docker.

Dev container is an open spec which is supported by these supporting tools (eg. GitHub Codespaces, VS Code Dev containers extension etc)

[craiglpeters]

Given that what options do you have for contributors beyond running VSCode locally with Docker.

Dev container is an open spec which is supported by these supporting tools (eg. GitHub Codespaces, VS Code Dev containers extension etc)

Additionally, the dev container can be run locally using the dev containers cli https://github.com/devcontainers/cli. The VS Code Dev Containers extension allows you to run, connect to, and manage the containers within VS Code on your local machine.

[psschwei]

Do devcontainers only work with VS Code, or is there integration with other editors / IDEs as well?

[craiglpeters]

@psschwei dev containers implementors are listed here https://containers.dev/supporting

[samruddhikhandale]

I am currently setting up dev container configuration by referring to the DEVELOPMENT.md#install-requirements section. The test/README.md#presubmit-tests are passing with it.

For running end to end tests and conformance tests, looks like it needs some manual environment setup (setting up knative serving cluster, docker hub, etc), still taking a look and trying to understand that bit 👀

My question, I'd like to understand how to measure if the dev config is appropriate for this serving repo. Would being able to run the tests be sufficient?

Any pointers would be helpful and appreciated, thank you!

[dprotaso]

Following up from the discussion between Productivity <=> GitHub last week. We're going to run this as an experiment utilizing Google Summer of Code contributors as the test cohort.

In terms of what would be good to have in the .devcontainer image

Our CI uses the following Docker Image - https://gcr.io/knative-tests/test-infra/prow-tests

It would be good for the devcontainer to just leverage that image to start - maybe just use the latest tag. Is there a way to keep the devcontainer up to date when changes are made to that image?

It would be great if kind clusters could be started in the container and being able to install serving.

Similar to what we do in our e2e action - https://github.com/knative/serving/blob/main/.github/workflows/kind-e2e.yaml

[samruddhikhandale]

Thank you @dprotaso for the context, appreciate it.

I have opened a PR https://github.com/knative/serving/pull/13838 and tested creation of local container registry, kind clusters, and deployment of serving and ingress with this setup script in a codespace. I can add it to the postCreateCommand if needed.

It would be good for the devcontainer to just leverage that image to start - maybe just use the latest tag.

Thanks, yes using the prow-tests image as base, added a Go Feature (added some thoughts here), and added configuration for docker startup.

Is there a way to keep the devcontainer up to date when changes are made to that image?

Unfortunately, one needs to create a new codespace or rebuild the current one to fetch the latest image.

[samruddhikhandale]

Closing as PR is merged, thank you!

[dprotaso]

Some feedback rolled in from https://cloud-native.slack.com/archives/C04LMU0AX60/p1680819416766089

I was trying to setup local dev environment using devcontainer config which was recently merged . I ran into issues both in local (kind fails) and in codespaces ( high mem usage ). So was wondering if there is any support from GitHub or we should fix by ourselves.

I asked that they follow up in this issue

[samruddhikhandale]

I ran into issues both in local (kind fails)

Looks like the prow-test image is not supported for ARM architecture, hence, testing it on a windows machine.

and in codespaces ( high mem usage ).

For a 2 cores codespace, you will get the high CPU usage notification as the setup script needs a powerful machine. However, if you ignore that message, the script continues to run on its own pace. Also, you can reload the codespace if you feel like some step is getting stuck.

You can also switch to higher cores to avoid that issue.

[samruddhikhandale]

On a windows machine, I tried the dev container with the Dev Containers extension, it built fine for me (no errors).

Can I take a look at the creation logs to identify the issue?

[naveenrajm7]

Hi Thanks for your answers.

In codespaces , due to high memory usage the kubectl verify commands in the script fails. Re-running the scripts manually will eventually be complete. But not able get it working for example service maybe due to resource constrains. Can you share what codespace spec you used ? . Mine was default 2 core CPU, 3 GB Memory.

In local devcontainer setup , I run into this kind issue

 ✗ Preparing nodes 📦 📦
ERROR: failed to create cluster: could not find a line that matches "Reached target .*Multi-User System.*"

I am not sure if its my own setup issue.

[samruddhikhandale]

In codespaces , due to high memory usage the kubectl verify commands in the script fails. Re-running the scripts manually will eventually be complete. But not able get it working for example service maybe due to resource constrains. Can you share what codespace spec you used ? . Mine was default 2 core CPU, 3 GB Memory.

That's strange, I am able to get the setup script to execute successfully with 2 cores; 4GB RAM ; 32 GB codespace

In local devcontainer setup , I run into this kind issue

Looks like something is interfering with your local config.

[dprotaso]

@samruddhikhandale So the image pull and unpacking is pretty slow when setting up the codespace - how do we take advantage of prebuilds?

[dprotaso]

Also looks like the post create script is not working

Error: error processing import paths in "config/core/999-cache.yaml": error resolving image references: found strict reference but ko://knative.dev/serving/cmd/queue is not a valid import path: error loading package from knative.dev/serving/cmd/queue: err: go resolves to executable in current directory (./go): stderr: 2023/04/24 20:53:02 error during command execution:error processing import paths in "config/core/999-cache.yaml": error resolving image references: found strict reference but ko://knative.dev/serving/cmd/queue is not a valid import path: error loading package from knative.dev/serving/cmd/queue: err: go resolves to executable in current directory (./go): stderr:

The post create script &ko command should run in the root of repo

[samruddhikhandale]

@samruddhikhandale So the image pull and unpacking is pretty slow when setting up the codespace - how do we take advantage of prebuilds?

https://docs.github.com/en/codespaces/prebuilding-your-codespaces provides information on setting up prebuilds. Let me know if that helps.

The post create script &ko command should run in the root of repo

Taking a look 👀 I wonder why this started to fail now, everything was working fine when the PR was merged 🤔

[dprotaso]

If I setup pre-builds in knative/serving will it work when people fork the main repo?

[samruddhikhandale]

If I setup pre-builds in knative/serving will it work when people fork the main repo?

Unfortunately, no. Pre-builds are helpful for folks who create a codespace with the parent repo with write permissions

[dprotaso]

Unfortunately, no. Pre-builds are helpful for folks who create a codespace with the parent repo with write permissions

Hmm.. that doesn't really work for our community since most people will be working in forks.

[aliok]

Related: https://github.com/knative/community/issues/1295