Epic: Support `devcontainer.json`

[svenefftinge]

Given VS Code's and GitHub's dominance in the developer world, VS Code's devcontainer.json format is going to become a quasi-standard eventually. Supporting it will allow us to lower the entry barrier for teams who already use it so they can easily spin up their dev environments in Gitpod and experience the speed (startup time, prebuilds) and efficiency (lower cost) of our product.

[ntziolis]

This would be a welcome addition, we currently make heavy use of the docker-compose based vscode devcontainer feature. And had to adjust our setups significantly to ensure we don't have duplication between lokal devcontainer and gitpod setups. This wasn't straight forward as in gitpod docker-compose needs to be called separately from starting the development container where in vscode the devcontainer is part of the docker compose file.

We are happy to provide some insights how we have ensured that we are sharing the same compose file across vscode and gitpod. What we could not circumvent is having seperate Dockerfiles for the devcontainers as vscode and gitpod use different base images. To reduce duplication we have extracted the setup steps into scripts that are copied into and executed during image creation of the devcontainer. That said I think there is room for tooling that does this for you.

Note: Our devs were very clear that while they love gitpod they absolutely wane be able to start a project in a local devcontainer as well if needed.

[jeluard]

@svenefftinge Would this be an openvscode-server level feature of gitpod one?

[jorroll]

Just to pile on, one benefit of the .devcontainer.json specification is that it operates the same way locally as in other places (e.g. Gitpod). I've used devcontainers for a while now (years? not sure) because I find them easier than tools like nvm and I like being able to provide a convenient onboarding experience for my open source projects. I'm just checking out Gitpod (for the second time, actually) and looking at the .gitpod.yml format (which seems nice as well) but I realize that, in order to port my devcontainer.json config over to the gitpod format, it doesn't look like I can easily test .gitpod.yml locally in VS Code. It looks like I'll need to try a config, spin up a gitpod instance, tweak settings, spin up another gitpod instance, etc. Not great.

As a new user to Gitpod, I don't expect I'll want to jump whole-hog into coding online. I expect my default will still be just coding on my personal machine inside a devcontainer. I like the option of being able to use Gitpod when on someone else's computer though (such as a work computer) which happens somewhat often (it also seems nice for my open source projects). Because of this, having one standard config that works the same across environments is definitely a feature I appreciate.

Edit (2/5/22): FWIW, after running into this issue I ended up losing interest in Gitpod and not pursuing it further. Never ended up getting .gitpod.yml working.

[CarlosDomingues]

Our company has been using .devcontainer / .devcontainer.json for a few years so we can have reproducible dev environments.

If we could use those with Gitpod, migration would be trivial.

Also, as @jorroll mentioned, the fact that .devcontainer.json also works locally makes it very powerful for testing dev environments in fast iterations.

Maybe it's because I'm still learning, but I've found the process of trying stuff in .gitpod.yml quite slow and time consuming.

[bigint]

I need this badly 🙏🏻 after seeing the today's GH universe 😄

[ntindle]

+1 on this, my projects are tied to .devcontainer format

[sdepablos]

Another +1, it would make the transition to Gitpod way easier

[0Grit]

Heck, even if I still had to specify in .gitpod.yml which .devcontainer.json to use. I don't mind reminding myself whose product I'm using.

Be nice if there was some open governed group maintaining standards like these for devtools...

[svenefftinge]

See https://github.com/gitpod-io/gitpod/issues/6922

[liflovs]

Nice that you guys already scheduled it, I`m introducing devcontainers into our team's culture, but its still a task to set them up and make sure they work stable. Having gitpod support it will allow to boost cooperation with our short-time freelance devs.

[jessitron]

As a new user, it completely shocked me that gitpod opens VSCode but does not use VSCode devcontainer configuration. It took me a lot of googling until I found this issue, and then I was like "What??"

This may be a showstopper, if pointing gitpod.yml to .devcontainer/Dockerfile doesn't work. (edit: it did)

[blieusong]

Exactly as per @jessitron jessitron. Impressed at first, but felt let down next, after realising that nothing in the .devcontainer folder of the project wasn't taken into account.

It probably is best not to have too many files format to do the same thing, and I'm afraid devcontainer will become a de facto standard instead of the .gitpod.yml, so might as well focus on that.

[Chuxel]

Hey there! Just wanted to mention that we're in the process of opening up the dev container spec to be like the VS Code LSP. We plan to house the spec at https://github.com/microsoft/dev-container-spec and are looking into the release of a reference implementation. We're still early in the process, but we're happy to collaborate on the spec even before all of this is done - there's already a few things we plan to do like providing ways for services and other tools to add their own metadata into the format you'll see mentioned in this repository (and we'll also be moving the two VS Code specific properties we have under a tool specific property to follow suit). I'm sure that aspect would be useful to GitPod among other things.

[andreafalzetti]

We have now made some small progress towards this. During a Gitpod hackathon we've created this branch.

There's a lot of clean-up required. We commented out useful code during our hack which needs to be restored. Overall, we feel that we would like to continue with this piece of work as we could be very close to shipping an initial working solution.

The current approach consists in modifying the server component. The cleaner version would still be a change in the server component but specifically to the config-inferrer code.

Also, we discussed with @filiptronicek that we could have the converter utility abstracted out from the code of the server, so that it could also be used independently (e.g. using npx such as npx devcontainer-to-gitpod <path>). I suggest we write the converter so that it's easy to be published as a node module but wait for a use-case before actually publishing it.

cc @mads-hartmann

[jwpjrdev]

I'm just checking out Gitpod (for the second time, actually) and looking at the .gitpod.yml format (which seems nice as well) but I realize that, in order to port my devcontainer.json config over to the gitpod format, it doesn't look like I can easily test .gitpod.yml locally in VS Code. It looks like I'll need to try a config, spin up a gitpod instance, tweak settings, spin up another gitpod instance, etc. Not great.

I agree with this, it's quite a painstaking process to test .gitpod.yml and .gitpod.Dockerfile changes. Perhaps some sort of hot reload feature could be provided to simply refresh the current workspace?

[andreafalzetti]

I agree with this, it's quite a painstaking process to test .gitpod.yml and .gitpod.Dockerfile changes. Perhaps some sort of hot reload feature could be provided to simply refresh the current workspace?

This is definitely on our radar, we have captured it in https://github.com/gitpod-io/gitpod/issues/7671. It's something that I personally cannot wait to start working on; because the value that brings is huge.

[jwpjrdev]

It would also be useful to store .gitpod.yml and .gitpod.Dockerfile in /.devcontainer. Here's an example of what this directory stores: https://github.com/github/docs/tree/main/.devcontainer

[bamurtaugh]

Hi all 👋! I wanted to provide some updates on the dev container specification.

We recently released the reference implementation as an open source CLI, in addition to an initial version of the spec. We now have a devcontainers org for these repos and a site where you can read more about this effort: https://containers.dev/.

As @Chuxel mentioned, we also introduced a customizations property in which tool-specific properties live (additional info).

The CLI and spec are both in active development, so they'll continue to evolve, especially with external feedback. We'd love any feedback or questions you may have, and we're more than happy to collaborate on things that'd make this a better fit for you. Thanks!

[jwpjrdev]

@bamurtaugh thanks for the hard work!!

[shaal]

I tested devcontainer locally on my machine and in Gitpod.

Locally, I used the Remote Containers VSCode extension. On a repo with devcontainer.json it automatically suggests opening the dev environment in the container. But when I tried doing the same thing in Gitpod, I noticed that the extension is not available in Open VSX

@Chuxel @bamurtaugh Is there a plan to publish that extension in Open VSX so it can be used in Gitpod?

[bamurtaugh]

Thanks @jwpjrdev and @shaal for the feedback!

@shaal You don't need to use the Remote-Containers extension to use the dev container CLI - there is guidance here. While Remote-Containers won't be published to Open VSX, you'll still be able to use dev containers outside of VS Code as the CLI is provided through an npm package, and other editors like Gitpod could add tooling around the CLI to support it (as this issue describes supporting devcontainer.json).

[glebbash]

For anyone interested in using both GitPod and GitHub Codespaces on the same repository you can check out how I did it: https://github.com/glebbash/lole-lisp.

This includes creating a shared Dockerfile with the needed tools (LLVM in this case), and separate Dockerfiles for GitPod and Codespaces that use the shared one. And then adding vscode configs in .gitpod.yml and devcontainer.json files, which unfortunately need to have extensions definition duplicated.

This also highlights one problem that needs to be accounted for if GitPod will ever support devcontainer.json: it will need to have a different base Dockerfile to be compatible with the one from Codespaces. Or even better someone should create platform agnostic base image which can be extended for both GitPod and Codespaces.

[shaal]

@glebbash what do you mean by "a different base Dockerfile"? I think the the current Dockerfile doesn't have to be "gitpod-specific", but I do not know Codespaces, and what are its requirements

[glebbash]

I am not sure about specifics. I know that user names differ in the images and I guess Codespaces base image has some git or ssh config and GitPod's image at least includes GitPod cli.

But yeah, I just saw that base images are different and I didn't bother to create GitPod workspace with Codespaces image as it would probably break something.

[Chuxel]

@glebbash Just as a FYI - there's nothing in the dev container spec or Codespaces that requires particular image contents. You can use the base debian image if you prefer. It just is missing most of the common utilities developers expect to be present.

[purkhusid]

Has a decision been made on if Gitpod will support devcontainer.json instead of gitpod.yml?

[glebbash]

@Chuxel good to know that, will definitely need to try a setup with single Dockerfile for both GitPod and Codespaces.

[Craftzman7]

Is this confirmed to be on the roadmap?

[davidwallacejackson]

Just here to be another squeaky wheel -- I'm doing devex for my team, and we have a good mix of VS Code and JetBrains developers. It would be really sweet to have a first-class way to share a containerized development environment between both, and it seems like there's potential in the developing devcontainer spec to encode what both tools would need.

The hard thing about using e.g. a shared Dockerfile is that the devcontainer CLI actually generates a Dockerfile when the image is built. It does this, as far as I understand it, to support features, so that support for e.g. languages can be shared in a modular way. So if I make a devcontainer.json and use features to add support for, say, Go, NodeJS, and Python, I have no way of sharing that with a GitPod container -- even if they're both using the same base Dockerfile.

[jwpjrdev]

quite honestly i do think it would be smart to deprecate the .gitpod.yml file and just adopt devcontainer.json only. it would result in clearly organised projects that work across architectures and would definitely improve the devx of maintaining infra between environments. it's a bit of a radical change and would have to be rolled out over a long period of time (and would require still supporting .gitpod.yml for a long time to come! although i'm sure that notices would be plastered everywhere until then).

i feel like .gitpod.yml is a lesser-used (only in gitpod!) format and if we follow the ideology that "it's the better format and devcontainer sucks so we have this better thing," it'll just become yet another standard that just gets lost in the sea of environment configuration files. it's best to just adopt the existing, more widely used format. you know the xkcd. if someone could help me out, please list some things that .gitpod.yml does that devcontainer.json can't do. i'd like a definitive list so others can see what i mean by this.

[Coriago]

Please support this feature. The spec is open source and already building in room to support other platforms. I don't want to maintain multiple standards and risk having inconsistent environments because one spec lacks features. After all, isn't consistency the whole point of doing dev containers?

[glebbash]

With Codespaces now having free hours and devcontainer features it's probably time to take devcontainer spec seriously.

[ThePlenkov]

Yes - I already see many projects where things are just maintained parallelly to support Gitpod and Codespaces/VS Code. Very often those environments become different. So it's even more annoying. Now Docker team is trying to introduce their own way to open projects locally. To be honest - this idea is great and it's a good alternative to Gitpod/Codespaces if you want to open project locally: But again the issue stays same - they require their own compose-dev.yaml file which makes things even more complicated. I think it's a right time that in 2023 communty has to come to a single format and support it together. Development containers became a dedicated spec https://containers.dev/ so why not join altogether and not consolidate our efforts to build the best dev experience ever? With ability to open same project with same set up in doesn't matter where (Gitpod, Codespaces, Docker Desktop, VS Code) - it will be using same files and running things equally. And to use files like gitpod.yml only for additional value, exlusive features not available in the open spec.

[Karreg]

True. With the traction CodeSpace is gaining, not supporting devcontainer would not be a wise move for GitPod... I still prefer GitPod, because I'm more GitLab than GitHub, but I don't think I will continue to manager gitpod and devcontainer in all my projects for long.

[nathancarter]

Can't emphasize this enough:

Devs come to GitPod already familiar with developing in containers, and having heard your marketing that GitPod lets you do that in the cloud. Then they open their repo and the first thing they find out is that you've ignored their development container definition. (Wait, wasn't containerized dev...the whole...wait, what?) This is truly a very significant negative experience that makes those potential new users feel like you've betrayed your explicitly advertised raison d'etre before they've typed a single line of code. This should be on your "must fix and how did we let this happen in the first place" list.

Between now and when it's fixed, you could at least add a notification when a user opens a repo with a .devcontainer folder that GitPod is ignoring, and say, "Hey, we just noticed you have a VS Code .devcontainer folder. We don't support that yet, but until we do, check out these docs to help you convert your container definition to our format..." That would tell those users that their expectation is both reasonable and on your roadmap, and give you a greater chance of keeping them on your platform while they wait. Right now they just go, "Well, that sucks" and leave.

The 10-15 people who posted comments up above about their experience are the small subset from that larger group who chose to kindly give you their feedback before going back to whatever dev env they used before.

[enricopolanski]

Can't second this enough.

Not supporting devcontainer definitions just makes it impossible for me to move out of GitHub's codespaces.

[ghuntley]

With Codespaces now having free hours and devcontainer features it's probably time to take devcontainer spec seriously.

🆙 x 💯

not supporting devcontainer would not be a wise move for GitPod

🆙 x 💯

Can't second this enough.

🆙 x 💯

[loujaybee]

Thank you all for the comments, context and information ! 👋

A note on updates to this issue

For avoidance of doubt: This is currently the most upvoted issue in our issue tracker (we are very aware!). Infrequent updates to this issue is not a signal that it is "ignored" by Gitpod. Your additional context + comments are useful. GitHub issues do not reflect internal discussions, investigations, or conversations etc.

What type of support are you after?

From the comments I can summarise the following broad reasons for the demand in devcontainer support. Until we have further updates here, a signal of the areas most important to you would be very useful to us:

👀 - Migrating projects to Gitpod (Gitpod for a project that already uses devcontainer) 😄 - Local development fallback support or other features of devcontainer 🚀 - Keeping your devcontainer + gitpod.yml scripts and configuration in sync (e.g. "reduced duplication") 🎉 - Support for alternate locations of configurations, e.g. .devcontainer

Any other use cases not covered above?

If your use case is not adequately covered in these areas, please comment below. The more information you can provide us, e.g. your specific use case, a link to a repo or replication, or concrete examples of what you're trying to achieve, the better. This helps us to identify what "type" of devcontainer support would be most useful for you.

Again, thank you for your patience, hopefully we have more updates for you soon 🙏

For those looking for faster .gitpod.yml configuration updates

For those who commented about iteration speed on .gitpod.yml configurations, please keep an eye on the following issue (as mentioned by @andreafalzetti above) as this is another area of Gitpod that many of you have told us is important to you all. I hope we have more to share with you soon on the work we are doing to improve this "inner loop" aspect of Gitpod.

• https://github.com/gitpod-io/gitpod/issues/7671

[loujaybee]

"Hey, we just noticed you have a VS Code .devcontainer folder. We don't support that yet, but until we do, check out these docs to help you convert your container definition to our format..." - @nathancarter

You make a good point, we should document current approaches of what you can do today (and also in future) to help manage a devcontainer and gitpod.yml setup (link below). Many good ideas have been shared here. A good point to remind is that one of our guiding principles is to "integrate, don't dictate" [1], and we honour this wherever we can.

• https://github.com/gitpod-io/website/issues/3312

[robartsd]

Any other use cases not covered above?

Multiple devcontainer configurations for the same repository.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[TheBestMoshe]

I don't think this issue should be closed.

[potelux]

@loujaybee It has been just over a year since this last comment:

Again, thank you for your patience, hopefully we have more updates for you soon 🙏

Is there anything to update here? Is there somewhere else we can follow the progress/discussions?

[agardnerIT]

Hi all, any update here? I am leading the virtual env. work for public demos at my employer and we want to standardise on Gitpod but I will not do so until Gitpod supports devcontainer.

[mbrevoort]

Hey all, I’m thrilled to share that we’ll be launching support for the devcontainer.json configuration format later this year, along with some additional surprises. 🙂