Kubernetes container

[bhack]

Support for containers running in kubernetes (pod, deployment etc..)

[Chuxel]

@bhack Where are you running Kubernetes? If it's local, you should be able to attach to it after you spin it up. F1 > Remote-Containers: Attach to Running Container...

Would that work for you?

[bhack]

Often it is remote specially when I need special (hardware/data) resources. I need always to use kubectl cp when I edit a file in vscode.

[bhack]

Have you collaboration with https://github.com/Azure/vscode-kubernetes-tools Azure team?

[Chuxel]

@bhack Definatley something we've been thinking about which is partly why I asked about the scenario. We actually have a dev container defintion that allows you to use a Docker sandbox container with all the needed tools (kubctl, helm docker, etc) so you can containerize your build toolchain and still use Kubernetes extension and deploy to a Kubernetes cluster from there: https://github.com/Microsoft/vscode-dev-containers/tree/master/containers/kubernetes-helm

F1 > Create Container Configuration File... > Kubernetes & Helm

[bhack]

F1 > Remote-Containers: Attach to Running Container... for remote K8s could be a good initial feature here.

[bhack]

Just to clarify I meant, as first useful feature, the same use case at (but for K8s): https://code.visualstudio.com/docs/remote/containers#_attaching-to-running-containers

While using VS Code to spin up a new container can be useful in many situations, it may not match your workflow and you may prefer to "attach" VS Code to an already running container.

Cause my K8S remote workflow "may not match" VS Code spin up.

[Chuxel]

@bhack Yep - totally makes sense. Use kubectl to get to the pod instead of the Docker CLI. K8s configs are more complicated to manage in general so starting with an attach flow makes sense.

[bhack]

I've a question about the server side/extensions. Cause the container/pod nature it is ephemeral how do you handle server side extensions over containers? Do you need to re-configure the remote container extensions every time?

[bhack]

Also I don't understand what could be the case between remote vs container for remote-container K8s case: We need to suppose that the running code is in the container (i.e. from init container, persitent volume, remote mount, etc) but remote/local edits need to be in sync with the local/remote files. So there in not a volume mount of a local path like in your current "local" container case. We need to have an auto-sync over the "source code block" in the diagram.

[bhack]

About source code block sync check https://github.com/ernoaapa/kubectl-warp/issues/10. As mentioned there I don't know if a sidecar injector could be a good strategy.

[chrmarti]

Extensions running in the container need to be installed in each container. That happens automatically for the extensions listed in the devcontainer.json.

The files don't have to be local for VS Code to work on them. You could just check out the files in the container.

[bhack]

No the problem is that you work on remote files but generally you git commit changes locally. So we need a sync.

[chrmarti]

If you commit / push through the Git viewlet in VS Code, you do it on the remote side currently.

[bhack]

Generally you don't have git push credentials on remote containers.

[chrmarti]

We currently register a credentials helper for Git in the container (if git is installed) that forwards requests to the local Git to lift that limitation.

[bhack]

@chrmarti I still think that a sync is optionally required over the source code about volume block in the diagram. A very frequent use case is that you test a change on a small scale in a local container (i.e. Docker container) and then you want to test on an active developing Pod (container in k8s). Generally you don't commit until changes are tested locally and on the pod resources. So if the change works locally then you want to sync "code block" on the remote existing developing pod/container. If all works also remotely probably you can commit also with your mentioned helper. So a local to existing pod sync it is required IMHO.

[kdvolder]

@Chuxel said:

Where are you running Kubernetes? If it's local, you should be able to attach to it after you spin it up. F1 > Remote-Containers: Attach to Running Container...

Would that work for you?

If it'd allow me to somehow connect to pods inside my local k8s cluster... yes it would (at least for my current local experimenations). But I don't think that this actually works right now (is it supposed to work already?).

When I use the "Remote-Containers: Attach to Running Container" it shows a list of things that seems to correspond to the stuff you get running docker ps command locally. But the pods running in my k8s cluster (i.e. the things that you get running kubdctl get pods) is not amongst the things it lets me choose from.

So what we need really is what you suggested here:

Yep - totally makes sense. Use kubectl to get to the pod instead of the Docker CLI.

That, by the way, would then also work with any k8s cluster be it local or remote.

[bhack]

Cause this Is still not going in the montly roadmap there Is a boiloperate workaround in the meantime: https://okteto.com/blog/vs-code-remote-development-in-kubernetes/

[mlgill]

Cause this Is still not going in the montly roadmap there Is a boiloperate workaround in the meantime: https://okteto.com/blog/vs-code-remote-development-in-kubernetes/

To clarify, it looks like Oketo has to be installed locally and on Kubernetes and does the equivalent of running syncthing (bi-directional sync) locally and remotely as a sidecar container, correct?

Assuming this is correct, VS Code development works through file sync and it's not possible to use any of the debugging features within the remote container.

I do hope this feature gets put into the roadmap so that it will work similar to remote development over SSH.

[bhack]

@mlgill Yes but It Is better than nothing cause there Is anything planned in the next iteraton https://github.com/microsoft/vscode-remote-release/issues/1390

[mlgill]

I get that. Just want to be clear about the limitations.

[bhack]

@mlgill Cause It Is closed source we cannot do anything more that upvote this. So, if you can, please try to help to collect upvotes here in your GitHub network of vscode users.

[kdvolder]

I experimented a bit with using the ssh support to connect to a k8s container. Was able to get this working but the main drawback of the approach is that you have to explicitly install ssh daemon into your 'development contiainer'. It's also a fairly involved process to set it all up. Anyhow... just in case anyone is interested. I put this up here: https://github.com/kdvolder/vscode-remote-java-dev-env

[kdvolder]

@bhack said

Cause It Is closed source

Is that true? If so, I am very surprised! I assumed otherwise given that we are here, filing issues and discussing on github (which is a platform for open source software development). But perhaps it is true after all since when I go and try to find the source code, indeed I can't seem to find any.

I've filed this ticket https://github.com/microsoft/vscode-remote-release/issues/1402 as I think the github readme and docs for vscode-remote are really not clear on that point.

[bhack]

@kdvolder Yes, see this old thread https://github.com/microsoft/vscode-remote-release/issues/179

[chrmarti]

There are a few FAQs about the license.

[rberrelleza]

Cause this Is still not going in the montly roadmap there Is a boiloperate workaround in the meantime: https://okteto.com/blog/vs-code-remote-development-in-kubernetes/

To clarify, it looks like Oketo has to be installed locally and on Kubernetes and does the equivalent of running syncthing (bi-directional sync) locally and remotely as a sidecar container, correct?

Assuming this is correct, VS Code development works through file sync and it's not possible to use any of the debugging features within the remote container.

I do hope this feature gets put into the roadmap so that it will work similar to remote development over SSH.

@mlgill if you follow the instructions on the blog post that @bhack linked, you'll be able to use all the features of VS Code Remote-SSH, including debugging. You'll also have your development environment running in Kubernetes and with bidirectional file synchronization (this helps if you want to commit from your local machine, instead of forwarding your credentials).

I'm one of the maintainers of github.com/okteto/okteto. We're working on a smother integration with VS Code Remote, your feedback on the scenario and approach would be great.

[mlgill]

Thanks for the reply. Using Oketo requires admin access to the Kubernetes cluster to install a package, correct?

If so, this is not an option for me.

On Thu, Sep 12, 2019, at 01:56, Ramiro Berrelleza wrote:

Cause this Is still not going in the montly roadmap there Is a boiloperate workaround in the meantime: https://okteto.com/blog/vs-code-remote-development-in-kubernetes/

To clarify, it looks like Oketo has to be installed locally and on Kubernetes and does the equivalent of running syncthing (bi-directional sync) locally and remotely as a sidecar container, correct?

Assuming this is correct, VS Code development works through file sync and it's not possible to use any of the debugging features within the remote container.

I do hope this feature gets put into the roadmap so that it will work similar to remote development over SSH.

@mlgill https://github.com/mlgill if you follow the instructions on the blog post https://okteto.com/blog/vs-code-remote-development-in-kubernetes/ that @bhack https://github.com/bhack linked, you'll be able to use all the features of VS Code Remote-SSH, including debugging. You'll also have your development environment running in Kubernetes and with bidirectional file synchronization (this helps if you want to commit from your local machine, instead of forwarding your credentials).

I'm one of the maintainers of github.com/okteto/okteto. We're working on a smother integration with VS Code Remote, your feedback on the scenario and approach would be great.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/microsoft/vscode-remote-release/issues/12?email_source=notifications&email_token=AANNAOK3CQVO5O7O24IR2ALQJHK2VA5CNFSM4HKE3FZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6QX4AY#issuecomment-530677251, or mute the thread https://github.com/notifications/unsubscribe-auth/AANNAOMERQZ6R6XLTHUZDLDQJHK2VANCNFSM4HKE3FZA.

[rberrelleza]

Thanks for the reply. Using Oketo requires admin access to the Kubernetes cluster to install a package, correct? If so, this is not an option for me. … On Thu, Sep 12, 2019, at 01:56, Ramiro Berrelleza wrote: >> Cause this Is still not going in the montly roadmap there Is a boiloperate workaround in the meantime: >> https://okteto.com/blog/vs-code-remote-development-in-kubernetes/ > To clarify, it looks like Oketo has to be installed locally and on Kubernetes and does the equivalent of running syncthing (bi-directional sync) locally and remotely as a sidecar container, correct? > Assuming this is correct, VS Code development works through file sync and it's not possible to use any of the debugging features within the remote container. > I do hope this feature gets put into the roadmap so that it will work similar to remote development over SSH. @mlgill https://github.com/mlgill if you follow the instructions on the blog post https://okteto.com/blog/vs-code-remote-development-in-kubernetes/ that @bhack https://github.com/bhack linked, you'll be able to use all the features of VS Code Remote-SSH, including debugging. You'll also have your development environment running in Kubernetes and with bidirectional file synchronization (this helps if you want to commit from your local machine, instead of forwarding your credentials). I'm one of the maintainers of github.com/okteto/okteto. We're working on a smother integration with VS Code Remote, your feedback on the scenario and approach would be great. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12?email_source=notifications&email_token=AANNAOK3CQVO5O7O24IR2ALQJHK2VA5CNFSM4HKE3FZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6QX4AY#issuecomment-530677251>, or mute the thread https://github.com/notifications/unsubscribe-auth/AANNAOMERQZ6R6XLTHUZDLDQJHK2VANCNFSM4HKE3FZA.

@mlgill You don't need admin permissions to the cluster, just deploy privileges on a namespace. If you can deploy an application to your cluster, you can use Okteto. We have an okteto channel in the Kubernetes slack, we can continue the conversation there 😄 .

[bhack]

@rberrelleza

The dev image includes an SSH server

It Is still quite unmpratical to hack every image but we haven't any other solution currently.

[kwlzn]

is there some reason that the plugin couldn't use e.g. kubectl exec the same way e.g. Remote SSH uses ssh to install and start up the remote vscode process group?

[chrmarti]

There are a few other Docker-specific things we use. (E.g., an NPM package to inspect containers / images and to listen for events.)

[bhack]

There are many NPMs also for Kubernetes like https://www.npmjs.com/package/kubernetes-client

[bhack]

I think that also some code could be reused from the other Microsoft plugin https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools

[bhack]

This issue now is the 2nd most upvoted feature request in the container sub-component/label. I hope it will start to join in the imminent October roadmap cycle ticket.

[vanbroup]

We created a proof of concept ssh proxy for visual studio code that deploys a pod in your selected namespace with a custom docker image (make sure you have wget or curl in your image) on Kubernetes for the authenticated user.

https://github.com/digitorus/remote-vsc-k8s

Please check the repository of you want to give it a try or want to help with contributions. Feedback is welcome!

[bhack]

October iteration plan was just opened: https://github.com/microsoft/vscode-remote-release/issues/1661

I hope we will see something about this in the container section.

[kdvolder]

I hope we will see something about this in the container section.

Agreed. I hope so too.

In the mean time it might be interesting for people who are looking to connect / run a development environment to something deployed in a k8s cluster to consider using Theia instead of vscode. Interesting fact, Theia is fully compatible with vscode extension api. You can drop a vscode .vsix file into Theia and it usually just works (tried it with several of our own vscode extensions with great success in the gitpod version of Theia). Theia also provides a very similar look and feel based on the same Monacco editor as vscode, and even down to the level of keyboard shortcuts it looks and feel like you are using vscode inside a web browser.

So if vscode remote support for k8s does not seem to be coming soon, it is reassuring to know that perhaps Theia might be an alternative way for us to provide a k8s based IDE experience. Our team will start exploring this a bit soon.

[davidwindell]

I also strongly recommend looking at Eclipse Che which provides Their based K8's workspaces.

[pchico83]

We have released a VS Code plugin for remote development in Kubernetes: Release blog post: https://okteto.com/blog/remote-kubernetes-development Marketplace: https://marketplace.visualstudio.com/items?itemName=okteto.remote-kubernetes @bhack it does not require to modify your image to include an ssh server any more

[ggaaooppeenngg]

Suppose I want to multiplex clients requests through a proxy, ssh can not help, I would like other protocol (Websocket HTTP) than SSH to remotely control my pod.

[pchico83]

@ggaaooppeenngg what is the use case for having multi clients?

[ggaaooppeenngg]

@pchico83 I am searching for a tool like vscode to provide coding environment in cloud to customers. For ssh, I need to provide a direct external tcp service to user, in cloud environment it could be ip address:port or a loadbalancer which costs time to be established, while if through HTTP protocol I can dynamically config a route from proxy to to a pod like Jupyter Notebook. BTW, we don't provide K8S access control to users.

[bhack]

Kubectl operatios like exec use API: https://erkanerol.github.io/post/how-kubectl-exec-works/

I suppose we are talking about a solution without SSH for Kubernetes.

[pchico83]

@ggaaooppeenngg We can support your use case, ping me at twitter (@pchico83) if you want to connect. @bhack why do you like it better via kubectl exec? Our ssh connection is done via port-forward to the pod. The ssh server is injected in the pod itself. ssh is more standard and supports things like reverse proxying. We can adapt our solution to work via kubectl exec + kubectl port-forward but we would need to understand the benefits better because it requires much more work.

[bhack]

@pchico83 Cause I've seen that in the last two years this upstream proposal didn't got momentum: https://github.com/kubernetes/kubernetes/issues/42950

[bhack]

If you follow that upstream ticket It Is not clear why they have not adopted ssh protocol or kubectl ssh subcommand. So I am not sure what are the upstream drawback.

[ggaaooppeenngg]

@pchico83

I found http CONNECT method can tunnel ssh.

package main

import (
    "fmt"
    "log"
    "net/http"

    "github.com/elazarl/goproxy"
)

func httpsHandler(host string, ctx *goproxy.ProxyCtx) (*goproxy.ConnectAction, string) {
        // apply some rules to fake-host and proxy to real host
    return goproxy.OkConnect, "your-real-ssh-host"
}

func main() {
    proxy := goproxy.NewProxyHttpServer()
    proxy.Verbose = true
    proxy.OnRequest().HandleConnectFunc(httpsHandler)
    log.Fatal(http.ListenAndServe(":8892", proxy))
}

ssh user@fake-host -o "ProxyCommand=nc -X connect -x 127.0.0.1:8892 %h %p" could connect to the real-ssh-host

[pchico83]

@ggaaooppeenngg thanks for the code snippet :blush: Yes, we could leverage the ssh connection through https very easily.

[bhack]

Seems that this Is not in the November Plan:

https://github.com/microsoft/vscode-remote-release/issues/1775