Dev container improvements

jonah-iden commented 2 months ago

What it does

This adds 4 new supported properties of the devcontainer JSON.

remoteUser which sets the user theia is started with in the container
postCreateCommand a command executed after creating and initally starting the container
extensions to set extensions installed when creating the container
settings preferences to set when creating the container

Also adds a new command line option to theia which sets preferences on startup.

How to test

Here is a devcontainer.json and a Dockerfile for testing. devcontainer.json Dockerfile

Put both of these in a .devconainer folder and run Dev Container: Reopen in Container

When creating the container in the output tab you should see the output of the postCreateCommand at the end

In the opened Container the following properties should be set:

the user (for example when opening a terminal and executing bin/bash) should be node
the extensions defined in devcontainer.json should be installed.
the html.format.templating property should be set to true

Follow-

Review checklist

[x] As an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

As a reviewer, I agree to behave in accordance with the review guidelines

jonah-iden commented 2 months ago

Thats weird both go and the forwarded port worked fine for me locally. Did you by any chance have a service allready running on 1313 thats the only thing i could explain this with. I'll take another look tomorrow maybe i can reproduce this

planger commented 2 months ago

Thanks for the fast response! I'll double-check at latest tomorrow and report back. I think I made sure to kill and prune all containers in order to test the setup. But I'll check.

planger commented 2 months ago

I can confirm that remoteUser and extensions work fine with this change. This is awesome!

Unfortunately, however, I don't really have better news after retrying from scratch regarding the ENV and forwardPorts. I ran the following:

docker stop $(docker ps -q) to stop all containers
docker system prune -a --volumes to ensure all containers are rebuilt from scratch
ss -tulnp to check that 1313 is not used anywhere yet
Started Theia and re-opened folder in devcontainer
Now ss -tulnp shows 0.0.0.0:1313 (local) / 0.0.0.0:* (peer) and [::]:1313 / [::]:*
The Ports view however still shows no forwarded ports (which it would in VS Code)
Opening a Terminal in Theia and typing go fails (because it isn't on the path), so does hugo server
Running export PATH="/go/bin:${PATH}" in the terminal fixes that tough
Now hugo server works and is listening on port 1313
Opening localhost:1313 fails though (ERR_CONNECTION_RESET)
Trying to add a port in the Port view manually fails with EADDRINUSE (because the container already uses that port, I guess)

When I close Theia, 1313 remains blocked and docker ps still shows the container, which it shouldn't as I've closed Theia.

Comparing to VS Code, I see that ss -tulnp shows 127.0.0.1:1313 / 0.0.0.0:* (VS Code) instead of 0.0.0.0:1313 / 0.0.0.0:* (Theia). Also Theia has another entry [::]:1313 / [::]:*, which isn't there at all in VS Code.

I didn't debug into all of that in more detail, but in summary I have the following observations:

Something seems to be different with respect to the port setup as mentioned above (maybe this is the reason for the port not being reachable from the host and the port not showing up in the Port view)
The terminal in the containerized Theia backend doesn't seem to respect the ENV command in the Dockerfile for me
The container keeps running after I close Theia

I hope this helps! If it doesn't, please let me know. I'm happy to try provide more info or if you like even jump on a short screenshare.

Thank you for all your work on this!

jonah-iden commented 2 months ago

thanks for this in depth analysis. I'll take a look today

jonah-iden commented 2 months ago

@planger Small update: I am able to reproduce the problem with the PATH. It seems it works fine when using a docker exec instance (for example when using the docker desktop exec tab) but not when opening a terminal through theia. Still no idea why though yet. But seems it has something to do with the temrinal library theia is using. Launching an application from theia also has the correct path

Regarding the port forwarding it works fine for me. But by default there is no service running in my container listening on port 1313 so i had to start one. Also from the Dockerfile it seems there is no service started

planger commented 2 months ago

@jonah-iden Thanks for the update and great you have a lead for the path issue!

Regarding the port: Yeah the Dockerfile / devcontainer doesn't start a process on the port automatically, but only if you run hugo server (or the corresponding task from the tasks.json). In VS Code the behavior is that it already shows the port in the port view (and seems to reserve it on the host), even though nothing is reachable on that port in the container. Once there is a process listening to the port in the container, it starts actively forwarding it to the host. I'm not sure if this can be done statically, or if VS Code actually listens in the container for a process coming up on that port and the re-initializes the port forwarding.

jonah-iden commented 1 month ago

I Found the issue with the PATH environement. It seems the /etc/profile file was overwriting the PATH and thus removing the docker changes. VSCode seems to be modifiying the profile file so that the path is not overwritten. Will push a fix soon

jonah-iden commented 1 month ago

the problem may be, that currently the forwarded ports are forwarded via the docker container. VSCode is using their own dynamic port forwarding mechanism for this. Could it be the hugo server is running on localhost and not 0.0.0.0. Maybe that could explain it?

planger commented 1 month ago

the problem may be, that currently the forwarded ports are forwarded via the docker container. VSCode is using their own dynamic port forwarding mechanism for this. Could it be the hugo server is running on localhost and not 0.0.0.0. Maybe that could explain it?

Excellent hint! That is indeed the reason. By default hugo server binds to 127.0.0.1. If I explicitly bind to 0.0.0.0 (hugo server --bind=0.0.0.0), then it works also in Theia. VS Code indeed seems to do some dynamic forwarding there. Thanks for pointing this out!

Do you think we should also look into replicating this behavior of VS Code or is it too much effort?

jonah-iden commented 1 month ago

The dynamic forwarding feature allready exists for theia, so it shouldn't be to difficult to change this behaviour. Should be at maximum a day of work i would guess. I think it would be good to mimic vscodes bahaviour here. Otherwise others could have similar problems to yours where it works in vscode but not theia

planger commented 1 month ago

Right, I agree. For compatibility reasons, I think it makes sense to also support that in Theia. Thank you very much!

jonah-iden commented 1 month ago

forwardPorts should now use the dynamic port forwarding mechanism

eclipse-theia / theia