GNS3 / gns3-server

GNS3 server
GNU General Public License v3.0
811 stars 263 forks source link

Cannot Start containers #2069

Closed EarlRamirez closed 1 year ago

EarlRamirez commented 2 years ago

I recently upgraded to GNS3 2.2.32 and I am no longer able to start any containers that were working previously because of the following error exec /gns3/init.sh: no such file or directory

I downgraded to the previous version 2.2.31 and I still getting the same error.

Here is the log from the gns3_server.log Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/gns3server/web/route.py", line 198, in control_schema await func(request, response) File "/usr/lib/python3.10/site-packages/gns3server/handlers/api/compute/docker_handler.py", line 89, in start await container.start() File "/usr/lib/python3.10/site-packages/gns3server/compute/docker/docker_vm.py", line 497, in start raise DockerError(logdata) gns3server.compute.docker.docker_error.DockerError: exec /gns3/init.sh: no such file or directory exec /gns3/init.sh: no such file or directory

In /usr/lib/python3.10/site-packages/gns3server/compute/docker/docker_vm.py line 367 there is params["Entrypoint"].insert(0, "/gns3/init.sh") # FIXME /gns3/init.sh is not found?

Does this need to be fixed in the container?

EarlRamirez commented 2 years ago

An interim solution, line 367 was commented out and the container starts as expected

grossmj commented 2 years ago

Can you try to start Python and type the following lines?

from gns3server.utils.get_resource import get_resource
get_resource("compute/docker/resources")

Thanks

EarlRamirez commented 2 years ago

Here you go @grossmj

from gns3server.utils.get_resource import get_resource get_resource('compute/docker/resources') '/usr/lib/python3.10/site-packages/gns3server/compute/docker/resources'

EarlRamirez commented 2 years ago

@grossmj where is there a need to insert /gns3/init.sh as an entry point?

grossmj commented 2 years ago

Here you go @grossmj

Thanks, what is the content of /usr/lib/python3.10/site-packages/gns3server/compute/docker/resources?

EarlRamirez commented 2 years ago

Here you go @grossmj

Thanks, what is the content of /usr/lib/python3.10/site-packages/gns3server/compute/docker/resources? image

grossmj commented 2 years ago

Thank you, for some reason the container doesn't get the /gns3 volumes that contain init.sh

Please try to manually start a new container in a terminal to see if you have any error:

docker run -v /usr/lib/python3.10/site-packages/gns3server/compute/docker/resources:/gns3:ro -it ubuntu:latest /bin/bash

EarlRamirez commented 2 years ago

Thank you, for some reason the container doesn't get the /gns3 volumes that contain init.sh

Please try to manually start a new container in a terminal to see if you have any error:

docker run -v /usr/lib/python3.10/site-packages/gns3server/compute/docker/resources:/gns3:ro -it ubuntu:latest /bin/bash

I tried manually and I can see that the volume is mounted in the container

grossmj commented 2 years ago

An interim solution, line 367 was commented out and the container starts as expected

Do you see the volume after you started the container from GNS3?

EarlRamirez commented 2 years ago

An interim solution, line 367 was commented out and the container starts as expected

Do you see the volume after you started the container from GNS3?

Yes, I do but I do not see any other active interfaces with the exception of the loopback

grossmj commented 2 years ago

The interfaces are set up by the /gns3/init.sh script that should be in that volume.

After commenting line 367, do you confirm you see the /gns3/init.sh script inside the container?

Do you have the problem with whatever the container is?

EarlRamirez commented 2 years ago

The interfaces are set up by the /gns3/init.sh script that should be in that volume.

After commenting line 367, do you confirm you see the /gns3/init.sh script inside the container?

Yes, I can confirm that the script is in the container

Do you have the problem with whatever the container is? That is correct, any containers I have the problem with not active network interfaces except for the loopback

rcanonico commented 2 years ago

The interfaces are set up by the /gns3/init.sh script that should be in that volume. After commenting line 367, do you confirm you see the /gns3/init.sh script inside the container?

Yes, I can confirm that the script is in the container

Do you have the problem with whatever the container is? That is correct, any containers I have the problem with not active network interfaces except for the loopback

Having the same problem in a GNS3 server 2.2.33.1 running in VMWare Fusion VM in a M1-based MacBook. If I run manually the container in the GNS3 VM with docker run --privileged -v /usr/local/lib/python3.8/dist-packages/gns3server/compute/docker/resources:/gns3:ro -it container_name /bin/sh the file /gns3/init.sh is in the volume but when executed manually at the command prompt, it gives the errors:

/gns3/init.sh: line 77: grep: not found
/gns3/init.sh: line 78: sleep: not found

as in: https://github.com/GNS3/gns3-server/issues/1823

grossmj commented 2 years ago

@rcanonico thanks for this info. What container to you use to reproduce the issue?

EarlRamirez commented 2 years ago

@rcanonico thanks for this info. What container to you use to reproduce the issue?

For any of the containers that I use I get the error

EarlRamirez commented 2 years ago

@grossmj any interim solution for this error "exec /gns3/init.sh: no such file or directory?"

grossmj commented 2 years ago

I have added a change for this in v2.2.34: https://github.com/GNS3/gns3-server/commit/129f0a631c0916d5fb9db8d00675f392d5a28636

Do you have the same issue using v2.2.34?

EarlRamirez commented 2 years ago

I have added a change for this in v2.2.34: 129f0a6

Do you have the same issue using v2.2.34?

This change is for grep and sleep not found but not for the original issue I have reported where the container will not start because it cannot find init.sh also this line is still the same in the docker_vm.py In /usr/lib/python3.10/site-packages/gns3server/compute/docker/docker_vm.py line 367 there is params["Entrypoint"].insert(0, "/gns3/init.sh") # FIXME /gns3/init.sh is not found?

grossmj commented 2 years ago

Sorry, you are right, that fix is not for this issue. We haven't found how to fix it yet.

EarlRamirez commented 2 years ago

Sorry, you are right, that fix is not for this issue. We haven't found how to fix it yet.

okay, thanks @grossmj

EarlRamirez commented 2 years ago

Sorry, you are right, that fix is not for this issue. We haven't found how to fix it yet.

I will continue to use the interim solution by commenting out line 367 and starting the interface with ip link set eth0 up

kelvinlnx commented 2 years ago

Hi, I faced a similar problem with starting containers on my Nobara Linux (Fedora fork). It seems that /gns3/init.sh wasn't found. But after reading this issue #2069, I went and tried a number of things. Finally I'm able to start my containers and have the interface config (from GNS3) work in my containers.

The problem that I had was the busybox image that was used.

The /usr/lib/python3.10/site-packages/gns3server/compute/docker/resources/bin/busybox version from Nobara/fedora couldn't work. So, I tried to copy the busybox command from docker.io/library/busybox:latest image. Not surprising, it failed too.

A final attempt. I decided to extract Ubuntu focal's busybox static command to try. I downloaded the debian package from launchpad.net http://launchpadlibrarian.net/471755664/busybox-static_1.30.1-4ubuntu6_amd64.deb. Finally it worked.

If I ever face this again, I'll probably just use the GNS3 VM and copy the binary to my local machine. I believe that should work as well because in GNS3 VM, it worked flawlessly.

Thanks.

EarlRamirez commented 2 years ago

Hi, I faced a similar problem with starting containers on my Nobara Linux (Fedora fork). It seems that /gns3/init.sh wasn't found. But after reading this issue #2069, I went and tried a number of things. Finally I'm able to start my containers and have the interface config (from GNS3) work in my containers.

The problem that I had was the busybox image that was used.

The /usr/lib/python3.10/site-packages/gns3server/compute/docker/resources/bin/busybox version from Nobara/fedora couldn't work. So, I tried to copy the busybox command from docker.io/library/busybox:latest image. Not surprising, it failed too.

A final attempt. I decided to extract Ubuntu focal's busybox static command to try. I downloaded the debian package from launchpad.net http://launchpadlibrarian.net/471755664/busybox-static_1.30.1-4ubuntu6_amd64.deb. Finally it worked.

If I ever face this again, I'll probably just use the GNS3 VM and copy the binary to my local machine. I believe that should work as well because in GNS3 VM, it worked flawlessly.

Thanks.

Thanks for the details, I tried what you did with the version of busybox and I get the same error stating the init file cannot be found.

kelvinlnx commented 2 years ago

busybox

I'll try to explain what I did and reconfirm the problem for my case:

Reconfirm it was busybox problem for me, by restoring the wrong file.

$ sudo su -
# cd /usr/lib/python3.10/site-packages/gns3server/compute/docker/resources/bin
# mv busybox busybox.new
# mv busybox.old busybox

Use GNS3 client to start the Ubuntu Docker Guest and I get the "_exec /gns3/init.sh: no such file or directory_" error.

I monitored the gns3_server.log in ~/.config/GNS3/2.2 and saw the following errors when I attempted to start the container.

2022-11-01 11:51:11 INFO ubridge_hypervisor.py:83 Connected to uBridge hypervisor on localhost:37645 after 0.1010 seconds
2022-11-01 11:51:11 **ERROR** docker_vm.py:490 Container UbuntuDockerGuest-1 failed to start
2022-11-01 11:51:11 INFO base_node.py:677 Stopping uBridge hypervisor localhost:37645
2022-11-01 11:51:11 INFO hypervisor.py:208 Stopping uBridge process PID=28949
2022-11-01 11:51:11 **ERROR** docker_vm.py:496 exec /gns3/init.sh: no such file or directory

Without going further with the log files. It shows clearly there is a problem trying to invoke/run the /gns3/init.sh script. So, I edited /usr/lib/python3.10/site-packages/gns3server/compute/docker/docker_vm.py and commented out the whole line that had the /gns3/init.sh line.

# vi  /usr/lib/python3.10/site-packages/gns3server/compute/docker/docker_vm.py
    #params["Entrypoint"].insert(0, "/gns3/init.sh")  # FIXME /gns3/init.sh is not found?

I restarted the GNS3 server by restarting the gns3 client(gui). Now I'm able to start the docker container. Open a console to troubleshoot the container.

# cd /gns3
# ./init.sh
bash: ./init.sh: /gns3/bin/busybox: bad interpreter: No such file or directory
# ls bin
busybox     busybox.new
# /gns3/bin/busybox
bash: /gns3/bin/busybox: No such file or directory

So the lines above show me it was busybox that is having problem. So I tried to manually run the busybox command and that exact error message was displayed.

Next I tried running the busybox.new command (which is the one that worked) and it shows very clearly everything is ok with this version of busybox that I've downloaded.

/gns3/bin/busybox.new 
BusyBox v1.30.1 (Ubuntu 1:1.30.1-4ubuntu6) multi-call binary.
BusyBox is copyrighted by many authors between 1998-2015.
Licensed under GPLv2. See source distribution for detailed
copyright notices.
#output omitted due to too long..

If you are having this same problem, you could try to copy the busybox command from the GNS3 VM provided by gns3.org and save it into this directory /usr/lib/python3.10/site-packages/gns3server/compute/docker/resources/bin. You don't need to stop the container as it's a bind mount. Just try running the new busybox command inside the container after you copy.

If this works, just edit back the /usr/lib/python3.10/site-packages/gns3server/compute/docker/docker_vm.py file and remove the hash(#) for the params... /gns3/init.sh line. This is very important. This is the one that helps us initialize the docker container network interface. This solved the "No such file or directory" error for me, and initialize the network interface based on the configs.

EarlRamirez commented 2 years ago

I'll try to explain what I did and reconfirm the problem for my case:

Thank you very much, @kelvinlnx for the detailed explanation, I used busybox 1.30.1 and it worked. Saw that busybox is a dependency of gns3server which gets downloaded from the OS repository. My containers are working with all the network interfaces available without any hacks.

Thanks again, @kelvinlnx

ArchitectingSoftware commented 1 year ago

Ive been going nuts, nuts, nuts trying to get gns3 working with open source routing (frrouting via docker) for a class lab on a Mac M1. Just got it working - love gns3. Basically I loaded the ARM VM. I pulled v 2.2.34 from GitHub-which was the latest release that had an arm64 VM. I then updated to v 2.2.35.1 via the Upgrade option on the VM.

Next from the GNS - GUI running natively on the M1, I unchecked "Enable Local Server", then setup the VM as the remote main server.

GNS3 working, but docker was not within the VM. Basically, the problem referenced here with init.sh. It looks like the busybox binary in the /bin directory is not correct for the ARM64. So from /usr/local/lib/python3.8/dist-packages/gns3server/compute/docker/resources/bin I ran sudo cp /usr/bin/busybox . and then everything working great.

Hope this helps others, I got a lot of help from the community.

grossmj commented 1 year ago

We should consider back-porting https://github.com/GNS3/gns3-server/issues/1890 to the 2.2 branch...

Gaggu097 commented 1 year ago

@ArchitectingSoftware

hi i know the issue is closed but i am having a little problem even after following your instructions, which helped btw, really thanks.

so i am using gns3, on macbook air m1 and getting the following error: connection refused after i start the router in gns3gui.