Acekorneya / Ark-Survival-Ascended-Server

Ark Survival Ascended Server Docker Image for running a server on Linux
MIT License
135 stars 24 forks source link

Cannot execute Setup #37

Open osxster opened 8 months ago

osxster commented 8 months ago

Hello,

I have followed the guide on a Ubuntu 22.04.4 LTS server VM. Installed Docker and all of that stuff. When I try to execute the setup I get:

pokuser@ubuark01:~$ ./POK-manager.sh -setup Found user with UID 1000: pokuser. Proceeding... Found group with GID 1000: pokuser. Proceeding... Checking for updates to POK-manager.sh... ----- POK-manager.sh is already up to date ----- Using Docker Compose command: '' (read from file). ./POK-manager.sh: line 473: [: : integer expression expected ./POK-manager.sh: line 473: [: : integer expression expected No user found with UID (). You may need to create a user with this UID or change an existing user's UID. To create a new user with the specified UID, run the following command: sudo useradd -u -m -s /bin/bash Replace with your preferred username.

UID and GID are correct, I added this user as 1000. It is having problems with the variables it seems.

pokuser@ubuark01:~$ id uid=1000(pokuser) gid=1000(pokuser) groups=1000(pokuser),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),110(lxd)

Any pointers would be great. Thanks!

Acekorneya commented 8 months ago

does it work if you use sudo ./POK-manager.sh you can bypass the PUID and PGID check that way and got to make suer to also have docker compose installed

i also pushed a new fix run the script again and see if that fixes the issue and let me know :)

osxster commented 8 months ago

Your fix did the trick, I did not need to use sudo. I did run into another issue with yq, I had to manually install yq. Got the error below:

jq has been successfully installed. Checking for yq... yq not found. Attempting to install Mike Farah's yq... /usr/local/bin/yq: Permission denied Failed to install Mike Farah's yq.

I did 'sudo snap install yq' to get around that and it let me finish the install.

Not sure if I should open a new issue but my instances are not starting properly. See below:

-----Starting Server----- Using Docker Compose command: 'docker compose' (read from file). Using docker compose for ... Pulling Docker image: acekorneya/asa_server:2_0_latest permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.44/images/create?fromImage=acekorneya%2Fasa_server&tag=2_0_latest": dial unix /var/run/docker.sock: connect: permission denied permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/json?all=1&filters=%7B%22label%22%3A%7B%22com.docker.compose.config-hash%22%3Atrue%2C%22com.docker.compose.project%3Dinstance_%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied -----Server Started for ----- You can check the status of your server by running -status -all or -status .

I didn't want to use sudo against your script in fears it could create files owned by root which I don't think would be desirable. Thanks for your help.

Acekorneya commented 8 months ago

you can still run sudo, as any file created by the root user will have its permissions corrected to PUID 1000:1000, allowing the user to use it correctly. However, this is a separate issue that I am trying to resolve. Using sudo will also fix the issue, but I am attempting to find a more permanent solution. This is somewhat challenging because the user needs to be added to the Docker users group, which requires sudo privileges. Therefore, it is not a straightforward process, which is why I implemented the sudo bypass. This will resolve any issues by ensuring that everything runs with PUID and PGID 1000:1000.

Acekorneya commented 8 months ago

i pushed a fix see if that helped

osxster commented 8 months ago

Same issue. I don't see an option to delete an instance, but I recreated it using the same instance name.

-----Starting Server----- Using Docker Compose command: 'docker compose' (read from file). Using docker compose for ... Pulling Docker image: acekorneya/asa_server:2_0_latest permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.44/images/create?fromImage=acekorneya%2Fasa_server&tag=2_0_latest": dial unix /var/run/docker.sock: connect: permission denied permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/json?all=1&filters=%7B%22label%22%3A%7B%22com.docker.compose.config-hash%22%3Atrue%2C%22com.docker.compose.project%3Dinstance_%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied -----Server Started for -----

Acekorneya commented 8 months ago

sorry for got to to say to do -update first before trying again lol my bad but ill work on it all let you know instead of asking you to keep trying lol lol

osxster commented 8 months ago

Hah, thats okay.. It didn't work unfortunately. I am not sure how to delete an instance properly, but I just recreated it again using the same name. Here is my output..

Docker Compose configuration for Instance Name has been finalized. Checking and adjusting ownership and permissions for /home/pokuser/... Ownership and permissions adjustment on /home/pokuser/ completed. ----- Checking for updates to POK-manager.sh ----- POK-manager.sh is already up to date. ----- Checking for updates to Docker image and server files ----- Pulling latest Docker image... permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.44/images/create?fromImage=acekorneya%2Fasa_server&tag=2_0_latest": dial unix /var/run/docker.sock: connect: permission denied SteamCMD is already installed. ----- ARK server files are already up to date with build id: 13686182 ----- ----- Update process completed ----- -----Starting Instance Name Server----- Using Docker Compose command: 'docker compose' (read from file). Using docker compose for Instance Name... Pulling Docker image: acekorneya/asa_server:2_0_latest permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.44/images/create?fromImage=acekorneya%2Fasa_server&tag=2_0_latest": dial unix /var/run/docker.sock: connect: permission denied permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/json?all=1&filters=%7B%22label%22%3A%7B%22com.docker.compose.config-hash%22%3Atrue%2C%22com.docker.compose.project%3Dinstance_ Instance Name%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied -----Server Started for Instance Name ----- You can check the status of your server by running -status -all or -status Instance_Name.

Acekorneya commented 8 months ago

ok think i fix it give it a try

osxster commented 8 months ago

Hello,

Looks better. It prompts me for my sudo password now. It did say it started up now. In order to check the status though I need to use sudo. It doesn't show running if I don't use sudo. Another thing, when I try to telnet to the server (VM) primary IP address over port 7777, the default port I am using, I get connection refused. I apologize as I don't know docker at all, but I believe something needs to be enabled to pass my incoming ports hitting the server to the docker instance. I do not think that part is working. The status utility also lists my internet IP address and of coarse the port (7777), obviously I would need to open that in my FW. But I still can't connect to the local machine over port 7777, so that would be a problem.

Acekorneya commented 8 months ago

OK i FINALLY got it fix make sure to delete the file /config/POK-manager/config.txt and now it will work correctly without asking for the password the reason was i was missing chown $USER /var/run/docker.sock to fix the permission issue but should be fix now after you delete that file

Acekorneya commented 8 months ago

Hello,

Looks better. It prompts me for my sudo password now. It did say it started up now. In order to check the status though I need to use sudo. It doesn't show running if I don't use sudo. Another thing, when I try to telnet to the server (VM) primary IP address over port 7777, the default port I am using, I get connection refused. I apologize as I don't know docker at all, but I believe something needs to be enabled to pass my incoming ports hitting the server to the docker instance. I do not think that part is working. The status utility also lists my internet IP address and of coarse the port (7777), obviously I would need to open that in my FW. But I still can't connect to the local machine over port 7777, so that would be a problem.

You will still need to open port 7777 for you to be able to connect to the server or anything on you host machine and router as well in the docker compose we issue

    ports:
      - "7777:7777/tcp"
      - "7777:7777/udp"
-

which exposes those ports to the host machine and you just have to expose them to you host firewall if you have one or allow the connection to port 7777 then local host should work.... and the same with the router to allow outside connections

if using ufw it can be

sudo ufw allow 7777

or if using debian you can do

 sudo iptables -A INPUT -p tcp --dport 7777 -j ACCEPT && sudo iptables -A INPUT -p udp --dport 7777 -j ACCEPT
osxster commented 8 months ago

Hello,

Looking into my vm, ufw was not running but apparently I am running iptables. I ran your iptables command as listed above and I can telnet to it although it immediately disconnects. Perhaps that is working although the immediate disconnect bothers me. On boot of the VM, I can see the docker instance starting if I do a ps -ef. I cannot display the status, stop, start, or restart it though. If I do sudo ./POK-manager.sh -status -all with or without sudo, it does not show any instances as running. I also cannot stop it using the ./POK-manager.sh -stop INSTANCENAME. If I kill the processes and try to manually start them using ./POK-manager.sh -start INSTANCENAME I get bind address already in use errors. Something is still kind of wrong.

When I log into ARK on a client PC, I can see it now. However when I join I get an Unknown Error or Connection Timeout connecting to it. ShooterGame.log doesn't show anything other than it started. Looks like it is slower than it started when I was trying to get it to run native on Wine. Do you have any ideas on what to look at? Thanks again for your help.

Acekorneya commented 8 months ago

from my testing i can do

acekorneya@knyserver:/home/factorioserver/ASA_Server$ ./POK-manager.sh -status -all
Checking for updates to POK-manager.sh...
----- POK-manager.sh is already up to date -----
Using Docker Compose command: 'docker compose' (read from file).
User has chosen to run Docker commands without 'sudo'.
----- Processing -status command for all running instances Please wait... -----
----- Server PVEIsland: Command: status -----
Displaying server status...
Server Name:    POK-PVE-Community-Cluster-ARK-Server-NO-WIPE
Map:            TheIsland_WP
Day:            1706
Players:        2 / 70
Mods:           927084,975282
Cluster ID:     kny512
Server Version: 35.16
Server Address: 75.85.199.1:8780
Server Ping:    143 ms
Server is up
----- Server PVPIsland: Command: status -----
Displaying server status...
Server Name:    POK-PVP-Community-CLuster-ARK-Server-NO-WIPE
Map:            TheIsland_WP
Day:            1
Players:        0 / 70
Mods:           927084,975282
Cluster ID:     kny512
Server Version: 35.16
Server Address: 75.85.199.1:8777
Server Ping:    81 ms
Server is up
----- All running instances processed with -status command. -----
acekorneya@knyserver:/home/factorioserver/ASA_Server$

acekorneya@knyserver:/home/factorioserver/ASA_Server$ ./POK-manager.sh -stop PVPIsland
Checking for updates to POK-manager.sh...
----- POK-manager.sh is already up to date -----
Using Docker Compose command: 'docker compose' (read from file).
User has chosen to run Docker commands without 'sudo'.
-----Stopping PVPIsland Server-----
[+] Running 2/2
 ✔ Container asa_PVPIsland             Removed                                                                                                                                10.8s
 ✔ Network instance_pvpisland_default  Removed                                                                                                                                 0.1s
Instance PVPIsland stopped successfully.
acekorneya@knyserver:/home/factorioserver/ASA_Server$

without using using sudo on a user with pudi and pgid of 1000 so it looks like its working

Maybe doing a complex instance name is causing issue with the script never really tested that

osxster commented 8 months ago

How do you properly delete an instance? There is no -delete option in the script. I was going to try recreating the instance, renamed the Instance_xxxxx in the home directory but I get 'Error starting userland proxy: listen tcp4 0.0.0.0:7777: bind: address already in use' when trying to start it.

I can see docker-proxy owns the port when I do a netstat.

As for your comment about Complex Instance Names, I named it IronMiners_Ark. There is an underscore in it, nothing other complex about it. I had that same name configured both as the POK-Manager instance name and the SESSION_NAME within the configuration.

Should I be running this on Debian instead of Ubuntu in that there are just enough differences to break your script? Sorry for the dumb questions, I have linux admin experience and know linux well, just no docker experience. I'd actually rather not use Docker as I only played with Docker (on a Mac though) and it was very slow. But you have the only guide that I found for running Ark Ascended server on linux and I hate to spin up a windows VM just to run this. Linux is so more more scriptable. I tried running it on Wine on a different Ubuntu VM using the steam client and stuff. The server starts up but the listener never listens. I could be doing something wrong in wine, but no logs anywhere other than it started, so I have no where to go with that. Thanks again for your help in this!

Acekorneya commented 8 months ago

How do you properly delete an instance? There is no -delete option in the script. I was going to try recreating the instance, renamed the Instance_xxxxx in the home directory but I get 'Error starting userland proxy: listen tcp4 0.0.0.0:7777: bind: address already in use' when trying to start it.

I can see docker-proxy owns the port when I do a netstat.

As for your comment about Complex Instance Names, I named it IronMiners_Ark. There is an underscore in it, nothing other complex about it. I had that same name configured both as the POK-Manager instance name and the SESSION_NAME within the configuration.

Should I be running this on Debian instead of Ubuntu in that there are just enough differences to break your script? Sorry for the dumb questions, I have linux admin experience and know linux well, just no docker experience. I'd actually rather not use Docker as I only played with Docker (on a Mac though) and it was very slow. But you have the only guide that I found for running Ark Ascended server on linux and I hate to spin up a windows VM just to run this. Linux is so more more scriptable. I tried running it on Wine on a different Ubuntu VM using the steam client and stuff. The server starts up but the listener never listens. I could be doing something wrong in wine, but no logs anywhere other than it started, so I have no where to go with that. Thanks again for your help in this!

As of right now you will need to do -stop instance name or -stop -all then you can delete the instance folder with no issues

other way is to do find the docker container id name which would be asa_instance_name usually

docker ps

docker stop <container_id_or_name>

docker rm <container_id_or_name>

This will remove left over images that didnt get stop before it folder got delete will fix that in the future to make sure we have a -delete command to delete the instance selected and find any left over container and delete them etc thanks for the advice btw :)

osxster commented 8 months ago

Hello,

I deleted the containers but for whatever reason when I reboot, my Instance_XXXX always get recreated. I also see:

pokuser@ubuark01:~$ ps -ef | grep -i ark pokuser 1317 1315 0 13:01 ? 00:00:00 /bin/bash /home/pok/scripts/monitor_ark_server.sh pokuser 1984 1982 0 13:03 ? 00:00:00 /bin/bash /home/pok/scripts/monitor_ark_server.sh pokuser 2248 2246 0 13:05 ? 00:00:00 /bin/bash /home/pok/scripts/monitor_ark_server.sh pokuser 2576 2574 0 13:07 ? 00:00:00 /bin/bash /home/pok/scripts/monitor_ark_server.sh pokuser 2960 2958 0 13:09 ? 00:00:00 /bin/bash /home/pok/scripts/monitor_ark_server.sh pokuser 3786 3784 0 13:11 ? 00:00:00 /bin/bash /home/pok/scripts/monitor_ark_server.sh

Those processes keep coming back and keep incrementing. The directory /home/pok doesn't even exist.

pokuser@ubuark01:~$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

Shows no containers anymore, but the instance directories even after I delete them in /home/pokuser get recreated on reboot.

Listeners also stay running I assume of the Docker Proxy..

pokuser@ubuark01:~$ netstat -an | grep 7777 tcp 0 0 0.0.0.0:7777 0.0.0.0: LISTEN tcp6 0 0 :::7777 ::: LISTEN udp 0 0 0.0.0.0:7777 0.0.0.0: udp6 0 0 :::7777 :::

So I cannot create a new instance using that port although they don't seem to go away after deleting either. I haven't figured out what process is recreating these on reboot.

Seems odd that Ubuntu would be this much different than Debian as you are not seeing these issues.

Acekorneya commented 8 months ago

You can try removing the image as well see if that stop it from coming back up it seems something is restarting those containers on reboot you can try the step below to remove the image also

# 4. (Optional) List all Docker images to find the image ID
docker images

# 5. Remove the Docker image using its ID
docker rmi <image_id>
osxster commented 8 months ago

Funny, removed the images: pokuser@ubuark01:~$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE acekorneya/asa_server 2_0_latest ac4e0cd99997 16 hours ago 1.52GB acekorneya/asa_server 19308e33ad69 2 days ago 1.52GB pokuser@ubuark01:~$ docker rmi ac4e0cd99997 Untagged: acekorneya/asa_server:2_0_latest Untagged: acekorneya/asa_server@sha256:0fdf2dcb3109ce3e86a8c6e7a152571db837b8f40e24436481f3fd2ca44afe25 Deleted: sha256:ac4e0cd999971ed1c9c41bfc17d7f9440dc3fc67e8190d34241645c89269918a Deleted: sha256:6faca71f72d87f8ee6d9ecbb469b6910b03a92f01022aa0ea11f67fb4adef477 Deleted: sha256:2553671384a746e44913c382b65e643332368764f96031548d9ce4e38f67d625 Deleted: sha256:d4422d2ffa77b7b34976774f839df3a1778869b0a7146f15cf36c9d6bd988651 Deleted: sha256:1e9374c90cf1f1547f0a3cc9c4ea8246e842e3288c2bb84c71b775c1ca465393 pokuser@ubuark01:~$ docker rmi 19308e33ad69 Untagged: acekorneya/asa_server@sha256:a6c037da5e345675ad32926db07c9d6a6c71188ea2f73e4cc62da346cfdebbed Deleted: sha256:19308e33ad6909628a6394355fc4cf2837f843973322e3ca03442772736b1da3 Deleted: sha256:29fb747f7bfa7adfd0c55e7425dac20bd6d08be1af7a239379d9f8be6e60edc1 Deleted: sha256:2d8b8bb4e8dce5dd4995f792dc18d2ed1cae5d66e3b60363d9d071eb046e77f7 Deleted: sha256:34fe80ab644e95991bb166e7b73e02768b4439340a2d215e7e2f7b352342f535 Deleted: sha256:1f1b209592fb26817ac6cc5d7060136b22d0ac62e2656ddd552f61e959cf5d31 Deleted: sha256:d95ce305b1067decd7123cb1e634578c31a46e1412806aef8fd50e54eb79839d Deleted: sha256:d6d9d5a0118b1d1ba501a9f400772d92af2a0a5fd77e45ddae0416f401ba7ae8 Deleted: sha256:6ce36b534489a10a8207a964b8309be40dcba54a1933200f448c9fc185d2f6f5 Deleted: sha256:d660238725c30db244b6399aaaa018af271d465b11677d036173495a3fe6d82b Deleted: sha256:5493c4ab883fc3d4356660efcac6355d50630652a74527a321d9c5228ded26e1 Deleted: sha256:c07198906902b1f8c199f1f4b4c109c266e3be8de6aead9dbccacb902774a335 Deleted: sha256:e9ea3c09b4d15a56093ca53dbfe8bdbb447a04b9bebcb3c95a10c57ba84432e0 Deleted: sha256:c3c9c92ee5ee674348a54f8f923618dd4b946ded6914ab5ec40c797e57e3d903 Deleted: sha256:2665aa582cffb82fb5282f5a0fc49375750d43b02b84c7f3e8a38634c74e66c7 Deleted: sha256:cbd863f31e714bcb2ef4b78fb4eab8d76e1de32f38d77cf840a17606b59e88ec Deleted: sha256:b7cf9854a3aa582b24e462402d73a4dab625698c0c0d6ca5f9d7c260ad32feec Deleted: sha256:91a3dbea4468a7ab2f0b30be97d598da41444f115c4707cd7f47c824b05b9f57 Deleted: sha256:5faf9c0a9efe4675ecd21a4ec417d51077d5e75da9e673161a94e7d6cd43f92c pokuser@ubuark01:~$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE pokuser@ubuark01:~$ sudo reboot

So they were gone...

Now after reboot:

pokuser@ubuark01:~$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE acekorneya/asa_server 2_0_latest 19308e33ad69 2 days ago 1.52GB hello-world latest d2c94e258dcb 10 months ago 13.3kB pokuser@ubuark01:~$ docker rmi 19308e33ad69 Error response from daemon: conflict: unable to delete 19308e33ad69 (cannot be forced) - image is being used by running container acac9eef0a47 pokuser@ubuark01:~$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE acekorneya/asa_server 2_0_latest 19308e33ad69 2 days ago 1.52GB hello-world latest d2c94e258dcb 10 months ago 13.3kB pokuser@ubuark01:~$ docker rmi 19308e33ad69 Error response from daemon: conflict: unable to delete 19308e33ad69 (cannot be forced) - image is being used by running container acac9eef0a47

I also see:

pokuser@ubuark01:~$ ps -ef | grep proxy root 1204 968 0 13:56 ? 00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1211 968 0 13:56 ? 00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1226 968 0 13:56 ? 00:00:00 /usr/bin/docker-proxy -proto udp -host-ip 0.0.0.0 -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1233 968 0 13:56 ? 00:00:00 /usr/bin/docker-proxy -proto udp -host-ip :: -host-port 7777 -container-ip 172.18.0.2 -container-port 7777

So that is why I cannot recreate instances using port 7777.

Acekorneya commented 8 months ago

The output you've provided indicates that docker-proxy is already using port 7777 for both TCP and UDP protocols on both IPv4 (0.0.0.0) and IPv6 (::) addresses, forwarding traffic to a container with IP 172.18.0.2 on the same port. This is why you're unable to use port 7777 for new instances; the port is already in use.

To resolve this issue, you have a few options:

Stop or Remove the Current Container Using Port 7777: If the container using port 7777 is no longer needed, you can stop or remove it to free up the port. First, identify the container using the port, then stop or remove it.

To find the container ID, you can use:

docker ps | grep 7777

Once you have the container ID, you can stop the container with:

docker stop [container_id]

If you decide you no longer need the container, you can remove it with:

docker rm [container_id]

Another way is to just use another port but you should be able to remove the container running that port

osxster commented 8 months ago

Hmm, docker images disappeared on their own. 'docker ps' give me no images now. But the proxy is still running and my Instance directory is still there, always comes back when I delete it.

` pokuser@ubuark01:~$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES pokuser@ubuark01:~$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE pokuser@ubuark01:~$ ps -ef | grep 7777 root 1204 968 0 13:56 ? 00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1211 968 0 13:56 ? 00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1226 968 0 13:56 ? 00:00:00 /usr/bin/docker-proxy -proto udp -host-ip 0.0.0.0 -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1233 968 0 13:56 ? 00:00:00 /usr/bin/docker-proxy -proto udp -host-ip :: -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 pokuser 39780 1491 0 15:01 pts/0 00:00:00 grep --color=auto 7777 pokuser@ubuark01:~$ ps -ef | grep -i docker root 968 1 0 13:56 ? 00:00:12 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock root 1204 968 0 13:56 ? 00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1211 968 0 13:56 ? 00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1226 968 0 13:56 ? 00:00:00 /usr/bin/docker-proxy -proto udp -host-ip 0.0.0.0 -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1233 968 0 13:56 ? 00:00:00 /usr/bin/docker-proxy -proto udp -host-ip :: -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1854 1 0 13:56 ? 00:00:06 dockerd --group docker --exec-root=/run/snap.docker --data-root=/var/snap/docker/common/var-lib-docker --pidfile=/run/snap.docker/docker.pid --config-file=/var/snap/docker/2915/config/daemon.json root 1922 1854 0 13:56 ? 00:00:04 containerd --config /run/snap.docker/containerd/containerd.toml --log-level error pokuser 39893 1491 0 15:01 pts/0 00:00:00 grep --color=auto -I docker '

Just been looking around the system, haven't found where this proxy is configured. Any of the referenced config files in the ps output have nothing useable in it.

Acekorneya commented 8 months ago

Given the information you've provided, it seems there are multiple instances of Docker running on your system, which can lead to confusion and conflicts like the ones you're experiencing. Specifically, the presence of both a standard Docker daemon process (dockerd) and another instance that appears to be running through Snap (dockerd with paths pointing to /snap/docker/), suggests a mixed environment that might not be fully integrated.

Here are steps to troubleshoot and potentially resolve the issue:

  1. Identify and Stop Duplicate Docker Services:

    • Check if both Snap and traditional Docker services are running. If so, decide which one you intend to use consistently.
    • If you choose the traditional Docker installation, you may need to stop and disable the Snap Docker service:
      sudo snap stop docker
      sudo snap disable docker
    • Conversely, if you prefer to use the Snap version, consider removing the traditional Docker package to avoid conflicts.
  2. Clearing Docker Proxy Processes:

    • The docker-proxy processes you've identified are likely remnants of containers that were not properly stopped or removed. You can try stopping them manually using the kill command with their process IDs (PIDs):
      sudo kill 1204 1211 1226 1233
    • If these processes are persistently restarting, it indicates that the Docker daemon is automatically relaunching them, possibly due to configured restart policies on containers or services. Ensuring you've stopped and removed all containers and services related to these ports can help.
  3. Investigate Docker Configuration:

    • Review Docker's configuration files for any custom settings that might be causing this behavior. For the standard Docker installation, check /etc/docker/daemon.json. For the Snap version, you mentioned examining the referenced config files but consider also looking into Snap-specific configurations or services that might auto-start.
  4. Restart Docker Daemon:

    • After making any changes, restart the Docker daemon to ensure all settings are applied and any orphaned or stale processes are cleared:
      sudo systemctl restart docker

      Or, for the Snap-managed Docker (if you've chosen to stick with it):

      sudo snap restart docker
  5. Clean Up and Reassess:

    • After performing the steps above, reassess your environment. Check running containers, images, and network proxies again to ensure the environment is clean and configured as expected.
  6. Consult Logs for Additional Clues:

    • If issues persist, consult Docker logs for additional insights. For the standard installation, look into /var/log/docker.log or use journalctl -u docker.service. For the Snap version, logs might be accessed via Snap's logging facilities.
osxster commented 8 months ago

I ran both the snap and non snap dockers directly by accessing them in the appropriate paths. Perhaps they are executing using an environment variable and looking at the wrong docker version's repositories. But I do not see ANY containers installed, but obviously something keeps starting up that proxy. 'docker ps -a' and 'docker images' even when executing them from the appropriate paths doesn't show any containers installed right now. As for the proxies, I know I can kill them but I didn't bother as I don't see why they wouldn't start back up as I haven't figured out what is starting these up. The only daemon.json I found only has log level in it, nothing else. I obviously have a mess now, was more or less working with this to try to learn it and fix the problem. But obviously this now is a mess. I could just start all over with Debian 12.5, if that works properly with your scripts, it sounds like that will be easier. These are all VM's on an ESX server, so I don't really care about blowing it away as it won't affect anything.

osxster commented 8 months ago

Hello,

Reinstalled a new Debian VM, got everything installed, I think it starts properly. I'm running into FW issues it seems. This is Debian 12, I can telnet to localhost 7777 but not the IP address. I am assuming this uses iptables as it doesn't seem ufw is installed, but even 'iptables -F' results in a connection refused. Is there another FW in Debian that I can't seem to find?

pokuser@debianark01:~$ netstat -an | grep 7777 tcp 0 0 0.0.0.0:7777 0.0.0.0: LISTEN tcp6 0 0 :::7777 ::: LISTEN udp 0 0 0.0.0.0:7777 0.0.0.0: udp6 0 0 :::7777 :::

pokuser@debianark01:~$ sudo iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination

Chain FORWARD (policy DROP) target prot opt source destination

Chain OUTPUT (policy ACCEPT) target prot opt source destination

Chain DOCKER (0 references) target prot opt source destination

Chain DOCKER-ISOLATION-STAGE-1 (0 references) target prot opt source destination

Chain DOCKER-ISOLATION-STAGE-2 (0 references) target prot opt source destination

Chain DOCKER-USER (0 references) target prot opt source destination

pokuser@debianark01:~$ telnet localhost 7777 Trying ::1... Connected to localhost. Escape character is '^]'. Connection closed by foreign host. pokuser@debianark01:~$ telnet 192.168.ip 7777 Trying 192.168.ip... telnet: Unable to connect to remote host: Connection refused

Sorry for the newb questions, you sound like a Debian person and probably know the answer off the top of your head. Thanks.

Acekorneya commented 8 months ago

No worries.. but Why are you trying to telnet to the port that part I don't understand.

Ark Survival Ascended no longer needs a query port set, just the game port alone should be enough.

Default 7777 is gameport and 27015 is query port (in survival evolved)

In your router you have to open gameport (UDP/TCP) in the external port, in your case 7777, to the local IP address the server runs on. (internal port can be left empty)but really depends on your router

If you use Rcon, that port has to be opened on TCP. If you want to use a different rcon manager like a web base one

Like I said you can ignore the query port in ascended completely.

Then with firewall down or firewall rule set - your server should be reachable. If it's not, something is most likely set wrong in your router.

osxster commented 8 months ago

Hello,

The reason I am telneting to port 7777 is to verify it is listening from outside of the machine. It is not, so opening up a router / internet facing Firewall will not help if I can't even connect to the port via telnet. It almost seems like there is a Debian FW running that is blocking port 7777. I was looking on the internet and couldn't find any other FW which could be installed by default in Debian. I've never used Debian before, so I am not entirely sure what to look for to figure out what is blocking this. I had this exact same problem on Ubuntu, the iptables command fixed it. I can't fix it on this though. I see the process running although I can't check status using the status command.

pokuser@debianark01:~$ sudo ./POK-manager.sh -status IronMinersArk Running with sudo privileges. Skipping PUID and PGID check. Checking for updates to POK-manager.sh... ----- POK-manager.sh is already up to date ----- Using Docker Compose command: 'docker compose' (read from file). User has chosen to run Docker commands without 'sudo'. Processing -status command on IronMinersArk... Displaying server status... Error: OAuth request failed with error:

pokuser@debianark01:~$ ps -ef | grep IronMinersArk pokuser 2810 1946 0 10:24 ? 00:00:00 python3 /usr/local/bin/proton run /home/pok/arkserver/ShooterGame/Binaries/Win64/ArkAscendedServer.exe TheIsland_WP?listen?SessionName=IronMinersArk??RCONEnabled=True?RCONPort=27020?ServerPassword=miningark595?ServerAdminPassword=myironadminpwd -Port=7777 -WinLiveMaxPlayers=70 -clusterid=cluster -servergamelog -servergamelogincludetribelogs -ServerRCONOutputTribeLogs -NotifyAdminCommandsInChat -UseDynamicConfig -NoBattlEye pokuser 2812 2810 0 10:24 ? 00:00:00 c:\windows\system32\steam.exe /home/pok/arkserver/ShooterGame/Binaries/Win64/ArkAscendedServer.exe TheIsland_WP?listen?SessionName=IronMinersArk??RCONEnabled=True?RCONPort=27020?ServerPassword=miningark595?ServerAdminPassword=myironadminpwd -Port=7777 -WinLiveMaxPlayers=70 -clusterid=cluster -servergamelog -servergamelogincludetribelogs -ServerRCONOutputTribeLogs -NotifyAdminCommandsInChat -UseDynamicConfig -NoBattlEye pokuser 2886 1902 37 10:24 ? 00:05:05 Z:\home\pok\arkserver\ShooterGame\Binaries\Win64\ArkAscendedServer.exe TheIsland_WP?listen?SessionName=IronMinersArk??RCONEnabled=True?RCONPort=27020?ServerPassword=miningark595?ServerAdminPassword=myironadminpwd -Port=7777 -WinLiveMaxPlayers=70 -clusterid=cluster -servergamelog -servergamelogincludetribelogs -ServerRCONOutputTribeLogs -NotifyAdminCommandsInChat -UseDynamicConfig -NoBattlEye pokuser 4849 3321 0 10:38 pts/0 00:00:00 grep IronMinersArk

pokuser@debianark01:~$ netstat -an | grep 7777 tcp 0 0 0.0.0.0:7777 0.0.0.0: LISTEN tcp6 0 0 :::7777 ::: LISTEN udp 0 0 0.0.0.0:7777 0.0.0.0: udp6 0 0 :::7777 :::

pokuser@debianark01:~$ ps -ef | grep 7777 root 1833 898 0 10:23 ? 00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1840 898 0 10:23 ? 00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1857 898 0 10:23 ? 00:00:00 /usr/bin/docker-proxy -proto udp -host-ip 0.0.0.0 -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1864 898 0 10:23 ? 00:00:00 /usr/bin/docker-proxy -proto udp -host-ip :: -host-port 7777 -container-ip 172.18.0.2 -container-port 7777

Seems like everything is running, but

pokuser@debianark01:~$ telnet localhost 7777 Trying ::1... Connected to localhost. Escape character is '^]'. Connection closed by foreign host. pokuser@debianark01:~$ telnet 192.168.102.69 7777 Trying 192.168.102.69... telnet: Unable to connect to remote host: Connection refused

So the port is not accessible via the IP address and I get the same thing from another host on my network when I telnet.

If I try to connect to my Ubuntu ARK server I get:

pokuser@debianark01:~$ telnet 192.168.102.95 7777 Trying 192.168.102.95... Connected to 192.168.102.95. Escape character is '^]'. Connection closed by foreign host.

Which shows the port is listening and receiving connections. My Ubuntu ARK Server doesn't work from an ARK client, but I can connect to the port via telnet.

Acekorneya commented 8 months ago

Hello,

The reason I am telneting to port 7777 is to verify it is listening from outside of the machine. It is not, so opening up a router / internet facing Firewall will not help if I can't even connect to the port via telnet. It almost seems like there is a Debian FW running that is blocking port 7777. I was looking on the internet and couldn't find any other FW which could be installed by default in Debian. I've never used Debian before, so I am not entirely sure what to look for to figure out what is blocking this. I had this exact same problem on Ubuntu, the iptables command fixed it. I can't fix it on this though. I see the process running although I can't check status using the status command.

pokuser@debianark01:~$ sudo ./POK-manager.sh -status IronMinersArk Running with sudo privileges. Skipping PUID and PGID check. Checking for updates to POK-manager.sh... ----- POK-manager.sh is already up to date ----- Using Docker Compose command: 'docker compose' (read from file). User has chosen to run Docker commands without 'sudo'. Processing -status command on IronMinersArk... Displaying server status... Error: OAuth request failed with error:

pokuser@debianark01:~$ ps -ef | grep IronMinersArk pokuser 2810 1946 0 10:24 ? 00:00:00 python3 /usr/local/bin/proton run /home/pok/arkserver/ShooterGame/Binaries/Win64/ArkAscendedServer.exe TheIsland_WP?listen?SessionName=IronMinersArk??RCONEnabled=True?RCONPort=27020?ServerPassword=miningark595?ServerAdminPassword=myironadminpwd -Port=7777 -WinLiveMaxPlayers=70 -clusterid=cluster -servergamelog -servergamelogincludetribelogs -ServerRCONOutputTribeLogs -NotifyAdminCommandsInChat -UseDynamicConfig -NoBattlEye pokuser 2812 2810 0 10:24 ? 00:00:00 c:\windows\system32\steam.exe /home/pok/arkserver/ShooterGame/Binaries/Win64/ArkAscendedServer.exe TheIsland_WP?listen?SessionName=IronMinersArk??RCONEnabled=True?RCONPort=27020?ServerPassword=miningark595?ServerAdminPassword=myironadminpwd -Port=7777 -WinLiveMaxPlayers=70 -clusterid=cluster -servergamelog -servergamelogincludetribelogs -ServerRCONOutputTribeLogs -NotifyAdminCommandsInChat -UseDynamicConfig -NoBattlEye pokuser 2886 1902 37 10:24 ? 00:05:05 Z:\home\pok\arkserver\ShooterGame\Binaries\Win64\ArkAscendedServer.exe TheIsland_WP?listen?SessionName=IronMinersArk??RCONEnabled=True?RCONPort=27020?ServerPassword=miningark595?ServerAdminPassword=myironadminpwd -Port=7777 -WinLiveMaxPlayers=70 -clusterid=cluster -servergamelog -servergamelogincludetribelogs -ServerRCONOutputTribeLogs -NotifyAdminCommandsInChat -UseDynamicConfig -NoBattlEye pokuser 4849 3321 0 10:38 pts/0 00:00:00 grep IronMinersArk

pokuser@debianark01:~$ netstat -an | grep 7777 tcp 0 0 0.0.0.0:7777 0.0.0.0: LISTEN tcp6 0 0 :::7777 ::: LISTEN udp 0 0 0.0.0.0:7777 0.0.0.0: udp6 0 0 :::7777 :::

pokuser@debianark01:~$ ps -ef | grep 7777 root 1833 898 0 10:23 ? 00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1840 898 0 10:23 ? 00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1857 898 0 10:23 ? 00:00:00 /usr/bin/docker-proxy -proto udp -host-ip 0.0.0.0 -host-port 7777 -container-ip 172.18.0.2 -container-port 7777 root 1864 898 0 10:23 ? 00:00:00 /usr/bin/docker-proxy -proto udp -host-ip :: -host-port 7777 -container-ip 172.18.0.2 -container-port 7777

Seems like everything is running, but

pokuser@debianark01:~$ telnet localhost 7777 Trying ::1... Connected to localhost. Escape character is '^]'. Connection closed by foreign host. pokuser@debianark01:~$ telnet 192.168.102.69 7777 Trying 192.168.102.69... telnet: Unable to connect to remote host: Connection refused

So the port is not accessible via the IP address and I get the same thing from another host on my network when I telnet.

If I try to connect to my Ubuntu ARK server I get:

pokuser@debianark01:~$ telnet 192.168.102.95 7777 Trying 192.168.102.95... Connected to 192.168.102.95. Escape character is '^]'. Connection closed by foreign host.

Which shows the port is listening and receiving connections. My Ubuntu ARK Server doesn't work from an ARK client, but I can connect to the port via telnet.

The output from your telnet command indicates that a service is indeed listening on port 7777 on the host with IP address 192.168.102.95 (your Ubuntu ARK server), and it accepted the TCP connection initiated by telnet. However, because the ARK server does not use the Telnet protocol, it likely closed the connection immediately after it was established, as indicated by "Connection closed by foreign host."

This behavior is expected and normal for services that do not communicate using the Telnet protocol. The primary purpose of using telnet in this context is to check whether the port is open and a service is listening on it, rather than to establish a meaningful session with the service. The successful connection (even though it was immediately closed) serves as a basic network connectivity test, confirming that:

The network path between your Debian machine (pokuser@debianark01) and the Ubuntu ARK server (at 192.168.102.95) is clear, without any blocking firewalls or network issues preventing TCP connections on port 7777. The ARK server application is running and listening for incoming connections on port 7777. so now just open the ports on router to let people join your server and make sure to have selected show player server to see the server in-game when searching for it

osxster commented 8 months ago

Hello,

I'm sort of a network guy and old school unix guy, so I understand networks pretty well. While people use to think of telnet as over port 23 as a protocol, the telnet program isn't a protocol though. It is really a client utility for connecting clear text. You can use it for SMTP, HTTP, FTP (to an extent as long as you aren't transferring files), and any other clear text protocol so long as you type the appropriate commands. You can use telnet to send an email to someone over port 25. Just telnet to the port, send the appropriate commands and headers to forge an email, and then it sends. You can telnet over port 80 and display HTML text, emulate a web browser without having a web browser. As a result it is a great utility (probably the best) for testing to make sure a machine is listening on a port to check firewall connectivity.

On a host without a Firewall, usually Connection Refused would indicate the port is not listening and the machine sent back a RESET packet. If you telnet and get a hanging connection, that means you never got a response back which could indicate a FW is blocking. Although some FW's can return TCP Reset which would cause Telnet to display Connection Refused. If you get Connected to IP address and Escape character is '^]', you connected to the server and got a TCP Three Way Handshake. Telnet is really the best scanner for checking FW connectivity. However it is just TCP.

The problem is I never got a proper three way handshake even an immediate connection closed when I telnet to the machines primary IP address or from another host on the same network. What is confusing here is if this is a local machine FW issue or Docker problem. Perhaps ARK is only using UDP, and it is that Docker Proxy listening on both TCP 7777 and UDP 7777, perhaps that is the issue although it doesn't make sense why I can connect locally to 127.0.0.1 but not the machine IP address. I guess I can try opening up my external FW and see if I can connect, but I am still bothered by the telnet results over port 7777 TCP and why it fails. Looks like there is also a iptables-nft in Debian, I flushed that table as well, but it didn't help.

Acekorneya commented 8 months ago
  1. Docker Proxy and Network Modes Docker can manage network traffic to containers using different network modes, including bridge (default), host, and overlay among others. The docker-proxy process you've observed is part of how Docker implements port forwarding in the bridge network mode, enabling containers to communicate with the outside world.

  2. TCP vs. UDP ARK servers, by their nature, primarily use UDP for game traffic. If ARK is configured to use UDP on port 7777, a TCP connection attempt (like telnet) wouldn't be meaningful beyond checking if the port is open for listening. This could explain why you're seeing a connection closed immediately; the service isn't designed to handle TCP connections on that port. However, ARK servers also use TCP for RCON (remote console), often on a different port.

  3. Localhost vs. External IP Connecting to localhost (127.0.0.1) or ::1 (IPv6 localhost) is different from connecting to the machine's external IP address due to how network interfaces and Docker's network bridging work. If you can connect to the service via localhost but not the external IP, it might indicate:

A Docker network configuration issue where the port forwarding is not correctly set up for external interfaces. A local firewall or network policy restricting access to the port from non-localhost addresses.

  1. Flushing iptables-nft Debian, like many Linux distributions, has been transitioning from iptables to nftables as the default backend for its firewall management. Flushing iptables-nft without careful consideration can have unintended consequences, including disrupting valid network rules set up by Docker.
osxster commented 8 months ago

Appreciate the write up. I have since rebooted to 'unflush' my machine's local FW. The reason I had flushed it was to eliminate local FW. Nothing makes things more complicated than unnecessary local FW's blocking traffic. In any event, I guess whatever issue I am having is not a FW issue, but I essentially have the same issue with Debian as I do with Ubuntu connecting to the ARK server from a client. I just get Connection Timed Out. It shows up in the list, but I cannot connect to it. My FW (router) is opened on the internet for both UDP and TCP 7777, but looks like, from taking a packet capture on the debian machine, only UDP 7777 is being used and I can see traffic going back and forth between the client and my ARK server. Are there any tips for enabling any logging to troubleshoot what is wrong? My gaming client can connect to other ARK servers on the internet, so it doesn't appear to be a problem with my client.

SoMoney commented 7 months ago

I had a similar issue in the end with my Netgear router forwarding both TCP & UDP to 7777. Try disabling TCP7777! Some kind of bug with the newer Netgears. When I ONLY allow UDP7777 it works as expected.