ARK Survival Evolved - ./arkserver modupdate

[ToeiRei]

User Story

As a server admin, I want to work around steamcmds shortcomings so that mods are installed properly.

Basic info

• Distro: Debian 10
• Game: ARK Survival Evolved
• Command: modupdate (maybe?)
• LinuxGSM version: to be determined

Further Information

As lgsm is mostly shell script, I would suggest doing some parsing of the Game.ini file like cat Game.ini | grep ModIDS | awk -F'=' '{print $2}'

and run those through a loop like

for i in "${WORKSHOP[@]}"
do
  Engine/Binaries/ThirdParty/SteamCMD/Linux/steamcmd.sh +login anonymous +workshop_download_item ${GAMEID} ${i} validate +quit
done

That way we could update mods in a more failsafe way - or even install missing ones present in the config.

besides that, steamcmd should have exit codes as seen on https://github.com/GameServerManagers/LinuxGSM/issues/719#issuecomment-330470766 so we could at least catch some problems there or even retry on problems like

ERROR! Download item 1380777369 failed (Request has been canceled).Redirecting stderr to '/home/ark/Steam/logs/stderr.txt'

[dgibbs64]

Fixes are currently ongoing to get ark mods to autoinstall using -automanagedmods. Currently, the auto update is working. However, steamcmd may choose one of 3 locations to download the mods because 🤷‍♂. I am testing yet another update to auto select this directory. So this change should not be required. Please wait for update v20.1.5

[ToeiRei]

The problem is -automanagedmods fails with mods bigger than 1 GB as stuff times out. Best example is the steampunk mod.

[dgibbs64]

that is good to know can I have the link to that mod so I can investigate. thanks

[ToeiRei]

That mod is https://steamcommunity.com/sharedfiles/filedetails/?id=679529026

I even know the ID from the top of my head due to the headache with updating.

Steam Workshop :: Ark Steampunk Mod

Steam Workshop: ARK: Survival Evolved. MOD ID: 679529026 https://i.imgur.com/hoO1UkW.png hissssss Snap twang Clang Never mind that steady Whirr of what may be gears. Introduced to your ears are sounds from another time. Somet

[dgibbs64]

wow bad times that u can remember a mod id

[ToeiRei]

basically a manual 'workshop_download' will time out as follows:

Connecting anonymously to Steam Public...Loaded client id: 2154548373600770782
Listening for IPv4 broadcast on: 27036
Logged in OK
Waiting for user info...OK
Downloading item 679529026 ...Failed to set thread priority: per-thread setup failed
Failed to set thread priority: per-thread setup failed
Failed to set thread priority: per-thread setup failed
Failed to set thread priority: per-thread setup failed
Failed to set thread priority: per-thread setup failed
Failed to set thread priority: per-thread setup failed
Failed to set thread priority: per-thread setup failed
ERROR! Timeout downloading item 679529026depotreconstruct.cpp (1348) : Assertion Failed: m_nOutstandingWorkItems == 0
CWorkThreadPool::~CWorkThreadPool: work complete queue not empty, 3 items discarded.
completionportmanager_posix.cpp (358) : Assertion Failed: Async I/O on closed handle 45
Redirecting stderr to '/home/ark/Steam/logs/stderr.txt'
[  0%] Checking for available updates...
[----] Verifying installation...
Steam Console Client (c) Valve Corporation
-- type 'quit' to exit --
Loading Steam API...Failed to init SDL priority manager: SDL not found
Failed to set thread priority: per-thread setup failed
Failed to set thread priority: per-thread setup failed
OK.

so you usually add a 'validate' and retry to infinity and beyond until you end up with eventually finishing the download of the mod.

in the end a simple server 'restart & update' leaves the server unjoinable due to the mod actually not being updated. This is why we need a somehow solid workaround as this happens with every mod bigger than 1 GB.

[ToeiRei]

However, steamcmd may choose one of 3 locations to download the mods

Using the command the ARK Server aka ShooterGame uses (start the game, use ps axf | cat) makes it clear for you on how to use steamcmd. I used my loop from above to get the mods working on a couple of servers by now and the location has always been the same on everyone of them in a reproduceable fashion.

[HeapUnderfl0w]

I actually have a similar issue here #2764 and was proposing a hook system so a user can solve it themselves (which would also mean its the users responsibility to see that mods land in the correct directory)

[dgibbs64]

So possible solution is to use the validate command to download the workshop items as validate is required.

steamcmd +login anonymous +force_install_dir "serverfiles" +app_update "376030" +workshop_download_item 679529026 validate +quit

[HeapUnderfl0w]

yes and no.

yes, this would (as far as i am aware) continue the download in progress and continue downloading that specific mod

no, as mods that are larger sizes will take more calls to download fully
there are for example mods that are ~3gb in size, which need validation 3 to 5 times before being completely downloaded.

it is largely dependent on mod size how many times steamcmd will time out before completely downloading a mod.

also it is not needed to pass +app_update to download the mod instead it is called like +workshop_download_item 376030 679529026

related sources to this problem

• https://steamcommunity.com/discussions/forum/1/215439774859993377/
• https://steamcommunity.com/app/346110/discussions/10/530649887212662565/?ctp=2#c1640917625015373138
• https://github.com/FezVrasta/ark-server-tools/commit/5c5cd58e341466dd621f8082d54ce0cc258e344c

SteamCMD and Download Timeouts :: Help and Tips

Can SteamCMD be used to update mods? :: ARK: Survival Evolved Server Hosting & Advertisement

I'm playing around with SotF and a helpful player commented in another thread that in order to update SotF you have to manually copy over files from a client machine to update the mod on the server. I've been running a regular Ark server and updating it with SteamCMD since the game came out. I'm wondering if there is a way to use SteamCMD and kick off a script to update the mod as well?

[dgibbs64]

@HeapUnderfl0w do you think it also comes down to how fast it can download as well?

[dgibbs64]

related https://github.com/ValveSoftware/steam-for-linux/issues/6321

[HeapUnderfl0w]

@HeapUnderfl0w do you think it also comes down to how fast it can download as well?

i would assume yes. i remember reading something about a specific tcp package timing out being the culprit for the errors (dont quote me on that tho). that would mean that you have a time, not a size limit.

TL;DR yes, its time, not size afaik

[dgibbs64]

Interesting. There are a few options for steamcmd that have a timeout. Stab in the dark but one of them might help https://github.com/dgibbs64/SteamCMD-Commands-List/blob/master/steamcmd_commands.txt

GitHub

dgibbs64/SteamCMD-Commands-List

SteamCMD Commands List. Contribute to dgibbs64/SteamCMD-Commands-List development by creating an account on GitHub.

[HeapUnderfl0w]

well imho from the list there the only 2 containing timeouts that could maybe help, would be these 2

1. DepotDownloadProgressTimeout maybe its timing out because it doesnt meet a progress deadline?
2. @nClientCMUDPPingTimeoutUseGoodEnoughMS whatever this means (tho 120, which seems to be the default, seems a bit low)

[ToeiRei]

I tried the timeouts without big success. On the other hand, I could extend my script to get the mods from the ark config.

With some work we could even check against a collection of mods...

[dgibbs64]

I am doing a bunch of research into steamcmd currently and I am going to figure out the best way to support workshop items across all game servers moving forward

[ToeiRei]

I am suffering from steamcmd shortcomings since more than a year while still haven't found a clean solution. To be honest, I was even thinking of replacing steamcmd with my own as running a loop hoping for a good exit code is a little bit bold.

That's why I went the external scripting way as I didn't want to modify lgsm directly. Upstream compatibility...

[HeapUnderfl0w]

I was even thinking of replacing steamcmd with my own

is that even possible? i would think steam wouldnt allow 3rd party scmd clients.

I tried the timeouts without big success.

if wanted i can also investigate a bit with mods i know fail, and see if tweaking different options does any different.

[ToeiRei]

@HeapUnderfl0w Sadly I came to the same conclusion about 3rd party.

if wanted i can also investigate a bit with mods i know fail, and see if tweaking different options does any different.

Feel free to experiment as maybe LGSM could benefit from that

[ToeiRei]

As steamcmd.sh exit codes are anything but reliable, I switched to $(steamcmd.sh....) searching its output for 'Success.' as this doesn't show up on errors.

Not the most portable way, to be honest - but way better than hoping for valve to fix steamcmd.

[dgibbs64]

I latest update to LinuxGSM can so error checking on steamcmd. So if it fails LinxuGSM will either force steamcmd to try again or for specific errors like full disk then it will quit. This should help this issue move forward.

I think it will make sense to integrate updating steam workshop stuff into update and validate. The only issue with this is that I don't know if there is a method for checking for workshop updates.

So possible solutions

1. If workshop update checking is available - integrate it into update that will check for both game server and workshop updates before taking action.

2. if no workshop update checking is available - integrate it into update with no checking. Update will only update workshop when new gameserver update available. However, workshop updates can also be done via force-update and validate which would then mean a daily restart would be recommended to keep workshop items updated.

[ToeiRei]

For those suffering the same problems we had, here's my current version of the script:

#!/bin/bash
CWD=`pwd`
GAMEID=346110

WORKSHOP=(
        "849985437"
        "895711211"
        "731604991"
        "821530042"
        "847495728"
        "679529026"
        "655261420"
        "932816570"
        "1890796153"
        "1380777369"
        "1639458270"
        "1890796153"
        "2007400172"
        "2007461356"
        "2007447056"
        "2007441758"
        "2007430597"
        "2007418454"
        "2007411835"
)

cd /home/ark/serverfiles

for i in "${WORKSHOP[@]}"
do

  while [[ $result != *"Success"* ]]
    do
      result=$(Engine/Binaries/ThirdParty/SteamCMD/Linux/steamcmd.sh +login anonymous +workshop_download_item ${GAMEID} ${i} validate +quit)
      if [[ $result == *"ERROR"* ]]; then
        echo "Error downloading mod ${i}."
      else
        echo "Mod ${i} successfully downloaded."
      fi
    done

    result="ERROR"

done

cd ${CWD}

I use the same command and binary as ark itself. Not the prettiest thing, but it works.

[ToeiRei]

Got another mod here to be added to the list of troublemakers. In the meanwhile I refined my script to work around it and read the mods from the game.ini

#!/bin/bash
CWD=`pwd`
GAMEID=346110

WORKSHOP=( $(cat Game.ini | grep ModIDS | awk -F'=' '{print $2}') )

cd /home/ark/serverfiles

for i in "${WORKSHOP[@]}"
do

  while [[ $result != *"Success"* ]]
    do
      result=$(Engine/Binaries/ThirdParty/SteamCMD/Linux/steamcmd.sh +login anonymous +workshop_download_item ${GAMEID} ${i} validate +quit)
      if [[ $result == *"ERROR"* ]]; then
        echo "Error downloading mod ${i}."
      else
        echo "Mod ${i} successfully downloaded."
      fi
    done

    result="ERROR"

done

cd ${CWD}

[HeapUnderfl0w]

I havent touched Ark in quite a while, so i also didnt really look into the steamcmd docs any more than nessesary... but your solution seems like the most sensible workaround at the moment. From what i have glanced from steamcmd itself, there is really no way to increase the timeout as almost all other games have either small enough mods (<500MB with all assets), or they have an internal downloader (which circumvents the steamcmd problem). Due to this steam also is notoriously resitant to changing it i assume.

[ToeiRei]

@HeapUnderfl0w I just had to come up with this solution due to the ignorance on the steam side. I run this script as a part of my nightly maintenance cycle just in case something did update.

[HeapUnderfl0w]

yea i assume that is the best way forward for now. ima actually pilfer that script and add it to my toolbox of "random fixes".

[ToeiRei]

Mind sharing your toolbox as we could come up with some improved version?

[HeapUnderfl0w]

Its just a collection of scripts to administer different game servers, nothing really what will help here i think