System.UnhandledExceptionEventArgs "crash" when run as Linux Daemon

[wnewbery]

When running as a Linux daemon process I get this error and a crash log on every start. In my initial testing it didnt actually terminate the process and I was able to join the server. stdin will be opened to /dev/null in this configuration.

Running just the mono command results in no error, so I am guessing the problem is not having a terminal, rather than no stdin / stdout.

Using TShock Mintaka 4.3.24 for Terraria 1.3.5.3.

$ daemon --name=server --respawn --attempts=1 --limit=1 \
  --pidfile=/home/will/terraria-test/server.pid \
  --chdir=/home/will/terraria-test \
  --output=/home/will/terraria-test/server.log \
  --env="MONO_THREADS_PER_CPU=50" \
    -- mono /home/will/terraria-test/TerrariaServer.exe \
    -config /home/will/terraria-test/config.conf

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial

$ mono --version
Mono JIT compiler version 4.2.1 (Debian 4.2.1.102+dfsg2-7ubuntu4)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        LLVM:          supported, not enabled.
        GC:            sgen

TerrariaAPI Version: 2.1.0.0 (Protocol v1.3.5.3 (194), OTAPI 1.3.5.3)
TShock was improperly shut down. Please use the exit command in the future to prevent this.
TShock 4.3.24.0 (Mintaka) now running.
AutoSave Enabled
Backups Disabled
Welcome to TShock for Terraria. Initialization complete.
[Server API] Info Plugin TShock v4.3.24.0 (by The TShock Team) initiated.
Terraria Server v1.3.5.3

Resetting game objects 1%
...
Settling liquids 98%
Starting server...
Server started
Terraria Server v1.3.5.3

Listening on port 2000
Type 'help' for a list of commands.

TShock Notice: authcode.txt is still present, and the AuthToken located in that file will be used.
To become superadmin, join the game and type /auth 5042611
This token will display until disabled by verification. (/auth)
: Unhandled exception
System.UnhandledExceptionEventArgs
[Server API] Error ===================================================================================
[Server API] Error An unhandled exception has occured in TSAPI, and a crash report will be generated
[Server API] Error Generating a crash report, please wait...
[Server API] Error Crash report saved at crash_131480850184759850.zip
[Server API] Error Please upload the crash file and report it at http://tshock.co/
[Server API] Error The process will terminate.
[Server API] Error ===================================================================================
Saving world. Momentary lag might result from this.
Saving world data: 7%
Saving world data: 17%
Saving world data: 24%
Saving world data: 33%
Saving world data: 40%
Saving world data: 50%
Saving world data: 56%
Saving world data: 65%
Saving world data: 71%
Saving world data: 81%
Saving world data: 88%
Saving world data: 97%
Saving world data: 100%
Validating world save: 59%
World saved.
Backing up world file
192.168.1.234:57367 is connecting...
Test has joined.

{
  "os": {
    "x64": "yes",
    "x64Process": "yes",
    "Platform": "Unix",
    "Version": "Unix 4.4.0.92"
  },
  "hardware": {
    "CPUs": 1,
    "CPUID": null
  },
  "server": {
    "Players": 0,
    "MaxPlayers": 255,
    "WorldName": "World",
    "WorldFile": "/home/will/terraria-test/worlds/World.wld",
    "Time": 13500.0
  },
  "process": {
    "curRelease": 194,
    "TSAPIVersion": "1.3.5.3",
    "Uptime": "00:00:04.2379870",
    "WorkingDirectory": "",
    "Arguments": ""
  },
  "memory": [],
  "plugins": [
   {
     "Name": "TShock",
     "Author": "The TShock Team",
     "Version": "4.3.24.0"
   }
  ],
  "exception": {
    "message": "Object reference not set to an instance of an object",
    "type": "NullReferenceException",
    "trace": "  at Terraria.Main.startDedInputCallBack (System.Object threadContext) <0x40672f40 + 0x0013c> in <filename unknown>:0 \n  at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context (System.Object state) <0x7fae624b5720 + 0x00048> in <filename unknown>:0 \n  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, Boolean preserveSyncCtx) <0x7fae624af850 + 0x0016e> in <filename unknown>:0 \n  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, Boolean preserveSyncCtx) <0x7fae624af820 + 0x00020> in <filename unknown>:0 \n  at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () <0x7fae624b56b0 + 0x00053> in <filename unknown>:0 \n  at System.Threading.ThreadPoolWorkQueue.Dispatch () <0x7fae624b3cc0 + 0x001d6> in <filename unknown>:0 \n  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () <0x7fae624b5530 + 0x00008> in <filename unknown>:0 ",
    "inner": null
  }
}

[chenshuiluke]

Same problem here 😢 with systemd

[chenshuiluke]

I managed to stop tshock from crashing by adding the stdin and stdout options in the systemd service file:

[Unit]
Description=tshock

[Service]
Restart=always
# needed because tshock crashes when no stdin
StandardInput=tty
# Hijack tty11 for this purpose
TTYPath=/dev/tty11
# all output should stay in the journal though
StandardOutput=journal
WorkingDirectory=/home/luke/tshock/tshock
ExecStart=/usr/bin/mono /home/luke/tshock/TerrariaServer.exe -world '/home/luke/.local/share/Terraria/Worlds/Luke_Server.wld'

[Install]
WantedBy=multi-user.target

this link provided the insight necessary `

[wnewbery]

Yeah you can also give it a terminal using screen or such as well, but at best that is a hack (and needs some more work with systemd to make it detect status, restart, etc. correctly). I the case of /dev/ttynn conflicts are possible, but I guess its OK if you just running one server on a system, there is 64 to go through but some things might take one dynamically.

Really though, I think TShock should not be needing a full terminal, just plain stdin or signals would be enough. Really the only thing I care about running as a daemon is a simple graceful shutdown, which I think that config you provided is missing as well (at least on my system, I didn't appear to get a graceful shutdown with save etc. when systemd tries to stop it)

[Ihatetotank]

I'm really interested in the process you guys used to get a daemon setup. Currently I am trying to start Tshock with a single line of code: sudo MONO_THREADS_PER_CPU=50 mono ~/tshock/TerrariaServer.exe -world "~/My Games/Terraria/Worlds/Ubuntu_Server.wld"

If you have any insight into how to set the server up the way you did, it would be greatly appreciated!

[SignatureBeef]

I'm thinking this issue is related to #1450 as it is throwing an error in Terraria.Main.startDedInputCallBack I have made a PR for this over at the TSAPI repository - it needs testers ;)

[Ihatetotank]

I'd be happy to help you test this, but I'd have no idea how to implement this!

[chenshuiluke]

@Ihatetotank It's been a while since I did it, but I basically create a systemd service which automatically starts the server anytime my digitalocean server boots up.

First, you gotta install the mono-complete Ubuntu package with sudo apt-get install mono-complete, which should create /usr/bin/mono

Assuming your machine also has systemd, you can just create a service file in /etc/systemd/system called terraria.service, resulting in /etc/systemd/system/terraria.service and insert the following file:

[Unit]
Description=terraria

[Service]
Restart=always
# needed because tshock crashes when no stdin
StandardInput=tty
# Hijack tty11 for this purpose
TTYPath=/dev/tty11
# all output should stay in the journal though
StandardOutput=journal
WorkingDirectory=/home/luke/tshock/tshock
ExecStart=/usr/bin/mono /home/luke/tshock/TerrariaServer.exe -world '/home/luke/.local/share/Terraria/Worlds/Tektron_World.wld'

[Install]
WantedBy=multi-user.target

Then, you need to make a few modifications to the service file, make systemd aware of your service, make it auto start and then start it up:

1. Replace the part after the = in WorkingDirectory=/home/luke/tshock/tshock with where you extracted tshock + the tshock subdirectory. I extracted the tshock zip file to /home/luke/tshock/ (which contains TerrariaServer.exe) and there's a tshock subdirectory, hence /home/luke/tshock/tshock.

2. Replace /home/luke/tshock/TerrariaServer.exe with the location of your TerrariaServer.exe and /home/luke/.local/share/Terraria/Worlds/Tektron_World.wld with the location of your world.

3. Run sudo systemctl daemon-reload to make it pick up the new terraria.service file.

4. Run sudo systemctl enable terraria to make it start on bootup

5. Run sudo systemctl start terraria to run it.

6. Run sudo systemctl status terraria to see the status.

Hopefully, you're 💯 . I'm not too good at writing tutorials, so I hope it's clear.

[QuiCM]

Closing. Reopen if the above did not work