[Bug]: Rust Oxide Error with tmux in later debian and ubuntu builds

[austinv900]

User story

As a server owner, I want to be able to run OxideMod so that I can mod my server

Game

Rust

Linux distro

Other

Command

command: start

Further information

Hello, I'm a core developer over at OxideMod my main responsibility is fixing and updating the C# compiler that is used to compile plugins at runtime. We've had a couple users bring up that the compiler is failing to start so I've done some digging and have been able to reproduce it on my side.

To be perfectly honest I'm not too heavy into Linux so please correct me if I'm mistaken. I'm finding that LGSM's terminal emulator of choice ( "tmux" ) is causing issues with mono being able to access any console resources of sub processes.

The way the parent process communicates with the compiler process is its redirected I/O stream via the console and for some reason everytime tmux is set as the terminal emulator for the parent process we get a error.

TypeInitializationException: Exception while starting compiler (Exception: Magic number is wrong: 542) at System.TermInfoReader.ReadHeader (System.Byte[] buffer, System.Int32& position) [0x00028] in <fb001e01371b4adca20013e0ac763896>:0 at System.TermInfoReader..ctor (System.String term, System.String filename) [0x0005f] in <fb001e01371b4adca20013e0ac763896>:0 at System.TermInfoDriver..ctor (System.String term) [0x00055] in <fb001e01371b4adca20013e0ac763896>:0 at System.ConsoleDriver.CreateTermInfoDriver (System.String term) [0x00000] in <fb001e01371b4adca20013e0ac763896>:0 at System.ConsoleDriver..cctor () [0x0004d] in <fb001e01371b4adca20013e0ac763896>:0

All my googling tells me to run export TERM=xterm before launching the game server. I haven't been able to actually do that while utilizing LGSM. Running the game server directly everything runs fine.

While I've only tested this on a Rust server I'm sure the issue exists on all the other games supported by oxide and LGSM.

Relevant log output

03:48 [Info] [DEBUG] [CSharp] [4] Starting compiler with parameters: -unsafe true --setting:Force true -ms true --parent 18639 -l:file compiler_2023-07-20.log -v Debug
03:48 [Error] ExType: TypeInitializationException
03:48 [Error] [DEBUG] [CSharp] [4] Exception while starting compiler (Exception: Magic number is wrong: 542)
  at System.TermInfoReader.ReadHeader (System.Byte[] buffer, System.Int32& position) [0x00028] in <fb001e01371b4adca20013e0ac763896>:0 
  at System.TermInfoReader..ctor (System.String term, System.String filename) [0x0005f] in <fb001e01371b4adca20013e0ac763896>:0 
  at System.TermInfoDriver..ctor (System.String term) [0x00055] in <fb001e01371b4adca20013e0ac763896>:0 
  at System.ConsoleDriver.CreateTermInfoDriver (System.String term) [0x00000] in <fb001e01371b4adca20013e0ac763896>:0 
  at System.ConsoleDriver..cctor () [0x0004d] in <fb001e01371b4adca20013e0ac763896>:0 
03:48 [Error] [DEBUG] [CSharp] [4] Compiler may not be set as executable; chmod +x or 0744/0755 required
03:48 [Error] [DEBUG] [CSharp] [4] BaseException:  (Exception: Magic number is wrong: 542)
  at System.TermInfoReader.ReadHeader (System.Byte[] buffer, System.Int32& position) [0x00028] in <fb001e01371b4adca20013e0ac763896>:0 
  at System.TermInfoReader..ctor (System.String term, System.String filename) [0x0005f] in <fb001e01371b4adca20013e0ac763896>:0 
  at System.TermInfoDriver..ctor (System.String term) [0x00055] in <fb001e01371b4adca20013e0ac763896>:0 
  at System.ConsoleDriver.CreateTermInfoDriver (System.String term) [0x00000] in <fb001e01371b4adca20013e0ac763896>:0 
  at System.ConsoleDriver..cctor () [0x0004d] in <fb001e01371b4adca20013e0ac763896>:0

Steps to reproduce

1. Start a fresh distro of Ubuntu 23.04
2. Run apt update and apt upgrade
3. Reboot
4. Install LGSM dependencies for sudo dpkg --add-architecture i386; sudo apt update; sudo apt install curl wget file tar bzip2 gzip unzip bsdmainutils python3 util-linux ca-certificates binutils bc jq tmux netcat lib32gcc-s1 lib32stdc++6 libsdl2-2.0-0:i386 steamcmd lib32z1 netcat doesn't have a valid install candidate for 23.04 so I substituted it for netcat-traditional
5. Add the rustserver user
6. Switch to the rust server user
7. wget -O linuxgsm.sh https://linuxgsm.sh && chmod +x linuxgsm.sh && bash linuxgsm.sh rustserver
8. Run the install script using ./rustserver install
9. Download the latest oxidemod from the umod website and extract the files into serverfiles directory
10. Launch the server once and stop the instance.
11. Add any rust or covalence plugins it really doesn't matter.
12. Start the server again.
13. You'll notice all the plugins failed do to unable to start compiler. "You need a debug version of Oxide.CSharp.dll to get more output. I can provide this if needed or you can compile your own"

[dgibbs64]

Interesting problem. I am not much of a tmux expert. However my guess is that the term type is not being set to xterm in the tmux session normally you can get the term type by running echo $TERM. If this is the case then I think I should be able to force the terminal type in the tmux session. Since this is only effecting newer versions of debian and Ubuntu it migh tbe related to using a newer version of tmux

edit: From the tmux changelog...might be relevant https://github.com/tmux/tmux/blob/master/CHANGES

• When building, pick default-terminal from the first of tmux-256color, tmux, screen-256color, screen that is available on the build system (--with-TERM can override).

[dgibbs64]

Just did a quick test on debian 12 The default term type for Debian is xterm-256color The default term type or tmux is tmux-256color

[austinv900]

Just did a quick test on debian 12 The default term type for Debian is xterm-256color The default term type or tmux is tmux-256color

I've only personally tested it myself on Ubuntu 23.04 I believe it's the same on there too 'xterm-256color' but gets forced to tmux-256color for the game process when launching with LGSM.

[dgibbs64]

Just did more checks for tmux term types. So I am fairly confident tmux-256color is causing the issue. I will look at forcing the term type back to screen

Ubuntu 22.04 term = screen
Debian 11 term = screen
Debian 12 term = tmux-256color

[austinv900]

Just did more checks for tmux term types. So I am fairly confident tmux-256color is causing the issue. I will look at forcing the term type back to screen

Ubuntu 22.04 term = screen
Debian 11 term = screen
Debian 12 term = tmux-256color

We appreciate it.

[joeyrutledge2]

Running into this issue currently with LGSM and fresh install of Debian 12. Is there a simple workaround that I can do to get plugins working? Currently Oxide installs, but anytime I load a plugin it fails with errors as austin listed initially in this thread.

[austinv900]

Running into this issue currently with LGSM and fresh install of Debian 12. Is there a simple workaround that I can do to get plugins working? Currently Oxide installs, but anytime I load a plugin it fails with errors as austin listed initially in this thread.

Your gonna want to run a older version of Linux until there's a fix or workaround.

[joeyrutledge2]

think I might have figured it out. in ~rustuser/.profile I added

export TERM=screen

and in ~rustuser/.tmux.conf I added

set -g default-terminal "screen"

logged out and back in, restarted rust using ./rustserver restart and plugins loaded. I did this on my dev box (also Debian 12, but slow slow slow) but haven't tried on my production box as folks are still playing atm. EDIT: The same fix worked on my production machine as well.

Pretty sure the fix here was the .tmux.conf but the .profile was added as good measure.

[tumekee]

@joeyrutledge2 be interested to know if you can run RCON commands, specifically the “kick” command. I did what you did and was able to log in and play on the server but some RCON commands gave a magic number error ( and didn’t work )

[joeyrutledge2]

@tumekee yep just confirmed that RCON commands work for me. no errors on the kick command at all.

[AhrimFakhriy]

First of all, thank you for this, definitely saved me here @joeyrutledge2 However I had to rename the ~/.tmux.config file to ~/.tmux.conf in order for it to works.

Again, thanks!

[joeyrutledge2]

@AhrimFakhriy glad to help! I found it odd that this issue had been sitting for a bit without a published solution.

However I had to rename the ~/.tmux.config file to ~/.tmux.conf in order for it to works.

doh good call out. Mistyped it when I wrote up my comment here. I’ll edit my previous post.

[dgibbs64]

Relevant comment https://github.com/Homebrew/homebrew-core/issues/102748#issuecomment-1145378599

The summary, the issue is that for tmux 3.3a the new default term for tmux is now tmux-256color rather than screen-256color because it is meant to have more features than the screen TERM. Oxide breaks when tmux-256color is used.

I think the ideal solution would be for oxide to release a bug fix to allow it to work with tmux-256color. If this is not feasible I will need to add code to insert set -g default-terminal "screen-256color" into ~/.tmux.conf for Rust servers. My preference is to not need to change the default behavior of tmux as users might be using tmux for other purposes so I want to avoid editing a users tmux config.

I have also looked at if there is a way to change the term when starting a new session but haven't found a way yet.

@austinv900 please let me know what you think about the feasibility of OxideMod patching the tmux-256color term issue and if so what are your thoughts on timescale etc. If you think it is either unlikely or going to take a while I will update LinuxGSM with the workaround. Thanks :)

edit jsut seen this commit https://github.com/OxideMod/Oxide.Core/commit/debd6cbb6899d4299d1c7ca8db05b9b01024ff32 as a temp warning to users.

[austinv900]

To be perfectly honest I'm not to sure where to start with creating a bug fix. I feel like the issue might be higher than .NET as our compiler process just fails to start if we have the Std In/Out redirected. Only thing I can think of is adding in a alternate way to communicate with the compiler process. Maybe someone else has some insight that might help? Alternativly we can do what you'd do and just write to the tmux config file and force a restart to the process but that might cause a headache of complaints.

[austinv900]

Relevant comment https://github.com/Homebrew/homebrew-core/issues/102748#issuecomment-1145378599

The summary, the issue is that for tmux 3.3a the new default term for tmux is now tmux-256color rather than screen-256color because it is meant to have more features than the screen TERM. Oxide breaks when tmux-256color is used.

I think the ideal solution would be for oxide to release a bug fix to allow it to work with tmux-256color. If this is not feasible I will need to add code to insert set -g default-terminal "screen-256color" into ~/.tmux.conf for Rust servers. My preference is to not need to change the default behavior of tmux as users might be using tmux for other purposes so I want to avoid editing a users tmux config.

I have also looked at if there is a way to change the term when starting a new session but haven't found a way yet.

@austinv900 please let me know what you think about the feasibility of OxideMod patching the tmux-256color term issue and if so what are your thoughts on timescale etc. If you think it is either unlikely or going to take a while I will update LinuxGSM with the workaround. Thanks :)

edit jsut seen this commit https://github.com/OxideMod/Oxide.Core/commit/debd6cbb6899d4299d1c7ca8db05b9b01024ff32 as a temp warning to users.

Feel free to message me on discord for a faster response. I'm sure we can probably hammer out of details if you feel it would be better trying to add a patch into oxide

[dgibbs64]

@austinv900 is there anything in particular info you need from me? I have no knowledge of how Oxide works so I don't think I will be much use apart from doing some testing. But yeah attempting to patch Oxide I think is preferable in the first instance :)

[austinv900]

@austinv900 is there anything in particular info you need from me? I have no knowledge of how Oxide works so I don't think I will be much use apart from doing some testing. But yeah attempting to patch Oxide I think is preferable in the first instance :)

We just have no understanding of why tmux is causing issues. We get that exception and all I can understand from that is .NET is getting a unexpected response from the terminal. It's doesn't look like it has anything to do with oxide. Maybe it's a mono issue?

[blackuGT]

think I might have figured it out. in ~rustuser/.profile I added

export TERM=screen

and in ~rustuser/.tmux.conf I added

set -g default-terminal "screen"

logged out and back in, restarted rust using ./rustserver restart and plugins loaded. I did this on my dev box (also Debian 12, but slow slow slow) but haven't tried on my production box as folks are still playing atm. EDIT: The same fix worked on my production machine as well.

Pretty sure the fix here was the .tmux.conf but the .profile was added as good measure.

For me this is not working. Fresh Debian 12 and LinuxGSM instalation, added export to .profile and created .tmux.conf with default-terminal option and still when starting server and invoke any oxide command:

Exception while starting compiler version : (Exception: Magic number is wrong: 542) at System.TermInfoReader.ReadHeader (System.Byte[] buffer, System.Int32& position) [0x00028] in :0 at System.TermInfoReader..ctor (System.String term, System.String filename) [0x0005f] in :0 at System.TermInfoDriver..ctor (System.String term) [0x00055] in :0 at System.ConsoleDriver.CreateTermInfoDriver (System.String term) [0x00000] in :0 at System.ConsoleDriver..cctor () [0x0004d] in :0 Compiler may not be set as executable; chmod +x or 0744/0755 required

TERM variable is looking fine:

echo $TERM screen

Running process:

tmux -L rustserver new-session -d -x 80 -y 23 -s rustserver ./RustDedicated -batchmode +app.listenip 0.0.0.0 +app.port 28082 +server.ip 0.0.0.0 +server.port 28015 +server.queryport 28017 +server.tickrate 30 +server.hostname "[Redacted]" +server.identity "rustserver" +server.gamemode vanilla +server.level "Procedural Map" +server.seed 674000918 +server.salt 0 +server.maxplayers 150 +server.worldsize 4000 +server.saveinterval 300 +rcon.web 1 +rcon.ip 0.0.0.0 +rcon.port 28016 +rcon.password "[Redacted]" -logfile

When launching server in debug mode all is working really fine.

[dgibbs64]

@austinv900 is there anything in particular info you need from me? I have no knowledge of how Oxide works so I don't think I will be much use apart from doing some testing. But yeah attempting to patch Oxide I think is preferable in the first instance :)

We just have no understanding of why tmux is causing issues. We get that exception and all I can understand from that is .NET is getting a unexpected response from the terminal. It's doesn't look like it has anything to do with oxide. Maybe it's a mono issue?

Seems like a tough one. Could be a bug with the tmux terminal or .net. I think it's worth raising an issue with the Tmux developer providing the unexpected response error. They will definitely understand way more than I ever could

https://github.com/tmux/tmux