ich777 / docker-fivem-server

Docker that installs FiveM and updates on every restart
https://fivem.net/
39 stars 10 forks source link

When using with txAdmin, invoking a restart from txAdmin stops the container #9

Closed xhorntail closed 3 months ago

xhorntail commented 4 months ago

Typically on a bare metal install, txAdmin offers the ability to reset the fxServer while maintaining the txAdmin service to view startup logs. When hitting the restart button in txAdmin with this docker, the container simply terminates.

ich777 commented 4 months ago

A easy workaround would be to use the Docker flag --restart=unless-stopped this will ensure that the container restarts, that‘s also why it‘s in the run example.

ich777 commented 4 months ago

Thanks, I saw that I don't have it in this run example, I will add it. :) Can this be closed now?

xhorntail commented 4 months ago

Yep!

xhorntail commented 4 months ago

Sorry, I was wrong to say this should be closed :/ There's still an issue here. Usually, txAdmin is persistent above the fxServer. It seems whenever the fxServer process hangs or restarts, the container stops.

ich777 commented 4 months ago

Can you please be a bit more precise?

The container can‘t stop on it‘s own if you are using --restart=unless-stopped since the container must restart.

It would be very unlikely or better speaking impossible that the container is stopping when you are issuing a stop (from within the container) or even if the application in the container is stopping/crashing. The container will only stop if you are actively tell Docker to stop the container.

xhorntail commented 4 months ago

Yes, my apologies, I should say that it crashes or halts. It does indeed restart afterwards. I believe the same issue also crashes the container when the CFX webpack resource takes too long to pack a resource.

If I were to speculate, I would wager it has something to do with the term_handler in the start.sh script. I believe that fxServer utilizes a virtualization of an Alpine Linux install, perhaps txAdmin is issuing something to that virtualized layer that's being caught by the start.sh script? This wouldn't cover the webpack issue, however..

ich777 commented 4 months ago

I should say that it crashes or halts. It does indeed restart afterwards. I believe the same issue also crashes the container when the CFX webpack resource takes too long to pack a resource.

I now really don‘t understand… What is the exact issue? When does the issue happen? Does it happen all the time? Please lets discuss issue by issue and not speculate…

I would wager it has something to do with the term_handler in the start.sh script

No, this will only kick in when the container receives a restart or stop command from Docker itself.

xhorntail commented 4 months ago

Sure.

Desired result (to parity bare metal)

Additionally

ich777 commented 4 months ago
  • In txAdmin, press the restart button
  • Container crashes

This procedure is not a crash and expected, at least I assume since you are ending the main process in the container (wich is not txAdmin).

  • Click restart server button in txAdmin
  • txAdmin handles the restart of fxServer
  • txAdmin remains available, and login persists
  • fxServer comes back online

This is not that easy doable in Docker because the main process in Docker is fxServer not txAdmin and every time the main process ends the container will restart, so to speak these behaviour aren‘t crashes, this expected to happen wehn you are dealing with Docker. That‘s simply how Docker is working and nothing I can do about.

  • Alter a resource that is webpacked through CFX's webpack resource, to force repacking (as an example, the CFX default chat resource.)
  • If webpack takes too long to package the resource (longer than 10-15 seconds it seems, this is difficult to measure) the container crashes
  • Container restarts, and attempts to pack the next resource that needs to be webpack'd
  • Again, if this takes too long, container crashes
  • Eventually, all resources are packed (unsure if they're correctly packed, or if this damages the packing) and txAdmin becomes accessible, along with fxServer becoming stable
  • Happens every time

Sorry I‘m not super familiar with the game server anymore since I haven‘t used it in a long time but this shouldn‘t happen since as if the main process is running, the container will stay up and won‘t restart. Is it possible that fxServer is restarting after packing up a resource and therefore you experiencing these „crashes“ <- I actually don‘t think that these are crashes, what does the log say?

xhorntail commented 4 months ago

Which log?

ich777 commented 4 months ago

Which log?

The log from the container itself so to speak the Docker log, may I ask on what hardware and on what OS do you run the cotnainer? This is the first time I hear of these kind of issues.

xhorntail commented 4 months ago

Well, I can't post the full logs as they would take quite some time to cleanse of identifying information due to the amount of debugging I have enabled on the server, but here's a snapshot of the event. image

This is running through Unraid. Docker never reaches anything close to its provisioned limits.

ich777 commented 4 months ago

Well, I can't post the full logs as they would take quite some time to cleanse of identifying information due to the amount of debugging I have enabled on the server

No worries, you can upload the full log as .txt file here and I go through it.

However the Terminated line is a good sign that the main process in the container is enended and that‘s why the container is restarting.

I don‘t know how tepackaging works on txAdmin because I only tested the base functionallity. Could it be also the case that the main process is crashing because something goes wrong while repacking (seems like it‘s compiling something not repacking?).

Another possibility would be rhat some dependency (package) is missing in the container but I‘m not sure about that.

xhorntail commented 4 months ago

This is part of the standard fxServer, not necessarily the txAdmin. Either way, it's wrongfully using the word "compiling". It's actually running webpack on a resource that uses Typescript, so it is technically packaging/transpiling JS.

While you've asked me not to speculate, I fear that I'll have to do so in order to respond to some of the questions you've posed, so please be understanding of that.

I don't think this is a dependency or resolution issue, as the resources end up packing after several container restarts.

As I understand it, fxServer manages this through its inbuilt V8 engine, which appears to be one of the higher order components in the stack. Here's an alternative entrypoint file that loads the stack differently than just running the standard run.sh file that is ultimately being called in this repo's image. I'm not certain that's the reason the alternative container doesn't exhibit the issues discussed here, but it seems to be a reasonable deduction that it's a contributor to the fact, at minimum. (please remove the link if that's not allowed. In which case, my appologies.)

ich777 commented 4 months ago

While you've asked me not to speculate, I fear that I'll have to do so in order to respond to some of the questions you've posed, so please be understanding of that.

No worries about that, I just wanted to point that out above and didn't want to sound offensive. :)

Here's an alternative entrypoint file

From what I can see it does nothing different than what the basic run.sh does:

#!/bin/bash

# save the script directory
SCRIPT=$(readlink -f "$0")
SCRIPTPATH=$(dirname "$SCRIPT")

# run server
exec $SCRIPTPATH/alpine/opt/cfx-server/ld-musl-x86_64.so.1 \
    --library-path "$SCRIPTPATH/alpine/usr/lib/v8/:$SCRIPTPATH/alpine/lib/:$SCRIPTPATH/alpine/usr/lib/" -- \
    $SCRIPTPATH/alpine/opt/cfx-server/FXServer +set citizen_dir $SCRIPTPATH/alpine/opt/cfx-server/citizen/ $*

However, my container is designed to watch for ld-musl-x86_64.so.1, if that process is ended the container will restart or stop (depending if --restart=unless-stopped is used or not), please not that this is independent if run.sh is actually running or not.

So my best guess is that ld-musl-x86_64.so.1 is ended and therefore the container restarts in your case. However from what I can see, the other container should also behave the same as mine.

BTW, if you are running Unraid you could also install FiveM into a LXC container but of course then the installation and updates are completely up to you but you wouldn't suffer from such issues <- don't know if that's a viable solution for you, this came just to mind. :)

ich777 commented 4 months ago

Any update on this @xhorntail ?

ich777 commented 3 months ago

Closing due to no further response.