No "Installed Games" & "Server Games" after host + application reboot

[casudo]

Describe the bug Yet another issue (?) I've encountered today. I've restarted my Proxmox host, in which my Ubuntu VM is running GameVault Server as Docker container.

When I opened GameVault on my Windows 10 machine, it showed 0 installed games (despite having one game installed via GameVault) and 0 server games (I have 15 server games which I verified are also mounted inside the Docker container at /files). Also, the download tab is empty.

/files inside the GameVault Server:

GameVault Server log lines as soon as I start GameVault on my desktop:

As you can see, it also found those 15 games, but won't display them in the GUI.

I did not tinker with any file or folder permissions. I've just restarted my host system.

To Reproduce Steps to reproduce the behavior:

1. Restart the whole Docker host

Expected behavior The installed and server games should be shown correctly in the GUI.

Screenshots See above.

Desktop (please complete the following information):

• Windows 10

[Alfagun74]

Show me your docker compose file please

[casudo]

Show me your docker compose file please

version: "3"

networks:
  database:
    external: true

services:
  gamevault:
    container_name: gamevault
    image: phalcode/gamevault-backend:latest
    restart: unless-stopped
    ports:
      - 8088:8080
    volumes:
      - /mnt/arrmedia/games_nfs:/files
      - /mnt/docker/gamevault/images:/images
    environment:
      - TZ=Europe/Berlin
      - DB_HOST=postgresql # PostgreSQL Container Name
      - DB_USERNAME=redacted # PostgreSQL Username
      - DB_PASSWORD=redacted # PostgreSQL Password
      - DB_NAME=gamevault # Gamevault's PostgreSQL Database Name
      - SERVER_ADMIN_USERNAME=redacted # Gamevault Admin Username
    hostname: gamevault
    networks:
      - database

[Alfagun74]

Are you sure the volume of your postgres database is mounted correctly, so the data persists after a restart? because it seems like the data of the database resets honestly. Also try clearing the cache of your client and press F5 on the library so it reloads. Thanks in Advance

[casudo]

Are you sure the volume of your postgres database is mounted correctly, so the data persists after a restart? because it seems like the data of the database resets honestly. Also try clearing the cache of your client and press F5 on the library so it reloads. Thanks in Advance

Seems to be correct honestly. In PGAdmin4 I can see Gamevault session to the database. Also the "game" table is filled correctly with the game's I've added 3 days ago. What i am missing here are 3 more games I've added yesterday, when GameVault was still working properly. Also, what is the "deleted_at" timestamp? :D

[casudo]

UPDATE: I may found out why the "deleted_at" timestamp is filled. I think when the container rebooted, the NFS shares wasn't (correctly) mounted to the host yet - this might be why GameVault thought the game was deleted. I've now manually deleted the timestamp in the "deleted_at" row and now my 12 games are showing in the "Server Game" tab.

Now I am still missing my games in the "Installed Games" tab, aswell as the 3 new games I've added yesterday.

UPDATE 2: I've managed to get the installed game back by just re-downloading it (no extraction). This wouldn't be a great workaround tho if I had many and big games installed.

[casudo]

There might be a problem with the "game" table in Postgres. I've tried to add a new user in the GameVault GUI to see if it writes to the GameVault DB, which it does - I can see the new user I've created in the "gamevault_user" table.

However, it seems like it doesn't update the "game" table at all. I can see in the backend logs, that it indeed found 4 more games (12 in the PostgreSQL table):

I also tried to update games with their version and release date that are already in the table like this. mv Ghostrunner\ 2\ \[Brutal\ Edition\]\ \(v0.40570.441\).zip Ghostrunner\ 2\ \[Brutal\ Edition\]\ \(v0.40570.441\)\ \(2023\).zip and mv Gunfire\ Reborn\ \(2021\).zip Gunfire\ Reborn\ \(v420.69\)\ \(2021\).zip

However, the game wasn't updated:

[Alfagun74]

Can you try manually indexing through the admin panel? It seems like your GameVault servers filewatcher is not able to pick up these changes. If that helps lower the old school index timer via environment variable. Set it to about 5-10 minutes. You said you use a network share as the files root? I think that may be the cause of the problem. Why not host the server where the files are?

[Alfagun74]

GAMES_INDEX_INTERVAL_IN_MINUTES=5

[casudo]

Can you try manually indexing through the admin panel? It seems like your GameVault servers filewatcher is not able to pick up these changes. If that helps lower the old school index timer via environment variable. Set it to about 5-10 minutes. You said you use a network share as the files root? I think that may be the cause of the problem. Why not host the server where the files are?

Manually reindexing the games doesn't work unfortunately.

It always worked great with the NFS share, so I keep them separately due to safety reasons. It's probably something with the filewatcher because I can see all games (even the ones where I only added the version or release date) just fine under the /files folder inside of the GameVault Server.

[casudo]

I've now tried it with a whole new database starting from scratch. I can successfully register a new user and in the server logs I can see that it found 16 games.

Something seems strange with the filewatcher or the "game" table, because somehow GameVault Server cannot add a single entry to this table.

[Alfagun74]

Can you try an older server version?

[casudo]

I've tried the backend version 10.1.0 with the frontend version 1.8.2 but got the same result unfortunately.

[Alfagun74]

Can you please set these env vars

SERVER_LOG_LEVEL=debug 
DB_DEBUG=true

Switch to the latest versions again, Restart the server, Reproduce the error and send in a full log?

I will analyze it in the upcoming weeks as I am currently in vacation.

Greetings from Japan 🍙🎎

[Alfagun74]

The log I see is only partial, could you send in the full one please?

[Alfagun74]

Could you also attach a csv of the game table in its current state?

[casudo]

@Alfagun74 I've probably found the problem, will update this issue later to confirm my speculation. You don't need to work through the logs. ;)

[casudo]

Alright, I've managed to troubleshoot the problem... aaaand it was as simple as that -> the archives were password protected..

Therefore the filewatcher recognized all files correctly, but could not write to the DB because of the password. I noticed that Gamevault tries to write new games alphabetically into the DB. Since the game that is stored in the password-protected archive starts with “A”, the filewatcher crashed (?) there directly and did not skip the game. This could be implemented in the future.

Three further comments that I noticed during my troubleshooting:

• Verify download before being able to extract the game. In some cases, the download was not complete. The extraction then ran into an error. EDIT: Might be also implemented, in my case the .zip file was already not completely packed.
• Automated Game Box indexing didn't work in a single case. I always needed to manually add them.
• As I've mentioned earlier, I've updated “Gunfire Reborn (2023).zip” to “Gunfire Reborn (2023) (v420.69).zip”. I thought that the Filewatcher would simply update the game, but it created a brand a new database entry, resulting in two server games of Gunfire Reborn.

[Alfagun74]

Hi @casudo

First of all, password protected archives are a very new feature for GameVault, so thank you for finding this!

Could you please tell me how to create a file to reproduce this behavior? For example, what exact name, tool, format, and password can lead to this problem?

Regarding your other points:

• The download is verified by counting bytes, there is no hashing, although it might be a good idea to implement something like that.

• The Game Box Image Scraper is indeed broken at the moment, you can try using our temporary Rawg-2-Steam solution to have a temporary fix that gets all box art if youre usually playing PC games available on Steam.

• This behavior is intended. I cannot "track" a file at this moment, once the path or title and release year changes GameVault just soft-deletes the old entry and makes a new one, we should probably implement some sort of tracking system here, but I cannot think of a way that would be convenient. Using IDs in filenames for example seems too forcing on the user, hashes and bytesize will change with things like patches, and the name of the file is one thing that is likely to change if the user decides to move it or something. So we really need a way to "tag" a file so it gets tracked or something. I have no idea yet. Also, the games are not alphabetized by choice, it is just the way the files are scanned from top to bottom.

[casudo]

Could you please tell me how to create a file to reproduce this behavior? For example, what exact name, tool, format, and password can lead to this problem?

Normally I download and zip files on my server directly, however, I downloaded those two "faulty" games on my Windows PC and packed them with WinRAR into a .rar archive. I then copied them over to my GameVault server with WinSCP and then renamed the archvie from .rar to .zip to make it standardized - maybe this was a also part of the problem.

[Alfagun74]

Sounds like bad practice for me 😂

[Alfagun74]

But I meant how to reproduce the encrypted archive error! 😊

[casudo]

But I meant how to reproduce the encrypted archive error! 😊

Good question. 😅

Try to add a password-protected archive and a normal one which initial letter comes after the one with the password.

[Alfagun74]

Hi @casudo, im sorry but i cannot reproduce the issue.

I tried creating 2 archives via sevenzip, one password protected, and both populated fine.

Can you provide me a small encrypted file that produces this error?

[Alfagun74]

Moved into #297