search

truecharts / public

Community Helm Chart Repository
https://truecharts.org
GNU Affero General Public License v3.0
1.16k stars 616 forks source link

RomM Stuck at "Deploying 2/3" After Latest Update #15860

Closed csudderth closed 11 months ago

csudderth commented 11 months ago

App Name

RomM

Operating System

TrueNAS SCALE 22.12.4.2

App Version

2.1.0_3.0.2

Application Events

2023-12-07 15:51:41 Back-off restarting failed container
2023-12-07 15:51:35 Created container romm
2023-12-07 15:51:35 Started container romm
2023-12-07 15:51:34 Started container romm-system-redis-wait
2023-12-07 15:51:34 Container image "ghcr.io/zurdi15/romm:2.1.0@sha256:5403e58d0ec714a1aa45c5caf9c897b4eb7fecb403c3041468a707bce4f6bd3b" already present on machine
2023-12-07 15:51:33 Container image "tccr.io/truecharts/db-wait-redis:1.1.0@sha256:8affa086d097b948f62b0433d70f4219a22ec29843ebd5479391869341bdb638" already present on machine
2023-12-07 15:51:33 Created container romm-system-redis-wait
2023-12-07 15:51:14 Created container romm-redis
2023-12-07 15:51:14 Started container romm-redis
2023-12-07 15:51:13 Successfully assigned ix-romm/romm-redis-0 to ix-truenas
2023-12-07 15:51:13 Add eth0 [172.16.24.21/16] from ix-net
2023-12-07 15:51:13 Container image "tccr.io/truecharts/redis:v7.0.12@sha256:4a7b38a055a1183d226d8e2eb8bb1a55d2ef4ba217a74fd9c1562513c51e5064" already present on machine
2023-12-07 15:51:12 Created container romm-mariadb
2023-12-07 15:51:12 Started container romm-mariadb
2023-12-07 15:51:11 Successfully assigned ix-romm/romm-mariadb-799c4895b-mxr6x to ix-truenas
2023-12-07 15:51:11 Successfully provisioned volume pvc-7f4264e5-f8ed-4ee1-815b-f33714a19a77
2023-12-07 15:51:11 Add eth0 [172.16.24.20/16] from ix-net
2023-12-07 15:51:11 Container image "tccr.io/truecharts/mariadb:v10.11.4@sha256:35318404e9c3fa84f9b37c38d39a8f72b8aed6940edb3519deb21c36a3a1e2f2" already present on machine
2023-12-07 15:51:10 Started container romm-system-mariadb-wait
2023-12-07 15:51:10 Successfully provisioned volume pvc-0b58d372-5ef6-4e3c-9de2-07d2db87adad
2023-12-07 15:51:10 0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
2023-12-07 15:51:09 0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
2023-12-07 15:51:09 Updated LoadBalancer with new IPs: [] -> [10.0.3.75]
2023-12-07 15:51:09 Created container romm-system-mariadb-wait
2023-12-07 15:51:08 Created pod: romm-558cfcf446-skmhx
2023-12-07 15:51:08 External provisioner is provisioning volume for claim "ix-romm/data-romm-redis-0"
2023-12-07 15:51:08 Successfully assigned ix-romm/romm-558cfcf446-skmhx to ix-truenas
2023-12-07 15:51:08 waiting for a volume to be created, either by external provisioner "zfs.csi.openebs.io" or manually created by system administrator
2023-12-07 15:51:08 0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
2023-12-07 15:51:08 create Pod romm-redis-0 in StatefulSet romm-redis successful
2023-12-07 15:51:08 Add eth0 [172.16.24.19/16] from ix-net
2023-12-07 15:51:08 Container image "tccr.io/truecharts/db-wait-mariadb:1.1.0@sha256:492a9659511d3288ba9b6536fb17d1cb037fb3876f402dffa5dbcb040acbb85a" already present on machine
2023-12-07 15:51:07 Ensuring load balancer
2023-12-07 15:51:07 waiting for a volume to be created, either by external provisioner "zfs.csi.openebs.io" or manually created by system administrator
2023-12-07 15:51:07 Applied LoadBalancer DaemonSet kube-system/svclb-romm-56f686c1
2023-12-07 15:51:07 Scaled up replica set romm-mariadb-799c4895b to 1
2023-12-07 15:51:07 0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
2023-12-07 15:51:07 Created pod: romm-mariadb-799c4895b-mxr6x
2023-12-07 15:51:07 create Claim data-romm-redis-0 Pod romm-redis-0 in StatefulSet romm-redis success
2023-12-07 15:51:07 External provisioner is provisioning volume for claim "ix-romm/romm-mariadb-data"
2023-12-07 15:51:07 Scaled up replica set romm-558cfcf446 to 1

Application Logs

romm-mariadb

2023-12-07T20:51:12.404979901Z mariadb 20:51:12.40 
2023-12-07T20:51:12.405698192Z mariadb 20:51:12.40 Welcome to the Bitnami mariadb container
2023-12-07T20:51:12.406396476Z mariadb 20:51:12.40 Subscribe to project updates by watching https://github.com/bitnami/containers
2023-12-07T20:51:12.407067788Z mariadb 20:51:12.40 Submit issues and feature requests at https://github.com/bitnami/containers/issues
2023-12-07T20:51:12.407673997Z mariadb 20:51:12.40 
2023-12-07T20:51:12.408367462Z mariadb 20:51:12.40 INFO  ==> ** Starting MariaDB setup **
2023-12-07T20:51:12.416850403Z mariadb 20:51:12.41 INFO  ==> Validating settings in MYSQL_*/MARIADB_* env vars
2023-12-07T20:51:12.419453277Z mariadb 20:51:12.41 INFO  ==> Initializing mariadb database
2023-12-07T20:51:12.467243989Z mariadb 20:51:12.46 INFO  ==> Updating 'my.cnf' with custom configuration
2023-12-07T20:51:12.469796698Z mariadb 20:51:12.46 INFO  ==> Setting user option
2023-12-07T20:51:12.732436738Z mariadb 20:51:12.73 INFO  ==> Setting slow_query_log option
2023-12-07T20:51:12.734741277Z mariadb 20:51:12.73 INFO  ==> Setting long_query_time option
2023-12-07T20:51:12.736938271Z mariadb 20:51:12.73 INFO  ==> Installing database
2023-12-07T20:51:14.545031689Z mariadb 20:51:14.54 INFO  ==> Starting mariadb in background
2023-12-07T20:51:14.564777729Z 2023-12-07 20:51:14 0 [Note] Starting MariaDB 10.11.4-MariaDB source revision 4e2b93dffef2414a11ca5edc8d215f57ee5010e5 as process 117
2023-12-07T20:51:14.572467006Z 2023-12-07 20:51:14 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2023-12-07T20:51:14.573229471Z 2023-12-07 20:51:14 0 [Note] InnoDB: Number of transaction pools: 1
2023-12-07T20:51:14.573256693Z 2023-12-07 20:51:14 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2023-12-07T20:51:14.573263366Z 2023-12-07 20:51:14 0 [Note] mysqld: O_TMPFILE is not supported on /opt/bitnami/mariadb/tmp (disabling future attempts)
2023-12-07T20:51:14.666631012Z 2023-12-07 20:51:14 0 [Note] InnoDB: Using Linux native AIO
2023-12-07T20:51:14.666868933Z 2023-12-07 20:51:14 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2023-12-07T20:51:14.680072663Z 2023-12-07 20:51:14 0 [Note] InnoDB: Completed initialization of buffer pool
2023-12-07T20:51:14.681569560Z 2023-12-07 20:51:14 0 [Note] InnoDB: Buffered log writes (block size=512 bytes)
2023-12-07T20:51:14.693051376Z 2023-12-07 20:51:14 0 [Note] InnoDB: 128 rollback segments are active.
2023-12-07T20:51:14.693331226Z 2023-12-07 20:51:14 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2023-12-07T20:51:14.693349541Z 2023-12-07 20:51:14 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2023-12-07T20:51:14.693940231Z 2023-12-07 20:51:14 0 [Note] InnoDB: log sequence number 45452; transaction id 14
2023-12-07T20:51:14.693987550Z 2023-12-07 20:51:14 0 [Note] InnoDB: Loading buffer pool(s) from /bitnami/mariadb/data/ib_buffer_pool
2023-12-07T20:51:14.694007248Z 2023-12-07 20:51:14 0 [Note] Plugin 'FEEDBACK' is disabled.
2023-12-07T20:51:14.694429047Z 2023-12-07 20:51:14 0 [Note] InnoDB: Buffer pool(s) load completed at 231207 20:51:14
2023-12-07T20:51:14.695260594Z 2023-12-07 20:51:14 0 [Note] Server socket created on IP: '127.0.0.1'.
2023-12-07T20:51:14.695886530Z 2023-12-07 20:51:14 0 [Warning] 'user' entry 'root@romm-mariadb-799c4895b-mxr6x' ignored in --skip-name-resolve mode.
2023-12-07T20:51:14.695897481Z 2023-12-07 20:51:14 0 [Warning] 'user' entry '@romm-mariadb-799c4895b-mxr6x' ignored in --skip-name-resolve mode.
2023-12-07T20:51:14.695933249Z 2023-12-07 20:51:14 0 [Warning] 'proxies_priv' entry '@% root@romm-mariadb-799c4895b-mxr6x' ignored in --skip-name-resolve mode.
2023-12-07T20:51:14.697558279Z 2023-12-07 20:51:14 0 [Note] /opt/bitnami/mariadb/sbin/mysqld: ready for connections.
2023-12-07T20:51:14.697573087Z Version: '10.11.4-MariaDB'  socket: '/opt/bitnami/mariadb/tmp/mysql.sock'  port: 3306  Source distribution
2023-12-07T20:51:16.556233579Z mariadb 20:51:16.55 INFO  ==> Configuring authentication
2023-12-07T20:51:16.569320346Z 2023-12-07 20:51:16 5 [Warning] 'proxies_priv' entry '@% root@romm-mariadb-799c4895b-mxr6x' ignored in --skip-name-resolve mode.
2023-12-07T20:51:16.603476071Z mariadb 20:51:16.60 INFO  ==> Running mysql_upgrade
2023-12-07T20:51:16.620357451Z find: '/docker-entrypoint-startdb.d/': No such file or directory
2023-12-07T20:51:16.622501866Z mariadb 20:51:16.62 INFO  ==> Stopping mariadb
2023-12-07T20:51:16.623758218Z 2023-12-07 20:51:16 0 [Note] /opt/bitnami/mariadb/sbin/mysqld (initiated by: unknown): Normal shutdown
2023-12-07T20:51:16.623903102Z 2023-12-07 20:51:16 0 [Note] InnoDB: FTS optimize thread exiting.
2023-12-07T20:51:16.625970020Z 2023-12-07 20:51:16 0 [Note] InnoDB: Starting shutdown...
2023-12-07T20:51:16.625998705Z 2023-12-07 20:51:16 0 [Note] InnoDB: Dumping buffer pool(s) to /bitnami/mariadb/data/ib_buffer_pool
2023-12-07T20:51:16.626121658Z 2023-12-07 20:51:16 0 [Note] InnoDB: Buffer pool(s) dump completed at 231207 20:51:16
2023-12-07T20:51:16.900178699Z 2023-12-07 20:51:16 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
2023-12-07T20:51:16.900204358Z 2023-12-07 20:51:16 0 [Note] InnoDB: Shutdown completed; log sequence number 46702; transaction id 15
2023-12-07T20:51:16.902042843Z 2023-12-07 20:51:16 0 [Note] /opt/bitnami/mariadb/sbin/mysqld: Shutdown complete
2023-12-07T20:51:16.902058783Z 
2023-12-07T20:51:17.631279683Z mariadb 20:51:17.63 INFO  ==> ** MariaDB setup finished! **
2023-12-07T20:51:17.631288019Z 
2023-12-07T20:51:17.643617310Z mariadb 20:51:17.64 INFO  ==> ** Starting MariaDB **
2023-12-07T20:51:17.668636626Z 2023-12-07 20:51:17 0 [Note] Starting MariaDB 10.11.4-MariaDB source revision 4e2b93dffef2414a11ca5edc8d215f57ee5010e5 as process 1
2023-12-07T20:51:17.676006356Z 2023-12-07 20:51:17 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2023-12-07T20:51:17.676767949Z 2023-12-07 20:51:17 0 [Note] InnoDB: Number of transaction pools: 1
2023-12-07T20:51:17.676786174Z 2023-12-07 20:51:17 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2023-12-07T20:51:17.676792456Z 2023-12-07 20:51:17 0 [Note] mysqld: O_TMPFILE is not supported on /opt/bitnami/mariadb/tmp (disabling future attempts)
2023-12-07T20:51:17.746709100Z 2023-12-07 20:51:17 0 [Note] InnoDB: Using Linux native AIO
2023-12-07T20:51:17.746946279Z 2023-12-07 20:51:17 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2023-12-07T20:51:17.757771601Z 2023-12-07 20:51:17 0 [Note] InnoDB: Completed initialization of buffer pool
2023-12-07T20:51:17.759184258Z 2023-12-07 20:51:17 0 [Note] InnoDB: Buffered log writes (block size=512 bytes)
2023-12-07T20:51:17.769643214Z 2023-12-07 20:51:17 0 [Note] InnoDB: 128 rollback segments are active.
2023-12-07T20:51:17.769923966Z 2023-12-07 20:51:17 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2023-12-07T20:51:17.769949755Z 2023-12-07 20:51:17 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2023-12-07T20:51:17.770253510Z 2023-12-07 20:51:17 0 [Note] InnoDB: log sequence number 46702; transaction id 14
2023-12-07T20:51:17.770286242Z 2023-12-07 20:51:17 0 [Note] InnoDB: Loading buffer pool(s) from /bitnami/mariadb/data/ib_buffer_pool
2023-12-07T20:51:17.770300129Z 2023-12-07 20:51:17 0 [Note] Plugin 'FEEDBACK' is disabled.
2023-12-07T20:51:17.770741265Z 2023-12-07 20:51:17 0 [Note] InnoDB: Buffer pool(s) load completed at 231207 20:51:17
2023-12-07T20:51:17.771323128Z 2023-12-07 20:51:17 0 [Note] Server socket created on IP: '0.0.0.0'.
2023-12-07T20:51:17.771954545Z 2023-12-07 20:51:17 0 [Warning] 'proxies_priv' entry '@% root@romm-mariadb-799c4895b-mxr6x' ignored in --skip-name-resolve mode.
2023-12-07T20:51:17.772876593Z 2023-12-07 20:51:17 0 [Note] /opt/bitnami/mariadb/sbin/mysqld: ready for connections.
2023-12-07T20:51:17.772891531Z Version: '10.11.4-MariaDB'  socket: '/opt/bitnami/mariadb/tmp/mysql.sock'  port: 3306  Source distribution
2023-12-07T20:51:36.833089699Z 2023-12-07 20:51:36 13 [Warning] Aborted connection 13 to db: 'romm' user: 'romm' host: '172.16.24.19' (Got an error reading communication packets)

romm-romm

2023-12-07T21:18:18.907441665Z 16:18:18 Worker rq:worker:28ab4e88092c46aba31c38ff6afc63b3 started with PID 15, version 1.15.1
2023-12-07T21:18:18.907488755Z 16:18:18 Subscribing to channel rq:pubsub:28ab4e88092c46aba31c38ff6afc63b3
2023-12-07T21:18:18.908400633Z 16:18:18 *** Listening on high, default, low...
2023-12-07T21:18:19.715460653Z INFO:     [RomM][alembic.runtime.migration] Context impl MariaDBImpl.
2023-12-07T21:18:19.715499446Z INFO:     [RomM][alembic.runtime.migration] Will assume non-transactional DDL.
2023-12-07T21:18:19.759883082Z Traceback (most recent call last):
2023-12-07T21:18:19.759936343Z   File "/backend/watcher.py", line 72, in <module>
2023-12-07T21:18:19.759953916Z     observer.start()
2023-12-07T21:18:19.759967893Z   File "/backend/lib/python3.10/site-packages/watchdog/observers/api.py", line 261, in start
2023-12-07T21:18:19.759972552Z     emitter.start()
2023-12-07T21:18:19.759977581Z   File "/backend/lib/python3.10/site-packages/watchdog/utils/__init__.py", line 92, in start
2023-12-07T21:18:19.759982010Z     self.on_thread_start()
2023-12-07T21:18:19.759990536Z   File "/backend/lib/python3.10/site-packages/watchdog/observers/inotify.py", line 119, in on_thread_start
2023-12-07T21:18:19.759995105Z     self._inotify = InotifyBuffer(path, self.watch.is_recursive)
2023-12-07T21:18:19.759999864Z   File "/backend/lib/python3.10/site-packages/watchdog/observers/inotify_buffer.py", line 37, in __init__
2023-12-07T21:18:19.760007127Z     self._inotify = Inotify(path, recursive)
2023-12-07T21:18:19.760015593Z   File "/backend/lib/python3.10/site-packages/watchdog/observers/inotify_c.py", line 167, in __init__
2023-12-07T21:18:19.760020342Z     Inotify._raise_error()
2023-12-07T21:18:19.760024610Z   File "/backend/lib/python3.10/site-packages/watchdog/observers/inotify_c.py", line 430, in _raise_error
2023-12-07T21:18:19.760032566Z     raise OSError(errno.EMFILE, "inotify instance limit reached")
2023-12-07T21:18:19.760037004Z OSError: [Errno 24] inotify instance limit reached
2023-12-07T21:18:19.760910039Z INFO:      [RomM] [2023-12-07 16:18:19] Starting scheduler

romm-redis

2023-12-07T20:51:14.779719453Z redis 20:51:14.77 
2023-12-07T20:51:14.780388802Z redis 20:51:14.78 Welcome to the Bitnami redis container
2023-12-07T20:51:14.780986745Z redis 20:51:14.78 Subscribe to project updates by watching https://github.com/bitnami/containers
2023-12-07T20:51:14.781561855Z redis 20:51:14.78 Submit issues and feature requests at https://github.com/bitnami/containers/issues
2023-12-07T20:51:14.782205465Z redis 20:51:14.78 
2023-12-07T20:51:14.782964053Z redis 20:51:14.78 INFO  ==> ** Starting Redis setup **
2023-12-07T20:51:14.790867234Z redis 20:51:14.79 WARN  ==> You set the environment variable ALLOW_EMPTY_PASSWORD=yes. For safety reasons, do not use this flag in a production environment.
2023-12-07T20:51:14.792461928Z redis 20:51:14.79 INFO  ==> Initializing Redis
2023-12-07T20:51:14.798338989Z redis 20:51:14.79 INFO  ==> Setting Redis config file
2023-12-07T20:51:15.390921818Z redis 20:51:15.39 INFO  ==> Configuring replication mode
2023-12-07T20:51:15.402330774Z redis 20:51:15.40 INFO  ==> ** Redis setup finished! **
2023-12-07T20:51:15.402357244Z 
2023-12-07T20:51:15.408719974Z redis 20:51:15.40 INFO  ==> ** Starting Redis **
2023-12-07T20:51:15.412986092Z 1:C 07 Dec 2023 20:51:15.412 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2023-12-07T20:51:15.413012943Z 1:C 07 Dec 2023 20:51:15.412 # Redis version=7.0.12, bits=64, commit=00000000, modified=0, pid=1, just started
2023-12-07T20:51:15.413032841Z 1:C 07 Dec 2023 20:51:15.412 # Configuration loaded
2023-12-07T20:51:15.413220707Z 1:M 07 Dec 2023 20:51:15.413 * monotonic clock: POSIX clock_gettime
2023-12-07T20:51:15.413594596Z 1:M 07 Dec 2023 20:51:15.413 * Running mode=standalone, port=6379.
2023-12-07T20:51:15.413618491Z 1:M 07 Dec 2023 20:51:15.413 # Server initialized
2023-12-07T20:51:15.456144342Z 1:M 07 Dec 2023 20:51:15.456 * Creating AOF base file appendonly.aof.1.base.rdb on server start
2023-12-07T20:51:15.456639581Z 1:M 07 Dec 2023 20:51:15.456 * Creating AOF incr file appendonly.aof.1.incr.aof on server start
2023-12-07T20:51:15.456655361Z 1:M 07 Dec 2023 20:51:15.456 * Ready to accept connections

Application Configuration

Set values for Client Id, Client Secret, and SteamGridDB API Key

Configured App Library Storage, App Resources Storage, and App Logs Storage to all use NFS shares with localhost as the server and all of the NFS paths correctly configures.

These configurations were working before the application update.

Describe the bug

RomM updated last night and the last container fails to deploy. It was working with the above configuration prior to updating. I recreated the container after the update with the same settings and it failed to deploy. I recreated it a final time with only the required settings (Client ID and Client Secret) and it still fails to deploy in the same manner. Wasn't sure if I should go to the developer for this or not, but decided to file a bug report with TrueCharts since I am using it to deploy the application.

To Reproduce

  1. Provide valid values for Client ID and Client Secret
  2. Deploy the application

Expected Behavior

The application to deploy

Screenshots

N/A

Additional Context

Application was working prior to latest update. Now, it fails to start even with blowing away the container and using all default values.

I've read and agree with the following

PrivatePuffin commented 11 months ago

Due to missing screenshots of configuration and this not being pushed through support, i'm going to forward this to the support staff on discord instead. Please file a ticket there.

truecharts-admin commented 3 weeks ago

This issue is locked to prevent necro-posting on closed issues. Please create a new issue or contact staff on discord of the problem persists