I-am-PUID-0 / DMB

Debrid Media Bridge
MIT License
123 stars 9 forks source link

Riven backend errors on latest (0.17 / 0.18) #73

Closed profucius closed 2 weeks ago

profucius commented 2 weeks ago

Describe the bug

After my nightly backup stopped and restarted the stacks, Riven no longer works. It appears the backend updated itself, and I am getting the below errors. I have also forced my stack to update DMB to the latest 5.4.3 today, same issue.

The Riven frontend eventually starts, but because the backend does not, nothing works.

The error that I noticed might be of note is this line:

Unexpected error during migration: Can't locate revision identified by 'a00afa5531e7'
Migration failed: Can't locate revision identified by 'a00afa5531e7'
Error in main thread: Can't locate revision identified by 'a00afa5531e7'

Log

DDDDDDDDDDDDD        MMMMMMMM               MMMMMMMMBBBBBBBBBBBBBBBBB   
D::::::::::::DDD     M:::::::M             M:::::::MB::::::::::::::::B  
D:::::::::::::::DD   M::::::::M           M::::::::MB::::::BBBBBB:::::B 
DDD:::::DDDDD:::::D  M:::::::::M         M:::::::::MBB:::::B     B:::::B
  D:::::D    D:::::D M::::::::::M       M::::::::::M  B::::B     B:::::B
  D:::::D     D:::::DM:::::::::::M     M:::::::::::M  B::::B     B:::::B
  D:::::D     D:::::DM:::::::M::::M   M::::M:::::::M  B::::BBBBBB:::::B 
  D:::::D     D:::::DM::::::M M::::M M::::M M::::::M  B:::::::::::::BB  
  D:::::D     D:::::DM::::::M  M::::M::::M  M::::::M  B::::BBBBBB:::::B 
  D:::::D     D:::::DM::::::M   M:::::::M   M::::::M  B::::B     B:::::B
  D:::::D     D:::::DM::::::M    M:::::M    M::::::M  B::::B     B:::::B
  D:::::D    D:::::D M::::::M     MMMMM     M::::::M  B::::B     B:::::B
DDD:::::DDDDD:::::D  M::::::M               M::::::MBB:::::BBBBBB::::::B
D:::::::::::::::DD   M::::::M               M::::::MB:::::::::::::::::B 
D::::::::::::DDD     M::::::M               M::::::MB::::::::::::::::B  
DDDDDDDDDDDDD        MMMMMMMM               MMMMMMMMBBBBBBBBBBBBBBBBB   

                              Version: 5.4.3                                    

Setting up Zurg
Setting up Zurg w/ RealDebrid instance in directory: /zurg/RD/
Using nightly release version from environment variable
Preparing Zurg instance for RealDebrid
Zurg setup process complete
Automatic update disabled for Zurg
Starting Zurg w/ RealDebrid process
Zurg process started with PID: 9
Checking rclone flags
Configuring the rclone mount name to "DMB"
Configuring rclone for DMB
Zurg w/ RealDebrid subprocess: Logging to logs/zurg.log
Zurg w/ RealDebrid subprocess: Failed to sync logger: sync /dev/stdout: invalid argument
Zurg w/ RealDebrid subprocess: | zurg | Version: 2024.11.06.0016-nightly
Zurg w/ RealDebrid subprocess: | zurg | GitCommit: ef6c95c22b21c966e4defb36fffdf7009fc128f8
Zurg w/ RealDebrid subprocess: | zurg | BuiltAt: 2024-11-06T00:16:09
Zurg w/ RealDebrid subprocess: | zurg | Debug logging is enabled; if you are not debugging please set LOG_LEVEL=INFO in your system environment
Zurg w/ RealDebrid subprocess: | zurg | Your account will expire in 402 days
Zurg w/ RealDebrid subprocess: | zurg | Starting server on [::]:9786
Zurg w/ RealDebrid subprocess: | manager | Mounted 129 new downloads
Waiting to start the rclone process until the Zurg WebDAV http://localhost:9786/dav/ is accessible.
The Zurg WebDAV URL http://localhost:9786/dav is accessible. Starting rclone for DMB
Starting rclone w/ DMB process
rclone process started with PID: 19
rclone startup complete
Setting up PostgreSQL...
PostgreSQL data directory exists at /postgres_data.
Changed ownership and set permissions of /postgres_data.
Starting PostgreSQL process
PostgreSQL process started with PID: 29
Checking if PostgreSQL server has started...
PostgreSQL subprocess: starting PostgreSQL 16.4 on x86_64-alpine-linux-musl, compiled by gcc (Alpine 13.2.1_git20240309) 13.2.1 20240309, 64-bit
PostgreSQL subprocess: listening on IPv6 address "::1", port 5432
PostgreSQL subprocess: listening on IPv4 address "127.0.0.1", port 5432
PostgreSQL subprocess: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"
PostgreSQL subprocess: database system was shut down at 2024-11-06 12:04:13 EST
PostgreSQL subprocess: database system is ready to accept connections
Zurg w/ RealDebrid subprocess: | manager | Loaded 1785 dumped torrents
PostgreSQL server has started.
Database 'postgres' already exists.
Checking if pgAgent extension is installed in 'postgres' database...
pgAgent extension already exists in 'postgres' database.
Database 'riven' already exists.
Checking if PostgreSQL database is accessible...
PostgreSQL database is accessible.
Listing PostgreSQL database sizes...
Database: postgres, Size: 8028 kB
Database: riven, Size: 20 MB
Database: template1, Size: 7417 kB
Database: template0, Size: 7353 kB
Database: pgadmin, Size: 8401 kB
Creating pgAdmin database...
Database 'pgadmin' already exists.
system_stats extension already exists in 'pgadmin' database.
pgAdmin data directory exists at /pgadmin/data.
Changed ownership and set permissions of /pgadmin/data.
Starting pgAdmin process with external PostgreSQL database...
pgAdmin configuration file already exists at /pgadmin/data/config_local.py.
Created symlink for config_local.py at /pgadmin/venv/lib/python3.11/site-packages/pgadmin4/config_local.py.
Starting pgAdmin process
pgAdmin process started with PID: 52
pgAdmin process started successfully using PostgreSQL database.
Application Initialisation
pgAdmin subprocess: ======================================
pgAdmin subprocess: Starting pgAdmin 4. Please navigate to http://0.0.0.0:5050 in your browser.
pgAdmin subprocess: * Serving Flask app 'pgadmin'
pgAdmin subprocess: * Debug mode: off
Connecting to pgAdmin database at postgresql://DMB:postgres@127.0.0.1/pgadmin to add server 'DMB'...
pgAdmin 'server' table exists. Proceeding to check server.
Server 'DMB' already exists in pgAdmin database.
Starting pgAgent process
pgAgent process started with PID: 116
PostgreSQL and pgAdmin setup completed successfully.
Configuring riven_backend
Using main branch for riven_backend
Successfully downloaded main branch for riven_backend
Setting up Poetry environment in /riven/backend
Starting python_env_setup process
python_env_setup process started with PID: 120
Zurg w/ RealDebrid subprocess: | repair | No broken torrents to repair
Zurg w/ RealDebrid subprocess: | manager | Compiling 1750 torrents
Zurg w/ RealDebrid subprocess: | manager | Compiled into 250 unique torrents
Starting install_poetry process
...
Poetry environment setup complete at /riven/backend/venv
No .env file found in /riven/backend/data/.env
Automatic update disabled for riven_backend
Starting riven_backend process
riven_backend process started with PID: 174
Riven v0.18.0 starting!
Overseerr initialized!
Trakt initialized!
Torrentio initialized!
Rclone path symlinks are pointed to: /data/DMB/__all__
Symlinks will be placed in: /mnt
Symlink initialized!
Plex Updater initialized!
Your account expires in 402 days.

Unexpected error during migration: Can't locate revision identified by 'a00afa5531e7'
Migration failed: Can't locate revision identified by 'a00afa5531e7'
Error in main thread: Can't locate revision identified by 'a00afa5531e7'
Traceback (most recent call last):
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/base.py", line 254, in _catch_revision_errors
riven_backend subprocess: yield
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/base.py", line 462, in _upgrade_revs
riven_backend subprocess: for script in reversed(list(revs))
riven_backend subprocess: ^^^^^^^^^^
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 814, in iterate_revisions
riven_backend subprocess: revisions, heads = fn(
riven_backend subprocess: ^^^
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 1475, in _collect_upgrade_revisions
riven_backend subprocess: current_revisions = self.get_revisions(lower)
riven_backend subprocess: ^^^^^^^^^^^^^^^^^^^^^^^^^
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 542, in get_revisions
riven_backend subprocess: return sum([self.get_revisions(id_elem) for id_elem in id_], ())
riven_backend subprocess: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 542, in <listcomp>
riven_backend subprocess: return sum([self.get_revisions(id_elem) for id_elem in id_], ())
riven_backend subprocess: ^^^^^^^^^^^^^^^^^^^^^^^^^^^
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 565, in get_revisions
riven_backend subprocess: return tuple(
riven_backend subprocess: ^^^^^^
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 566, in <genexpr>
riven_backend subprocess: self._revision_for_ident(rev_id, branch_label)
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 637, in _revision_for_ident
riven_backend subprocess: raise ResolutionError(
riven_backend subprocess: alembic.script.revision.ResolutionError: No such revision or branch 'a00afa5531e7'
riven_backend subprocess: The above exception was the direct cause of the following exception:
riven_backend subprocess: Traceback (most recent call last):
riven_backend subprocess: File "/riven/backend/src/main.py", line 106, in <module>
riven_backend subprocess: app.program.start()
riven_backend subprocess: File "/riven/backend/src/program/program.py", line 161, in start
riven_backend subprocess: run_migrations()
riven_backend subprocess: File "/riven/backend/src/program/db/db.py", line 63, in run_migrations
riven_backend subprocess: command.upgrade(alembic_cfg, "head")
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/command.py", line 406, in upgrade
riven_backend subprocess: script.run_env()
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/base.py", line 586, in run_env
riven_backend subprocess: util.load_python_file(self.dir, "env.py")
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
riven_backend subprocess: module = load_module_py(module_id, path)
riven_backend subprocess: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
riven_backend subprocess: spec.loader.exec_module(module)  # type: ignore
riven_backend subprocess: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
riven_backend subprocess: File "<frozen importlib._bootstrap_external>", line 940, in exec_module
riven_backend subprocess: File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
riven_backend subprocess: File "/riven/backend/src/alembic/env.py", line 124, in <module>
riven_backend subprocess: run_migrations_online()
riven_backend subprocess: File "/riven/backend/src/alembic/env.py", line 91, in run_migrations_online
riven_backend subprocess: context.run_migrations()
riven_backend subprocess: File "<string>", line 8, in run_migrations
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/runtime/environment.py", line 946, in run_migrations
riven_backend subprocess: self.get_context().run_migrations(**kw)
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/runtime/migration.py", line 611, in run_migrations
riven_backend subprocess: for step in self._migrations_fn(heads, self):
riven_backend subprocess: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/command.py", line 395, in upgrade
riven_backend subprocess: return script._upgrade_revs(revision, rev)
riven_backend subprocess: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/base.py", line 450, in _upgrade_revs
riven_backend subprocess: with self._catch_revision_errors(
riven_backend subprocess: File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
riven_backend subprocess: self.gen.throw(typ, value, traceback)
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/base.py", line 286, in _catch_revision_errors
riven_backend subprocess: raise util.CommandError(resolution) from re
riven_backend subprocess: alembic.util.exc.CommandError: Can't locate revision identified by 'a00afa5531e7'
riven_backend subprocess: Traceback (most recent call last):
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/base.py", line 254, in _catch_revision_errors
riven_backend subprocess: yield
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/base.py", line 462, in _upgrade_revs
riven_backend subprocess: for script in reversed(list(revs))
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 814, in iterate_revisions
riven_backend subprocess: revisions, heads = fn(
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 1475, in _collect_upgrade_revisions
riven_backend subprocess: current_revisions = self.get_revisions(lower)
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 542, in get_revisions
riven_backend subprocess: return sum([self.get_revisions(id_elem) for id_elem in id_], ())
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 542, in <listcomp>
riven_backend subprocess: return sum([self.get_revisions(id_elem) for id_elem in id_], ())
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 565, in get_revisions
riven_backend subprocess: return tuple(
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 566, in <genexpr>
riven_backend subprocess: self._revision_for_ident(rev_id, branch_label)
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/revision.py", line 637, in _revision_for_ident
riven_backend subprocess: raise ResolutionError(
riven_backend subprocess: alembic.script.revision.ResolutionError: No such revision or branch 'a00afa5531e7'
riven_backend subprocess: The above exception was the direct cause of the following exception:
riven_backend subprocess: Traceback (most recent call last):
riven_backend subprocess: File "/riven/backend/src/main.py", line 106, in <module>
riven_backend subprocess: app.program.start()
riven_backend subprocess: File "/riven/backend/src/program/program.py", line 161, in start
riven_backend subprocess: run_migrations()
riven_backend subprocess: File "/riven/backend/src/program/db/db.py", line 63, in run_migrations
riven_backend subprocess: command.upgrade(alembic_cfg, "head")
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/command.py", line 406, in upgrade
riven_backend subprocess: script.run_env()
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/base.py", line 586, in run_env
riven_backend subprocess: util.load_python_file(self.dir, "env.py")
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
riven_backend subprocess: module = load_module_py(module_id, path)
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
riven_backend subprocess: spec.loader.exec_module(module)  # type: ignore
riven_backend subprocess: File "<frozen importlib._bootstrap_external>", line 940, in exec_module
riven_backend subprocess: File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
riven_backend subprocess: File "/riven/backend/src/alembic/env.py", line 124, in <module>
riven_backend subprocess: run_migrations_online()
riven_backend subprocess: File "/riven/backend/src/alembic/env.py", line 91, in run_migrations_online
riven_backend subprocess: context.run_migrations()
riven_backend subprocess: File "<string>", line 8, in run_migrations
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/runtime/environment.py", line 946, in run_migrations
riven_backend subprocess: self.get_context().run_migrations(**kw)
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/runtime/migration.py", line 611, in run_migrations
riven_backend subprocess: for step in self._migrations_fn(heads, self):
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/command.py", line 395, in upgrade
riven_backend subprocess: return script._upgrade_revs(revision, rev)
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/base.py", line 450, in _upgrade_revs
riven_backend subprocess: with self._catch_revision_errors(
riven_backend subprocess: File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__
riven_backend subprocess: self.gen.throw(typ, value, traceback)
riven_backend subprocess: File "/riven/backend/venv/lib/python3.11/site-packages/alembic/script/base.py", line 286, in _catch_revision_errors
riven_backend subprocess: raise util.CommandError(resolution) from re
riven_backend subprocess: alembic.util.exc.CommandError: Can't locate revision identified by 'a00afa5531e7'

Server has been stopped
Loading Riven settings
API key loaded successfully from /riven/backend/data/settings.json
Server config saved to /config/server.json
Attempt 1/10 to fetch settings from http://127.0.0.1:8080/api/v1/settings/get/all
Error fetching settings: HTTPConnectionPool(host='127.0.0.1', port=8080): Max retries exceeded with url: /api/v1/settings/get/all (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x790921408110>: Failed to establish a new connection: [Errno 111] Connection refused'))
...
Failed to load settings: Max retries reached

Configuring riven_frontend
Using main branch for riven_frontend
Successfully downloaded main branch for riven_frontend
Setting up riven_frontend
Starting npm_install process
...
Starting node_build process
...
npm install and build complete
Automatic update disabled for riven_frontend
Starting riven_frontend process
riven_frontend process started with PID: 326
riven_frontend subprocess: Listening on 0.0.0.0:3000
riven_frontend subprocess: TypeError: fetch failed
riven_frontend subprocess: at node:internal/deps/undici/undici:12500:13
riven_frontend subprocess: at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
riven_frontend subprocess: at async s (file:///riven/frontend/node_modules/.pnpm/@hey-api+client-fetch@0.4.2/node_modules/@hey-api/client-fetch/dist/index.js:1:5004)
riven_frontend subprocess: at async load (file:///riven/frontend/build/server/chunks/25-CMWtC2dw.js:6:13)
riven_frontend subprocess: at async load_data (file:///riven/frontend/build/server/index.js:1062:18)
riven_frontend subprocess: at async file:///riven/frontend/build/server/index.js:2485:18 {
riven_frontend subprocess: [cause]: Error: connect ECONNREFUSED 127.0.0.1:8080
riven_frontend subprocess: at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1605:16) {
riven_frontend subprocess: errno: -111,
riven_frontend subprocess: code: 'ECONNREFUSED',
riven_frontend subprocess: syscall: 'connect',
riven_frontend subprocess: address: '127.0.0.1',
riven_frontend subprocess: port: 8080
riven_frontend subprocess: }
riven_frontend subprocess: }
The Riven backend process is not running.
The Riven backend process is not running.
PostgreSQL subprocess: checkpoint starting: time
PostgreSQL subprocess: checkpoint complete: wrote 20 buffers (0.1%); 0 WAL file(s) added, 0 removed, 0 recycled; write=1.816 s, sync=0.029 s, total=1.884 s; sync files=18, longest=0.019 s, average=0.002 s; distance=92 kB, estimate=92 kB; lsn=0/129BC208, redo lsn=0/129BC1D0
The Riven backend process is not running.
GSKang94 commented 2 weeks ago

Hey @profucius I faced same issue this morning after latest Riven update Try dropping database with docker exec -it DMB sh -c "PGPASSWORD=postgres psql -U DMB -d postgres -c 'DROP DATABASE riven;'" and then restart docker container

profucius commented 2 weeks ago

Try dropping database with docker exec -it DMB sh -c "PGPASSWORD=postgres psql -U DMB -d postgres -c 'DROP DATABASE riven;'" and then restart docker container

That worked, thanks! Is there any way of knowing or predicting these kinds of issues? Or are we destined to just continue expecting things to break, reporting them here, and someone like yourself happening to know what to do about it. I always feel helpless against the constant shifting of tectonic plates within Riven especially, since it always updates itself even when I wanted to prevent that from happening.

I-am-PUID-0 commented 2 weeks ago

The root cause was an update to the Riven Backend that required a database reset.

Riven is very much still beta, so breaking changes are to be expected. I try to limit breaking changes to DMB, but some are necessitated by Riven changes.

If you want to lock the Riven Backend and Fronted versions, you can use RIVEN_BACKEND_VERSION= & RIVEN_FRONTEND_VERSION= respectively.

I recommend following the Riven & DMB Discord Servers so you can stay apprised of changes sooner.

I-am-PUID-0 commented 2 weeks ago

It looks like you may also have RIVEN_BACKEND_BRANCH=main & RIVEN_FRONTEND_BRANCH=main set; remove them, as Riven uses the main branches as dev.

profucius commented 2 weeks ago

I know it's usually not DMB but Riven instead; My beef with them is that they don't make a callout mention of breaking changes in their github releases. Maybe they mention it casually as a line item that someone in-the-know might recognize. But not everyone has the ability to constantly keep up with discord updates across hundreds of softwares. I've mentioned this to them in their discord, and it seems to have fallen on deaf ears. I'm just venting. Thanks for the help. And I'll make that change in my yaml.

I-am-PUID-0 commented 2 weeks ago

No worries — I totally get it! They use semantic versioning but don't use the first digit — e.g., 1.0.0 to 2.0.0 — to highlight breaking changes. This keeps me on my toes as well.