Piped backend and postgres issue - restart loop

[virustesztalt]

Official Instance

• [ ] The bug is reproducible on the official hosted instance, or is API-related.

Describe the bug

Hello!, last week or so my private piped instance has been unable to function i'm using the NGINX version i followed the official guide and everything worked perfectly since a long time ago(thanks! it's amazing, i love you guys:3)

the frontend is in good shape loads and everything good, but the backend(so videos etc) doesnt work, it's in forever loading screen i can go to login page but cant submit my account details

upon looking into the log it seems like it's getting some errors which cause it to restart multiple times over and over again (but i dont have idea what is the problem, i'm no programmer)

i was able to reproduce this in Libretube (android client) too and in different browser( firefox,ungoogled chromium) same on desktop, it wouldnt load videos at all also i can go to preferences tab

also it seems there's an issue with postgres too but i dont understand what's the issue

To Reproduce

1. Open url at https://piped.private.instance.example/trending
2. forewer is triyng to load

Expected behavior

everything should work as it was for couple months

Logs/Errors

piped-backend | org.postgresql.util.PSQLException: The connection attempt failed. piped-backend | at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:358) piped-backend | at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:54) piped-backend | at org.postgresql.jdbc.PgConnection.(PgConnection.java:273) piped-backend | at org.postgresql.Driver.makeConnection(Driver.java:446) piped-backend | at org.postgresql.Driver.connect(Driver.java:298) piped-backend | at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:683) piped-backend | at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:230) piped-backend | at me.kavin.piped.utils.LiquibaseHelper.init(LiquibaseHelper.java:32) piped-backend | at me.kavin.piped.Main.main(Main.java:82) piped-backend | Caused by: java.net.UnknownHostException: postgres piped-backend | at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:567) piped-backend | at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) piped-backend | at java.base/java.net.Socket.connect(Socket.java:757) piped-backend | at org.postgresql.core.PGStream.createSocket(PGStream.java:243) piped-backend | at org.postgresql.core.PGStream.(PGStream.java:98) piped-backend | at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:136) piped-backend | at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:262) piped-backend | ... 8 more piped-backend exited with code 1

postgres | postgres | PostgreSQL Database directory appears to contain a database; Skipping initialization postgres | postgres | 2024-04-30 21:02:22.410 UTC [1] LOG: starting PostgreSQL 15.6 (Debian 15.6-1.pgdg120+2) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit postgres | 2024-04-30 21:02:22.411 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 postgres | 2024-04-30 21:02:22.411 UTC [1] LOG: listening on IPv6 address "::", port 5432 postgres | 2024-04-30 21:02:22.414 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" postgres | 2024-04-30 21:02:22.423 UTC [29] LOG: database system was interrupted; last known up at 2024-04-21 18:33:22 UTC postgres | 2024-04-30 21:02:22.526 UTC [29] LOG: unexpected pageaddr 0/D69DA000 in log segment 0000000100000000000000D8, offset 10330112 postgres | 2024-04-30 21:02:22.526 UTC [29] LOG: invalid primary checkpoint record postgres | 2024-04-30 21:02:22.526 UTC [29] PANIC: could not locate a valid checkpoint record postgres | 2024-04-30 21:02:23.168 UTC [1] LOG: startup process (PID 29) was terminated by signal 6: Aborted postgres | 2024-04-30 21:02:23.168 UTC [1] LOG: aborting startup due to startup process failure postgres | 2024-04-30 21:02:23.171 UTC [1] LOG: database system is shut down postgres exited with code 1

Browser, and OS with Version.

Fedora 38- firefox 123, UGchromium 113 Android 13- libretube(latest) bromite 108

Additional context

the piped instance runs in a LXC container under Proxmox on the same instance i have Portainer and ttrss specs of the container:

[AlyoshaVasilieva]

Pretty sure your postgresql database is corrupted

[virustesztalt]

hmmm, and how could i resolve that? also is there stilll some way to export my subscriptions from my piped? idk why did this happen, there was a power outage recently which turned the server off, but all the storage is redundantly stored in raid 5 please if you have any idea how to solve it that would be great! :3

[jadtl]

Might be chiming in late, but it indeed sounds like a corrupt database.

Aside from storage redundancy, which only helps if one of your drives fails, do you make regular snapshots of your database? The most straightforward way to repair such database corruptions from a power outage is to roll back from a healthy snapshot.

If you don't have a snapshot system, you should set one up as soon as possible.

Now as a last resort to save your database, you should reset the write-ahead log which is corrupted, Postgres has a command for that.

Basically:

1. Backup your database in case it only makes its state worse.
2. Log into your Docker container (docker exec -it ${your_postgres_container_name} bash).
3. Run the command pg_resetwal -D /var/lib/postgresql/data or wherever your database is stored.

(I'm not sure this issue falls under "bug", since it doesn't relate to Piped per se but to IT)