iPromKnight / zilean

88 stars 7 forks source link

Why does the Zilean database disappear and stop working? #18

Open Telkaoss opened 2 months ago

Telkaoss commented 2 months ago

I've tried a lot of things with the compose.yaml. The Zilean database is initially in Postgres, but after some time, it disappears, and Zilean stops functioning.

logs zilean :

Processing DMM Hashlists: 99%
[13:53:16] | INFO | "Zilean.DmmScraper.Features.Dmm.DmmScraping" | All files processed
[13:53:18] | INFO | "Zilean.DmmScraper.Features.PythonSupport.ParseTorrentNameService" | RTN: Parsing 617543 torrents
[14:19:55] | INFO | "Zilean.DmmScraper.Features.Dmm.DmmScraping" | Parsed 617543 torrents
[14:19:55] | INFO | "Zilean.Database.Services.TorrentInfoService" | Storing torrents in the database...
[14:19:55] | EROR | "Zilean.Database.Services.TorrentInfoService" | An error occurred while executing a command.
Npgsql.PostgresException: 3D000: database "zilean" does not exist
  at async ValueTask<IBackendMessage> Npgsql.Internal.NpgsqlConnector.
     ReadMessageLong(bool async, DataRowLoadingMode dataRowLoadingMode, bool
     readingNotifications, bool isReadingPrependedMessage)
  at TResult
     System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder1.StateM
     achineBox1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.
     GetResult(short token)
  at async Task Npgsql.Internal.NpgsqlConnector.<Open>gOpenCore|213_1(
     NpgsqlConnector conn, SslMode sslMode, NpgsqlTimeout timeout, bool async,
     CancellationToken cancellationToken, bool isFirstAttempt)
  at async Task Npgsql.Internal.NpgsqlConnector.Open(NpgsqlTimeout timeout, bool
     async, CancellationToken cancellationToken)
  at async ValueTask<NpgsqlConnector> Npgsql.PoolingDataSource.OpenNewConnector(
     NpgsqlConnection conn, NpgsqlTimeout timeout, bool async, CancellationToken
     cancellationToken)
  at async ValueTask<NpgsqlConnector> Npgsql.PoolingDataSource.
     <Get>gRentAsync|34_0(NpgsqlConnection conn, NpgsqlTimeout timeout, bool
     async, CancellationToken cancellationToken)
  at async Task Npgsql.NpgsqlConnection.<Open>g__OpenAsync|42_0(bool async,
     CancellationToken cancellationToken)
  at async Task Zilean.Database.Services.BaseDapperService.ExecuteCommandAsync(
     Func<NpgsqlConnection, Task> operation, string taskMessage,
     CancellationToken cancellationToken) in
     /build/src/Zilean.Database/Services/BaseDapperService.cs:13
  at async Task Zilean.Database.Services.BaseDapperService.ExecuteCommandAsync(
     Func<NpgsqlConnection, Task> operation, string taskMessage,
     CancellationToken cancellationToken) in
     /build/src/Zilean.Database/Services/BaseDapperService.cs:14
Telkaoss commented 2 months ago

update logs postgres :

rage=0.002 s; distance=7 kB, estimate=101743 kB; lsn=0/EC5C8DA8, redo lsn=0/EC5C8D70
2024-08-21 16:28:25.872 UTC [17311] ERROR:  relation "__efmigrationshistory" does not exist at character 15
2024-08-21 16:28:25.872 UTC [17311] STATEMENT:  SELECT * FROM __EFMigrationsHistory LIMIT 20;
2024-08-21 16:28:26.184 UTC [17313] FATAL:  database "template0" is not currently accepting connections
2024-08-21 16:28:27.503 UTC [17316] ERROR:  cannot drop the currently open database
2024-08-21 16:28:27.503 UTC [17316] STATEMENT:  DROP DATABASE postgres;
2024-08-21 16:28:27.600 UTC [56] LOG:  checkpoint starting: immediate force wait
2024-08-21 16:28:27.609 UTC [56] LOG:  checkpoint complete: wrote 1 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.002 s, sync=0.002 s, total=0.009 s; sync files=1, longest=0.002 s, average=0.002 s; distance=12 kB, estimate=91570 kB; lsn=0/EC5CBFE8, redo lsn=0/EC5CBFB0
2024-08-21 16:29:22.866 UTC [17367] FATAL:  database "zilean" does not exist
2024-08-21 16:33:27.700 UTC [56] LOG:  checkpoint starting: time
2024-08-21 16:34:59.996 UTC [56] LOG:  checkpoint complete: wrote 923 buffers (5.6%); 0 WAL file(s) added, 0 removed, 0 recycled; write=92.267 s, sync=0.024 s, total=92.297 s; sync files=301, longest=0.001 s, average=0.001 s; distance=4272 kB, estimate=82840 kB; lsn=0/EC9F82E8, redo lsn=0/EC9F82B0
2024-08-21 17:00:14.305 UTC [18762] FATAL:  database "zilean" does not exist
2024-08-21 18:00:01.437 UTC [21485] FATAL:  unsupported frontend protocol 0.0: server supports 3.0 to 3.0
2024-08-21 18:00:01.764 UTC [21486] FATAL:  unsupported frontend protocol 255.255: server supports 3.0 to 3.0
2024-08-21 18:00:02.088 UTC [21487] FATAL:  no PostgreSQL user name specified in startup packet
2024-08-21 18:00:13.809 UTC [21496] FATAL:  database "zilean" does not exist
2024-08-21 18:17:34.838 UTC [22291] LOG:  invalid length of startup packet
2024-08-21 19:00:14.049 UTC [24228] FATAL:  database "zilean" does not exist
2024-08-21 20:00:13.823 UTC [26953] FATAL:  database "zilean" does not exist
2024-08-21 21:00:14.151 UTC [29675] FATAL:  database "zilean" does not exist
2024-08-21 22:00:14.160 UTC [32406] FATAL:  database "zilean" does not exist

I downgraded to 1.4.0, and it works well with Elasticsearch.

Varming73 commented 2 months ago

After restarting Zilean it started all over and spent an hour rebuilding the postgres database. I didn't notice in time to pick up the log but could be due to the same problem. On Elasticsearch it's not and is in general very stable. Got both versions running at the moment.

iPromKnight commented 1 month ago

Those errors from the Postgres logs seem to indicate that the permissions of the files within your Postgres data mount that the database resides in do not match the permissions of the user that Postgres is running as