superseriousbusiness / gotosocial

Fast, fun, small ActivityPub server.
https://docs.gotosocial.org
GNU Affero General Public License v3.0
3.76k stars 322 forks source link

[bug] Media pruning causes “panic: failed stopping *gtsmodel.Account cache after 5 tries” error? #1713

Closed hikari-no-yume closed 1 year ago

hikari-no-yume commented 1 year ago

Describe the bug with a clear and concise description of what the bug is.

GoToSocial is using a lot of disk space on my VPS. Until recently I was using 0.5.2, which didn't do sufficient pruning (AIUI). I upgraded to 0.8.1 and want to run the admin media prune remote command. However, it errors:

$ sudo ./gotosocial --config-path ./config.yaml admin media prune remote 
timestamp="26/04/2023 10:34:06.886" func=bundb.sqliteConn level=INFO msg="connected to SQLITE database with address file:sqlite.db?_txlock=immediate"
timestamp="26/04/2023 10:34:06.889" func=bundb.sqlitePragmas level=INFO msg="sqlite pragma journal_mode set to wal"
timestamp="26/04/2023 10:34:06.889" func=bundb.sqlitePragmas level=INFO msg="sqlite pragma synchronous set to 1"
timestamp="26/04/2023 10:34:06.889" func=bundb.sqlitePragmas level=INFO msg="sqlite pragma cache_size set to -8192"
timestamp="26/04/2023 10:34:06.889" func=bundb.sqlitePragmas level=INFO msg="sqlite pragma busy_timeout set to 300000"
timestamp="26/04/2023 10:34:06.893" func=bundb.doMigration level=INFO msg="there are no new migrations to run"
timestamp="26/04/2023 10:34:59.033" func=prune.glob..func2 level=INFO msg="DRY RUN: 41639 remote items are unused/stale and eligible to be pruned"
timestamp="26/04/2023 10:34:59.033" func=bundb.(*basicDB).Stop level=INFO msg="closing db connection"
timestamp="26/04/2023 10:34:59.037" func=cache.tryUntil level=PANIC msg="failed stopping *gtsmodel.Account cache after 5 tries"
panic: failed stopping *gtsmodel.Account cache after 5 tries

goroutine 1 [running]:
github.com/superseriousbusiness/gotosocial/internal/log.Panicf.func1()
    /drone/src/internal/log/log.go:139 +0x2a
github.com/superseriousbusiness/gotosocial/internal/log.Panicf({0x0, 0x0}, {0x26129f7, 0x18}, {0xc000b3bab0, 0x2, 0x2})
    /drone/src/internal/log/log.go:141 +0x139
github.com/superseriousbusiness/gotosocial/internal/cache.tryUntil({0xc000db8c00, 0x20}, 0x5, 0xc000b3bb18)
    /drone/src/internal/cache/util.go:80 +0xdb
github.com/superseriousbusiness/gotosocial/internal/cache.tryStop[...](0xc000000180, 0x1)
    /drone/src/internal/cache/util.go:69 +0xaa
github.com/superseriousbusiness/gotosocial/internal/cache.(*GTSCaches).Stop(0xc0000f89c0)
    /drone/src/internal/cache/gts.go:103 +0x46
github.com/superseriousbusiness/gotosocial/internal/cache.(*Caches).Stop(0xc0000f89c0)
    /drone/src/internal/cache/cache.go:62 +0x25
github.com/superseriousbusiness/gotosocial/cmd/gotosocial/action/admin/media/prune.(*prune).shutdown(0xc000707f20, {0x2b39b38, 0xc000042090})
    /drone/src/cmd/gotosocial/action/admin/media/prune/common.go:78 +0xf5
github.com/superseriousbusiness/gotosocial/cmd/gotosocial/action/admin/media/prune.glob..func2({0x2b39b38, 0xc000042090})
    /drone/src/cmd/gotosocial/action/admin/media/prune/remote.go:56 +0x28a
main.run({0x2b39b38, 0xc000042090}, 0x26b61d8)
    /drone/src/cmd/gotosocial/common.go:81 +0x12c
main.adminCommands.func22(0xc000697500?, {0x25e52dd?, 0x2?, 0x2?})
    /drone/src/cmd/gotosocial/admin.go:204 +0x31
github.com/spf13/cobra.(*Command).execute(0xc000697500, {0xc0004c2480, 0x2, 0x2})
    /drone/src/vendor/github.com/spf13/cobra/command.go:916 +0x862
github.com/spf13/cobra.(*Command).ExecuteC(0xc0003a0900)
    /drone/src/vendor/github.com/spf13/cobra/command.go:1044 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
    /drone/src/vendor/github.com/spf13/cobra/command.go:968
main.main()
    /drone/src/cmd/gotosocial/main.go:74 +0x1dd

What's your GoToSocial Version?

0.8.1

GoToSocial Arch

arm64 binary

What happened?

No response

What you expected to happen?

No response

How to reproduce it?

No response

Anything else we need to know?

No response

hikari-no-yume commented 1 year ago

Maybe related to https://github.com/superseriousbusiness/gotosocial/issues/1645? I'm using SQLite though.

hikari-no-yume commented 1 year ago

Ah, doing media pruning through the web interface seems to have worked. :)

So it's just a CLI issue.

trysdyn commented 1 year ago

Pruning works fine despite the error; it just looks a little grungy. What happened is you ran a dry run, which only counts how many media items you will prune and takes no other action.

Dry run is the default, you need --dry-run=false to actually prune from CLI. It'll still barf up that cache spindown error but it'll work.

There is a bug here that that error appears at all, but yeah CLI prunes should work.

hikari-no-yume commented 1 year ago

Ohh, I noticed the dry run warning but I was really confused because I hadn't passed the --dry-run flag, which I assumed was what you use for dry runs. The documentation for that flag could be improved I guess.

NyaaaWhatsUpDoc commented 1 year ago

Again (again) #1690 includes something which should help with this :)