theotherp / nzbhydra2

Usenet meta search
Other
1.23k stars 75 forks source link

why nzbhydra.mv.db is so big ? 649 MB #866

Open tigermanIT opened 1 year ago

tigermanIT commented 1 year ago

Please can you check why nzbhydra.mv.db keep getting bigger

C:\nzbhydra2-5.1.8-windows\data\database\nzbhydra.mv.db

file size : 649 MB

https://file.io/kpnuANf8RP7h

theotherp commented 1 year ago

That's honestly not that big compared to other users' databases. Nothing I can do about it.

On Sun, 11 Jun 2023, 06:49 tigermanIT, @.***> wrote:

Please can you check why nzbhydra.mv.db keep getting bigger

C:\nzbhydra2-5.1.8-windows\data\database\nzbhydra.mv.db

file size : 649 MB

https://file.io/kpnuANf8RP7h

— Reply to this email directly, view it on GitHub https://github.com/theotherp/nzbhydra2/issues/866, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADNUA6JP7V33LY2WVAYYJODXKVE6HANCNFSM6AAAAAAZCC75TI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

tigermanIT commented 1 year ago

but it keeps getting bigger it's 717 MB right now from 649 to 717 MB in 1 hour

tigermanIT commented 1 year ago

739 MB in a couple of minutes ! would be great if you can install nzbhydra2-5.1.8-windows and reproduce

theotherp commented 1 year ago

Not to be too flippant about this but come back when it's a couple of gigabytes. Unfortunately this is a known problem with the database library. When you restart hydra it should get smaller because then it will try to compress it. You can also try enabling advanced options and increasing the time for this (something with database compression or shutdown time).

On Sun, 11 Jun 2023, 08:09 tigermanIT, @.***> wrote:

739 MB in a couple of minutes ! would be great if you can install nzbhydra2-5.1.8-windows and reproduce

— Reply to this email directly, view it on GitHub https://github.com/theotherp/nzbhydra2/issues/866#issuecomment-1586032930, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADNUA6L4JA27PU3OFW3YA5LXKVOINANCNFSM6AAAAAAZCC75TI . You are receiving this because you commented.Message ID: @.***>

MicMillon commented 12 months ago

... mine is 45G

was 46g for a couple of days, just restarted it, and it went from 46 to 45g..

i do care about the stats.. its actually the main reason i use it. i want it to help me decide which indexers are preforming the best, so i know which ones are worth renewing. if i delete the file, will it regenerate? i dont mind loosing search history, i just want the stats.. are they maintained in another file?

Program info Version: 5.1.11 Container version: v5.1.11-ls110 Container release type: Release Container author: linuxserver.io

running inside a docker, on an unraid server (Version: 6.12.4)

root@RIG:~# ls -ltrh /mnt/user/System/Docker/AppData/nzbhydra2/database/ total 46G -rw-r--r-- 1 nobody users 22M Jan 19 2023 nzbhydra.trace.db.old -rw-r--r-- 1 nobody users 13M Sep 5 08:11 nzbhydra.trace.db -rw-r--r-- 1 nobody users 46G Sep 17 22:28 nzbhydra.mv.db root@RIG:~# ls -ltrh /mnt/user/System/Docker/AppData/nzbhydra2/database/ total 46G -rw-r--r-- 1 nobody users 22M Jan 19 2023 nzbhydra.trace.db.old -rw-r--r-- 1 nobody users 13M Sep 5 08:11 nzbhydra.trace.db -rw-r--r-- 1 nobody users 46G Sep 19 12:36 nzbhydra.mv.db root@RIG:~# root@RIG:~# ls -ltrh /mnt/user/System/Docker/AppData/nzbhydra2/database/ total 46G -rw-r--r-- 1 nobody users 22M Jan 19 2023 nzbhydra.trace.db.old -rw-r--r-- 1 nobody users 13M Sep 5 08:11 nzbhydra.trace.db -rw-r--r-- 1 nobody users 45G Sep 19 12:36 nzbhydra.mv.db root@RIG:~#

theotherp commented 12 months ago

Yes, the database will be recreated on the next start. But the stats are created on the fly from the history. Can you please upload your debug infos zip?

On Tue, 19 Sept 2023, 19:43 MicMillon, @.***> wrote:

... mine is 45G

was 46g for a couple of days, just restarted it, and it went from 46 to 45g..

i do care about the stats.. its actually the main reason i use it. i want it to help me decide which indexers are preforming the best, so i know which ones are worth renewing. if i delete the file, will it regenerate? i dont mind loosing search history, i just want the stats.. are they maintained in another file?

Program info Version: 5.1.11 Container version: v5.1.11-ls110 Container release type: Release Container author: linuxserver.io

running inside a docker, on an unraid server (Version: 6.12.4)

@.:# ls -ltrh /mnt/user/System/Docker/AppData/nzbhydra2/database/ total 46G -rw-r--r-- 1 nobody users 22M Jan 19 2023 nzbhydra.trace.db.old -rw-r--r-- 1 nobody users 13M Sep 5 08:11 nzbhydra.trace.db -rw-r--r-- 1 nobody users 46G Sep 17 22:28 nzbhydra.mv.db @.:# ls -ltrh /mnt/user/System/Docker/AppData/nzbhydra2/database/ total 46G -rw-r--r-- 1 nobody users 22M Jan 19 2023 nzbhydra.trace.db.old -rw-r--r-- 1 nobody users 13M Sep 5 08:11 nzbhydra.trace.db -rw-r--r-- 1 nobody users 46G Sep 19 12:36 nzbhydra.mv.db @.:# @.:# ls -ltrh /mnt/user/System/Docker/AppData/nzbhydra2/database/ total 46G -rw-r--r-- 1 nobody users 22M Jan 19 2023 nzbhydra.trace.db.old -rw-r--r-- 1 nobody users 13M Sep 5 08:11 nzbhydra.trace.db -rw-r--r-- 1 nobody users 45G Sep 19 12:36 nzbhydra.mv.db @.***:~#

— Reply to this email directly, view it on GitHub https://github.com/theotherp/nzbhydra2/issues/866#issuecomment-1726211092, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADNUA6NG55THG2YCWLWZWHTX3HKU5ANCNFSM6AAAAAAZCC75TI . You are receiving this because you commented.Message ID: @.***>

MicMillon commented 12 months ago

nzbhydra-debuginfos-2023-09-19-15-10.zip

Attached

MicMillon commented 11 months ago

Hey, any feedback from the attached debuginfo file?

i'd really like to downsize that database...

theotherp commented 11 months ago

Try creating a backup. Does that work? Is the file in there so big as well? If it's smaller just replace the existing one with the one from the backup while hydra is down. See if it stays smaller.

MicMillon commented 11 months ago

Hey, so i tried that.. created the backup. shutdown the container, extracted the 19g nzbhydra.mv.db file from the backup, replaced the 45g file in the /appdata/nzbhydra/database directory, and restarted the hydra container. it came up, with my data there. with the database sitting around 20gb... came back a day or two later, and now its back up to the same 45g as before...