search

keybase / keybase-issues

A single repo for managing publicly recognized issues with the keybase client, installer, and website.
902 stars 37 forks source link

Lots of unused files in keybase.leveldb #3258

Open sciurius opened 6 years ago

sciurius commented 6 years ago

In my .local/share/keybase/keybase.leveldb there are >100 XXXX.ldb files, some of them very old. Are these files still necessary?

maxtaco commented 6 years ago

they are temporary, and you can nuke them, it'll just slow down your performance. to nuke them, issue keybase db nuke

wjblanke commented 5 years ago

My keybase.leveldb folder is 17 GB and has been running just a month. Does it really need to be this big? Eventually all the ldbs and such fills up my storage and then KeyBase hangs because of no disk space. eg:

...

-rw-r--r-- 1 ubuntu ubuntu 2108854 Dec 7 15:10 580388.ldb -rw-r--r-- 1 ubuntu ubuntu 2108306 Dec 7 15:11 580443.ldb -rw-r--r-- 1 ubuntu ubuntu 2108672 Dec 7 15:11 580444.ldb -rw-r--r-- 1 ubuntu ubuntu 17 Dec 17 20:01 CURRENT -rw-r--r-- 1 ubuntu ubuntu 0 Nov 19 19:36 LOCK -rw-r--r-- 1 ubuntu ubuntu 137152405 Dec 21 04:42 LOG -rw-r--r-- 1 ubuntu ubuntu 202561906 Dec 17 19:58 LOG.old -rw-r--r-- 1 ubuntu ubuntu 59364172 Dec 21 04:42 MANIFEST-2724950 ubuntu-xxxxxxxxxx:~/.local/share/keybase/keybase.leveldb$ du -sh ../keybase.leveldb 17G ../keybase.leveldb ubuntu-xxxxxxxxxx:~/.local/share/keybase/keybase.leveldb$

maxtaco commented 5 years ago

Hmm, that's pretty big, way bigger than mine, and I've been running for years on this machine. Is there anything notable about your use pattern?

wjblanke commented 5 years ago

The KeyBase account is communicating large messages with a large number of teams containing around 10 users each. I'm hesitant to just nuke everything as KeyBase might grind to a halt reconstructing the DBs on next run. So I just keep increasing the partition...

wjblanke commented 5 years ago

FWIW, now 21 GB

... -rw-r--r-- 1 ubuntu ubuntu 2110511 Dec 27 18:06 5608462.ldb -rw-r--r-- 1 ubuntu ubuntu 2114433 Dec 7 14:06 571725.ldb -rw-r--r-- 1 ubuntu ubuntu 2108448 Dec 7 14:06 571727.ldb -rw-r--r-- 1 ubuntu ubuntu 17 Dec 17 20:01 CURRENT -rw-r--r-- 1 ubuntu ubuntu 0 Nov 19 19:36 LOCK -rw-r--r-- 1 ubuntu ubuntu 480530297 Dec 27 18:06 LOG -rw-r--r-- 1 ubuntu ubuntu 202561906 Dec 17 19:58 LOG.old -rw-r--r-- 1 ubuntu ubuntu 208258146 Dec 27 18:05 MANIFEST-2724950 ubuntu:~/.local/share/keybase/keybase.leveldb$ du -sh ../keybase.leveldb 21G ../keybase.leveldb ubuntu:~/.local/share/keybase/keybase.leveldb$

ToxicFrog commented 5 years ago

Only 21GB? Watch this:

$ du -shc --apparent-size .local/share/keybase/keybase.leveldb  
53G     .local/share/keybase/keybase.leveldb

I noticed this when I got a warning that the size of the /home backup had dramatically spiked.

I'm in two teams, one of them keybasefriends and the other a small (<100 users) private team with low traffic, and that's all I use it for.

I'm going to exclude these from backups and do a db nuke now, but yeesh.

If it helps debug this at all, I've been experimenting with rolling my own client, which means there's pretty much always a keybase chat api-listen running, frequent calls to keybase chat read and keybase chat list, and occasional calls to keybase chat list-channels, keybase chat list-members, and keybase chat send -- could repeatedly invoking the command line tools inflate the db more than just leaving the X client running does?

ToxicFrog commented 5 years ago

Also, given that these can apparently be deleted at any time and keybase will keep on trucking and simply recreate the ones it needs, with no ill effects other than temporary performance loss, they really should be in ~/.cache/ rather than ~/.local/share/, or marked with CACHEDIR.TAG, or both.

maxtaco commented 5 years ago

Are you running the latest version? We put in a cleaner. Cc @joshblum

ToxicFrog commented 5 years ago

Aah, I'm not -- looks like nixpkgs is an entire major version behind. Thanks for pointing that out. I'll update and see if it recurs.

joshblum commented 5 years ago

@ToxicFrog if you still see this after the upgrade can you do a log send so I can take a look? Note the cleaner won't instantly kill everything but will slowly work to remove old data

mlsteele commented 5 years ago

You can use this script to see the sizes of each section of a db. Keybase has to not be running when running the script.

For example on macOS

python ldbsizes.py ~/Library/Application\ Support/Keybase/keybase.leveldb
nealmcb commented 5 years ago

I also find the DB disconcertingly big. I'd love to understand the expected behavior a bit more. What version did the cleaner go in? How quickly does it remove old data? How much of a performance hit, for what, might we expect?

I tried to run the script, but it looks like it needs at least the plyvel module, perhaps at https://plyvel.readthedocs.io/en/latest/index.html

What other requirements does it have?

Thanks!

Update - perhaps this is the cleaner PR? https://github.com/keybase/client/pull/16033

joshblum commented 5 years ago

@nealmcb how big is your db? What version are you running? The cleaner has been in place since 3.1.2 and should keep the desktop/mobile db sizes to 2GB and 1GB respectively.

nealmcb commented 5 years ago

Ahh - good to know the details and settings. Mine is 2 GB, as I guess you expect. Still seems like a lot. Is it configurable? Any way to find out how often the 4-month old files in there are actually used, how much bandwitdth/time they save, etc? 

du ~/.local/share/keybase/keybase.leveldb
2081096

keybase version 4.2.0-20190710182743+e7c0bdc4a2
joshblum commented 5 years ago

Not yet configurable, we would like to add a setting that covers all of the various caches (kbfs, file attachments, these databases) but it hasn't been prioritized yet. You can run keybase db nuke if you want to see the app running from a clean db state, will likely take a while to fill to 2gb again.

taw00 commented 4 years ago

This is related to the original inquiry only in spirit. It's not the DB that is the issue for me but the journal.

The machine I am running KB on has only 55G of storage space. This is a bit insane. I use KBFS a bunch, but ... ??? Is there a way I can cap the usable space? I mean beyond creating a separate partition just for .local/share/keybase?

3.3G    kbfs_block_cache
33G kbfs_journal
111M    keybase.chat.leveldb
331M    keybase.leveldb

TL;DR: Go to the end and read my "Moral(s) of the story" statement.

UPDATE: I took a look at /keybase/.kbfs_status and I see a TON of UnflushedPaths listed (note, I see no conflicts, but they all do lead down a path that no longer exists):

"JournalManager": {
    "RootDir": "/home/todd/.local/share/keybase/kbfs_journal/v1",
    "Version": 1,
    "CurrentUID": "886108336ddb49377246f86cbdf13519",
    "CurrentVerifyingKey": "0120e586fec1137de58dd400fc037e16e8d424fe02462f9aff1357c406b1ac39c3060a",
    "EnableAuto": true,
    "EnableAutoSetByUser": false,
    "JournalCount": 2,
    "StoredBytes": 27772458361,
    "StoredFiles": 1920947,
    "UnflushedBytes": 27771149396,
    "UnflushedPaths": [
... long LONG list of file paths follows ...

Anyone know how to flush the journal?

UPDATE: as stated above, the paths don't exist because a directory common to all of them was nuked (probably by me) before they were flushed from the journal (I think?). Anyway. I do this...

$ keybase fs stat <one of the files listed>
▶ ERROR adult-desert doesn't exist

When I examine the missing directory...

$ keybase fs stat <the common missing directory>
▶ ERROR file or folder does not exist (code 5103)

I need not tell you that "adult-desert" is a bizarre and unhelpful (to me) error. There is no file with that filename and I don't think there ever was. It gives the same error for all the files listed.

Interestingly keybase fs uploads lists all the same files and 25.86 GB left to upload.

I attempted a keybase fs cancel-uploads /keybase/private/<username> and ... it failed with an apparent conflict I haven't seen evidence of, but it must be there.

Then I did a keybase fs clear-conflicts/keybase/private/` ...waited a long time (keybase really needs progress/activity indicators by the way)...

BAM! Hit the limits of the available storage.

▶ ERROR Disk limit timeout of 11s reached; requested 16441 bytes and 7 files, -4138658733 bytes and 803206 files available: context deadline exceeded

UPDATE: now in weird state. I did this:

$ keybase fs finish-resolving-conflicts /keybase/private/<username>
▶ ERROR 3b8ffab7bf6edd33df6c59fd552a2f16 is not a cleared conflict journal

Then I tried clear-conflicts again just to see ...

$ keybase fs clear-conflicts /keybase/private/<username>
▶ ERROR Single expected revision 0 not found

Moral(s) of the story:
Everything is fine now, but...