search

zadam / trilium

Build your personal knowledge base with Trilium Notes
GNU Affero General Public License v3.0
27.2k stars 1.9k forks source link

Deleted notes seem to persist in database. #908

Closed drawtheliner closed 2 years ago

drawtheliner commented 4 years ago

I want to understand how my data is kept locally, so I went into: C:\Users\USERNAME\AppData\Roaming\trilium-data

I found what I believe to be the database where all data is kept: document.db (I opened that in Notepad and indeed could search and find parts of my data there)

However, after I deleted all notes in Trilium, this file didn't decrease size (it actually increased a bit). Again, opening it in Notepad I could search and find parts of the data I had already deleted.

Before doing that I went into Options/Other and set Note erasure timeout to 5 seconds. The above happened after I waited much longer than that, including a few restarts.

Did I miss something or is this expected behaviour?

zadam commented 4 years ago

So I think there are two things at play:

  1. even though you can set the timeout at 5 seconds, it won't be necessarily erased after 5 seconds, but after up to 4 hours. This is because this feature was not designed with such short interval in mind. This might be fixed in a future release though if there is a use case for shorter interval.

  2. You can't really expect that the DB file will go down in size after deleting (and erasing) a note - this is not how database engines work. It's a bit similar to standard file system that deleted data is only marked as deleted and might be reused later. It is possible to run VACUUM command which will compact the database file and thus remove all space marked as deleted ... If you want an on-disk confidentiality of your data, it's better to use protected notes ...

drawtheliner commented 4 years ago

Thanks!

After 24 hours my data is still identifiable in the .db file. After vacuum it did clean up (and get smaller).

I don't really need the data to be encrypted locally. But I did expect data to be actually deleted eventually once I deleted it within the app.

zadam commented 4 years ago

But I did expect data to be actually deleted eventually once I deleted it within the app.

Why do you want this though? Even after VACUUM, your filesystem is probably not erasing the data from the hard disk anyway, just marking it as free.

drawtheliner commented 4 years ago

Yes, but is it searchable?

For example, suppose I have a file called notes.txt with the word "Blablabla" in there. If I delete it in Windows (and clear Recycle bin), there is no file where someone could search for "Blababla" and find it (I suppose an expert could probably do it using other means, but not as easily as it was to find my deleted content in the Trilium .db file).

I'm clearly not a dev, so sorry if I'm saying nonsense!

Heniker commented 4 years ago

document.db is the SQLite database file. You should not open it with programs like notepad, because you're basically viewing binary, which is not useful by most means.

Deleted data stays for a while in most databases - this is normal behavior, because it is faster for a DB to keep that data instead of actually deleting it.

To actually remove data you should compile SQLite with SQLITE_SECURE_DELETE or set SECURE DELETE PRAGMA on database.