search

MalloyDelacroix / DownloaderForReddit

The Downloader for Reddit is a GUI application with some advanced features to extract and download submitted content from reddit.
GNU General Public License v3.0
498 stars 47 forks source link

Painfuly slow preformance when deleting and database issue. #203

Open T-prog3 opened 3 years ago

T-prog3 commented 3 years ago

  1. When i choose to delete a user and all its files within the software i end up with waiting times that is riddiculous every time. Let say i have a user with 900 images downloaded, To delete all these files and database content it take about 8-15 minutes with 1 file being deleted every sec or so and it's not even large files. In the meantime i'm not allowed to do anything within the software. So all i can do is sit and watch 1 file disappare after another. Imagine doing this with even more files.

  2. No matter how many users i delete there's no effect on the size of the database file. Not even 1 byte gets removed from it when deleting. But the removed content is no longer visible within the software. So in other words it can gain in size but never lose it and over time we end up with lots of wasted storage space.

crccheck commented 3 years ago 
2. No matter how many users i delete there's no effect on the size of the database file.

Try doing a sqlite VACUUM. I ran this command in OSX (It's different for Windows):

$ sqlite3 dfr.db 'VACUUM;'

And the file size went from 192081920 to 183353344

thany commented 1 year ago

I think Sqlite may have been a suboptimal choice. It's really slow for deleting records. Like REALLY slow. There are optimisations that can be done, but these will only make it less slow - it will never be fast. I've ran into this with other programs too.

The "unfortunate" thing about this, is that Sqlite is really convenient to use, making it a very tempting choice for application storage.

Even Firefox uses Sqlite, but it does so by maintaining separated (smaller) Sqlite files for very specific types of data. That way if one becomes slow due to many deletions, the other(s) will still perform. Perhaps if stepping away from Sqlite is out of the question, splitting up the database in some way might be an option to explore.