It's appeared that CPU usage has been increased when downloading large chapters in v2.6.0 and upper. Causing application slowing down when downloading.
How is this happened ?
First, let me tell you about "Download tracking" system that has been added in v2.6.0. It's tracking downloaded chapters and images and it's purpose to check what chapters has been downloaded, so next time you run the application, it only download the chapters that has not been downloaded (latest chapters) without getting rate-limited by MangaDex API (see commit https://github.com/mansuf/mangadex-downloader/commit/865b7f5988a9cd92e21112ac8649a29299b5023f for more info). Also it verify chapters and images, so if you have corrupted chapters or images, the application will know and will re-download them.
The "Download tracking" system use JSON (JavaScript Object Notation) to store the information about downloaded chapters and images. I use JSON because it's easy to implement it. But, turns out it hurts application performance if it was downloading large chapters (i just tested it out). So yes i was messed up, i'm sorry. I only do basic testing for this project due to limited time i have.
The solution
In v2.8.0, i will change the type storage for "Download tracking" system from JSON to SQLite. SQLite handles data efficiently even it's containing large data. If you don't wanna wait for v2.8.0, you can use older version (before v2.6.0). The tracker can be disabled but it's not available yet (see #45)
mansuf
commented
1 year ago
What happened ?
It's appeared that CPU usage has been increased when downloading large chapters in v2.6.0 and upper. Causing application slowing down when downloading.
How is this happened ?
First, let me tell you about "Download tracking" system that has been added in v2.6.0. It's tracking downloaded chapters and images and it's purpose to check what chapters has been downloaded, so next time you run the application, it only download the chapters that has not been downloaded (latest chapters) without getting rate-limited by MangaDex API (see commit https://github.com/mansuf/mangadex-downloader/commit/865b7f5988a9cd92e21112ac8649a29299b5023f for more info). Also it verify chapters and images, so if you have corrupted chapters or images, the application will know and will re-download them.
The "Download tracking" system use JSON (JavaScript Object Notation) to store the information about downloaded chapters and images. I use JSON because it's easy to implement it. But, turns out it hurts application performance if it was downloading large chapters (i just tested it out). So yes i was messed up, i'm sorry. I only do basic testing for this project due to limited time i have.
The solution
In v2.8.0, i will change the type storage for "Download tracking" system from JSON to SQLite. SQLite handles data efficiently even it's containing large data. If you don't wanna wait for v2.8.0, you can use older version (before v2.6.0). The tracker can be disabled but it's not available yet (see #45)