[FEATURE] Logging and Progress Bar

picnixz commented 1 year ago

The idea

I would like to suggest reducing the number of logging messages and progress bars by replacing them by up stacked progression bars. More precisely,

Only print relevant messages on the console e.g., when starting a download or when a chapter / volume / part is completed depending on the merging procedure. For instance, when saving the output as cbz-volume, I am only interested when the whole is done and not when each page of each chapter is done. Such messages should be logged in DEBUG mode only.
By default, the logging verbosity should be rather WARNING than INFO, or it should be possible to suppress info messages. I haven't looked at the source code itself, but is there a way to configure the logger (if it is actually a logger) using an environment variable?
For progression bars, since tqdm supports stacked bars, it would be easier and cleaner to have a progression bar for the 1) volume (progression over the chapters), 2) the chapter volume (progression over the pages) and 3) the page download as it is currently. It should be possible to only show the volume progression or the volume + chapter or all bars.

I'll be happy to contribute to that but I'll need to dive into the code beforehand.

Why this feature should be added to the app ?

Having too many messages makes the terminal output quite unreadable. I have no idea about the ETA and I also have no idea which manga I am currently downloading if I am using the file:URL syntax.

mansuf commented 1 year ago

For progression bars, since tqdm supports stacked bars, it would be easier and cleaner to have a progression bar for the 1) volume (progression over the chapters), 2) the chapter volume (progression over the pages) and 3) the page download as it is currently. It should be possible to only show the volume progression or the volume + chapter or all bars.

Since it's stacked progress bars, the application should not have other outputs to the console while the progress bars is exist. So you cannot see soft errors or warnings while it's downloading (because the console could be messy).

Example of soft errors:

"xxx" is not valid MangaDex URL
Manga "xxx" has no *some language* chapters

You may notice these soft errors when you're using batch download.

So, are you okay with this design ? I don't have a problem for reduced logger output part. I just have to add option to change logger level.

mansuf commented 1 year ago

From your idea about stacked progress bar, the output should be like this (Correct me if i'm wrong):

*Some logger output before starting to download manga*

Pages progress bar
Chapters progress bar
Volume progress bar

# Next manga

*Some logger output before starting to download manga*

Pages progress bar
Chapters progress bar
Volume progress bar

Or if you want to show progress bar only (but stacked), i can make it at least 5 total progress bars

Pages
Chapters
Volume
Manga downloaded (if you're downloading MangaDexList)
Link processed (if you're using batch download)

picnixz commented 1 year ago

From your idea about stacked progress bar, the output should be like this (Correct me if i'm wrong):

Yes that's what I thought about. In addition, error messages can technically be shown before the progress bar using ANSI escape codes (not sure if it is well supported on Win32 systems though). This can be done by clearing the progression bar, printing the error, and re-rerendering normally. In general (soft) errors should not happen much also.

EDIT:

I actually thought about:

Progress Volume
Progress Chapter
Progress Page

instead.

mansuf commented 1 year ago

I have made a PR #72 that will close this issue, you can start test it by installing from development version.

Command for PyPI:

NOTE: That you must have git in order to install this version

pip install git+https://github.com/mansuf/mangadex-downloader.git@12201e7713d27659ea87c5f04d794701ffbea0a9

If you're installing mangadex-downloader from github releases, you can download the development version in here:

The download files are available in "artifacts" section

https://github.com/mansuf/mangadex-downloader/actions/runs/5208983563

Stacked progress bar

For stacked progress bar, i've made a option that you can use to switch from default progress bar to stacked mode. The option are --progress-bar-layout or -pbl.

The option accept one value from 3 options, the options is

default (individual progress bars)
stacked
none (no progress bar)

Example usage

mangadex-dl "insert manga url here" --progress-bar-layout "stacked"

By default, stacked progress bar order are

volumes
chapters
pages
file sizes
convert

Of course, you can customize it by using --stacked-progress-bar-order or -spb-order. It support multiple values separated by comma.

mangadex-dl "insert manga URL here" -pbl "stacked" -spb-order "volumes, chapters, pages"

Change logging level

As for this, i've made a new option that you can use to change logging level, the option are --log-level

For more information about logging levels you can see in https://docs.python.org/3/library/logging.html#logging-levels. It support both string and number (integer) value.

Example usage

mangadex-dl "insert manga url here" --log-level "WARNING"

Let me know when you have issues when using this new features 😀. If you encountered bugs when testing, please report it in here, don't make another issue. I just don't wanna get confused 😁

picnixz commented 1 year ago

Thanks a lot ! I'll review your PR when I have a bit of time (if you don't mind waiting a bit).

EDIT: I'll test it this w-e.

mansuf / mangadex-downloader