Add command to download manga information only

[Theo1996]

The idea

Add option to download a file with all the titles of the manga in the Library, even better if you can download small cover art ,author,artist,summary, multiple TLs of the title etc

Why this feature should be added to the app ?

It is very important to have a backup of all the manga you are reading because like other website if it shuts down you are fucked.

[mansuf]

Add option to actually downlaod all titles from Library

Isn't that feature already exists ? You can do it with library command:

Note: This require an authentication

mangadex-dl "library" --login

You can read here for more information -> https://mangadex-dl.mansuf.link/en/stable/cli_ref/manga_library.html

even better if you can download small cover art

This already exists too, you can do it with cover command

mangadex-dl "cover:https://mangadex.org/title/f9c33607-9180-4ba6-b85c-e4b5faee7192/official-test-manga"
# You will be prompted to select which cover you wanna download
# If you don't wanna be prompted, use --input-pos option

You can read here for more informations -> https://mangadex-dl.mansuf.link/en/stable/cli_ref/cover.html

[Theo1996]

What I meant is that if I select the "library" option it just prints 10 lines of manga titles and you need to type next for the next 10, I wan t to download just the titles/names of all my manga in my library to a text file or CSV. If I use the cover command I need to do it for every manga title in my library manually.

[mansuf]

What I meant is that if I select the "library" option it just prints 10 lines of manga titles and you need to type next for the next 10, I wan t to download just the titles/names of all my manga in my library to a text file or CSV.

No, it won't happen. But, you can do some hacking using internal API with python if you want to.

Just a reminder that the internal API can be changed at any time without notice.

from mangadex_downloader.network import Net
from mangadex_downloader.iterator import IteratorUserLibraryManga

# Login first
Net.mangadex.login(username="YOUR USERNAME", password="YOUR PASSWORD")

# List all manga titles in the library and write it to a file called "manga titles.txt"
fp = open("manga titles.txt", "w+")
for manga in IteratorUserLibraryManga():
    fp.writelines(manga.title + "\n")
fp.close()

[Theo1996]

1.I dont really understand why it cant be done. The API limits to 100 manga names/IDs per request cant your script just continue sending new requests till the total number of IDs in the Library of a user is reached ? I have aprox 3500 manga, so it would send 35 requests and put all that in a text file or something.

2. I also saw a limit variable in some part of your code which equals to "10", isnt that the 10 entries you get when requesting "library" ? Cant that also be increased or set by the user(though that is not very important)?

[mansuf]

I dont really understand why it cant be done. The API limits to 100 manga names/IDs per request cant your script just continue sending new requests till the total number of IDs in the Library of a user is reached ? I have aprox 3500 manga, so it would send 35 requests and put all that in a text file or something.

Command library are using IteratorUserLibraryManga class which i demonstrated how to get all manga from the user library with internal API. So from techincal perspective, the application is capable of retrieving all manga from the user library. And yes it respect API limits, so it would send multiple requests if the user have tons of manga in the library.

However, i don't understand when you say "just to download titles only", not to mention the other optional informations such as small cover art, author, artist, summary, multiple TLs of the title, etc. It's like download a manga in .cbz file and then remove chapters and images and you would left with ComicInfo.xml and the manga cover. And you also mention your purpose is to backup all manga from your library, which is doesn't make sense at all.

I'm not trying to be rude here, but can you explain more details why you want this feature ?

If you want to use this feature for analysis things (like this https://github.com/mansuf/mangadex-downloader/issues/60) because you say you wanna save manga titles to CSV file, i can create a separate command just for that.

I also saw a limit variable in some part of your code which equals to "10", isnt that the 10 entries you get when requesting "library" ? Cant that also be increased or set by the user(though that is not very important)?

The limit variable is for per page in library command, so when you using library command, you will be displayed first 10 manga that you have in the library, then if you want to see the next one, you could just type next and then it will display next 10 manga in the library.

[Theo1996]

What I suggested was to download JUST titles/name of the manga ,meaning only the text of the name that already appears in CMD, if you use the librarycommand, in groups of 10. The optional information, which I dont really care for but is nice to have, are what you and I already mentioned, but together with the titles(name/text) of each manga would make a more rich(cover,tags) backup of my library index. And by index I mean just a list of strings for every manga title/name. Basically the same thing that appears when I click on library in mangadex, but it has all 3500 titles instead of 100. Sorry for the misunderstanding.My original post wasnt clear enough.

If you want to use this feature for analysis things (like this https://github.com/mansuf/mangadex-downloader/issues/60) because you say you wanna save manga titles to CSV file, i can create a separate command just for that.

Yes, please. Maybe add the IDs too.

The limit variable is for per page in library command, so when you using library command, you will be displayed first 10 manga that you have in the library, then if you want to see the next one, you could just type next and then it will display next 10 manga in the library.

Yes ,I understood that.

[mansuf]

Alright, i will add this on my to-do list in v3.0.0 PR .

Keep in mind, i don't have ETA when this features is implemented, because i have to do other things (documentations, etc).

[mansuf]

Hello, the feature has been added in https://github.com/mansuf/mangadex-downloader/commit/5af3e31684f02e99d6014ba201b30be5a542a440. You can install development version if you want to test it.

Installtion with Git & pip

pip uninstall mangadex-downloader
pip install -U git+https://github.com/mansuf/mangadex-downloader.git@40041783ec97ec92ad8ec08f4888f280beb8cede

Installation for bundled executable (Windows only)

You can grab it in here -> https://github.com/mansuf/mangadex-downloader/actions/runs/9372055514
You can download it once it show success and it's available at Artifacts section

---

The feature is accesible from --create-manga-info and it's customizable. By default it save to CSV format

Usage:

mangadex-downloader "MangaDex URL" --create-manga-info

You can save it another format (JSON) using --manga-info-format

mangadex-dl "MangaDex URL" --create-manga-info --manga-info-format "json"

Also you can customize it where you want to place the file using --manga-info-filepath. By default, it will set to {download_path}/manga_info.{manga_info_format}

The placeholder {download_path} will resolve to value from --path option And the placeholder {manga_info_format} will resolve to value from --manga-info-format option

So for example, if you set --path to /home/sussybaka/manga and --manga-info-format to json The --manga-info-filepath will be resolved to /home/sussybaka/manga/manga_info.json

mangadex-dl "MangaDex URL" --create-manga-info --manga-info-filepath "./manga_index.{manga_info_format}"

For more information about placeholders, you can see it here -> https://mangadex-dl.mansuf.link/en/v3.0.0/cli_ref/path_placeholders.html

If you have a problem, let me know 👍

[Theo1996]

it works , but its only for individual manga urls. It was supposed to iterate over all the manga titles in library and save them all to a .csv. Maybe I am missing something....

Basically the same thing that appears when I click on library in mangadex, but it has all 3500 titles instead of 100. Sorry for the misunderstanding.My original post wasnt clear enough. from: https://github.com/mansuf/mangadex-downloader/issues/99#issuecomment-2023818576

[mansuf]

You just have to modify --manga-info-filepath to somewhere else if you want it one place. By default it placed each manga folders

[Theo1996]

Modifying the filepath doesnt change the fact that it only works for for one manga URL that I provide, Didnt we agree it was supposed to download all the info of each title in library. I tried using the library command but it doesnt work with create-info. Like the quote I posted earlier.

[mansuf]

It works for library command and all other commands such as list, followed-list, random, etc

python -m mangadex_downloader "library" --login --login-username "REDACTED" --login-password "REDACTED" --create-manga-info --manga-info-filepath "./manga_info.{manga_info_format}" --input-pos "*" --path "G:\Test\Manga\DownloadTest"

Output: manga_info.csv

---

However, you might found an issue where some manga cannot be written to .csv files due to file encoding. I'm gonna make a patch for this.

[INFO] Logging in to MangaDex
[INFO] Logged in to MangaDex
[INFO] Checking url = 3ae0025b-18ac-40c4-a1b5-1e8d0cd89a42
[INFO] Using English language
[INFO] Fetching all chapters...
[INFO] Download directory is set to "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\Ikinari-kon - Reijou Joshikousei Yumemigachiru"
[INFO] Using raw format
[INFO] Downloading cover manga Ikinari-kon - Reijou Joshikousei Yumemigachiru
File sizes: 100%|████████████████████████████| 350k/350k [00:00<00:00, 1.11MB/s]
[INFO] Download finished for manga "Ikinari-kon - Reijou Joshikousei Yumemigachiru"
[INFO] Checking url = 2b224079-1b78-43b6-ba24-df18c132c27f
[INFO] Using English language
[INFO] Fetching all chapters...
[INFO] Download directory is set to "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\Himitsu no GalShota"
[INFO] Using raw format
[INFO] Downloading cover manga Himitsu no GalShota
File sizes: 100%|██████████████████████████| 2.25M/2.25M [00:00<00:00, 3.81MB/s]
[INFO] Download finished for manga "Himitsu no GalShota"
[INFO] Checking url = c3295871-b930-4ee3-ac0f-ff7102ecec4e
[INFO] Using English language
[INFO] Fetching all chapters...
[INFO] Download directory is set to "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\Sasha-chan to Classmate Otaku-kun"
[INFO] Using raw format
[INFO] Downloading cover manga Sasha-chan to Classmate Otaku-kun
File sizes: 100%|██████████████████████████| 1.05M/1.05M [00:00<00:00, 3.94MB/s]
[INFO] Download finished for manga "Sasha-chan to Classmate Otaku-kun"
[INFO] Checking url = 2305416c-d758-4043-a451-4fe658b0408f
[INFO] Using English language
[INFO] Fetching all chapters...
[INFO] Download directory is set to "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\Boku no Kanojo ga Kyonyuu Dattara. Oppai Anthology Comic"
[INFO] Using raw format
[INFO] Downloading cover manga Boku no Kanojo ga Kyonyuu Dattara. Oppai Anthology Comic
File sizes: 100%|████████████████████████████| 971k/971k [00:00<00:00, 4.51MB/s]
[INFO] Download finished for manga "Boku no Kanojo ga Kyonyuu Dattara. Oppai Anthology Comic"
[INFO] Checking url = 73f589e1-0f08-4f3f-9bbe-eb882c87d7d1
[INFO] Using English language
[INFO] Fetching all chapters...
[INFO] Download directory is set to "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\Osananajimi no Ohimesama"
[INFO] Using raw format
[INFO] Downloading cover manga Osananajimi no Ohimesama
File sizes: 100%|████████████████████████████| 391k/391k [00:00<00:00, 3.91MB/s]
Traceback (most recent call last):
  File "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\mangadex_downloader\cli\__init__.py", line 87, in _main
    download(args)
  File "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\mangadex_downloader\cli\download.py", line 34, in download
    url(args, args.type)
  File "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\mangadex_downloader\cli\url.py", line 187, in __call__
    self.func(self.id, args)
  File "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\mangadex_downloader\cli\url.py", line 105, in download_manga
    dl_manga(*args)
  File "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\mangadex_downloader\main.py", line 200, in download
    download_manga(manga, formatted_path)
  File "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\mangadex_downloader\main.py", line 133, in download_manga
    create_manga_info(path, m, replace)
  File "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\mangadex_downloader\utils.py", line 363, in create_manga_info
    manga_info.write()
  File "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\mangadex_downloader\manga.py", line 320, in write
    self.write_to_csv()
  File "G:\NextcloudData\PROJECT\Programming\mangadex-downloader\mangadex_downloader\manga.py", line 378, in write_to_csv
    writer.writerow(row)
  File "C:\Users\mansuf\AppData\Local\Programs\Python\Python312\Lib\csv.py", line 164, in writerow
    return self.writer.writerow(self._dict_to_list(rowdict))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\mansuf\AppData\Local\Programs\Python\Python312\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\u2103' in position 26: character maps to <undefined>

[Theo1996]

I run your command and it start downloading the covers and creates a manga_info.csv but it fails on the second or third manga with the error :

Traceback (most recent call last): File "E:\python\python310\lib\site-packages\mangadex_downloader\cli__init__.py", line 87, in _main download(args)

File "E:\python\python310\lib\site-packages\mangadex_downloader\cli\download.py", line 34, in download url(args, args.type) File "E:\python\python310\lib\site-packages\mangadex_downloader\cli\url.py", line 187, in call self.func(self.id, args) File "E:\python\python310\lib\site-packages\mangadex_downloader\cli\url.py", line 105, in download_manga dl_manga(*args) File "E:\python\python310\lib\site-packages\mangadex_downloader\main.py", line 200, in download download_manga(manga, formatted_path) File "E:\python\python310\lib\site-packages\mangadex_downloader\main.py", line 133, in download_manga create_manga_info(path, m, replace) File "E:\python\python310\lib\site-packages\mangadex_downloader\utils.py", line 363, in create_manga_info manga_info.write() File "E:\python\python310\lib\site-packages\mangadex_downloader\manga.py", line 320, in write self.write_to_csv() File "E:\python\python310\lib\site-packages\mangadex_downloader\manga.py", line 378, in write_to_csv writer.writerow(row) File "E:\python\python310\lib\csv.py", line 154, in writerow return self.writer.writerow(self._dict_to_list(rowdict)) UnicodeEncodeError: 'cp932' codec can't encode character '\u2014' in position 104: illegal multibyte sequence

I dont know if it goes further than the 10th entry though.

After searching a little bit on google I found that maybe you have to use this: text = text.encode('cp932', errors='replace').decode('cp932')

[mansuf]

Fixed in https://github.com/mansuf/mangadex-downloader/commit/a0fe2c5fb8b08153ff9f034b554dce85e5eb8897

Installtion with Git & pip

pip uninstall mangadex-downloader
pip install -U git+https://github.com/mansuf/mangadex-downloader.git@a0fe2c5fb8b08153ff9f034b554dce85e5eb8897