search

mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.67k stars 952 forks source link

[mangadex] same subcategory for title and for chapter #6038

Closed dajotim937 closed 1 month ago

dajotim937 commented 2 months ago

gallery-dl.exe https://mangadex.org/title/0fa5d834-fdab-4ac4-b3c9-821f1580f991/ -K

and

gallery-dl.exe https://mangadex.org/chapter/5c62ad6e-3717-4407-a82f-c9e6517c36b3 -K

Both return:

subcategory
  chapter

Also, probably would be better to set "title" as '' (empty string) instead of 'None' if there is not any title for chapter. But it's just my 2 cents, because usually I didn't need to format variable if there is not any value in existing variable. In my cases None usually returns if I mess up and put nonexistent variable into filename/directory.

mikf commented 2 months ago 
$ gallery-dl -E https://mangadex.org/title/0fa5d834-fdab-4ac4-b3c9-821f1580f991/
Category / Subcategory
  "mangadex" / "manga"

-K displays the results of child extractors when appropriate (chapter results in this case)

Also, probably would be better to set "title" as '' (empty string) instead of 'None' if there is not any title for chapter

{title|''}
dajotim937 commented 2 months ago

K displays the results of child extractors when appropriate (chapter results in this case)

What the difference with bato?

gallery-dl.exe https://bato.to/title/115413 -K - subcategory: manga gallery-dl.exe https://bato.to/title/115413/2973463 -K - subcategory: chapter

I'm asking because gallery-dl.exe https://mangadex.org/title/0fa5d834-fdab-4ac4-b3c9-821f1580f991/ doesn't trigger 

"manga": {
    "directory": [
        "{manga}",
        "c{chapter:>03}{chapter_minor} {title:?: //}"
    ]
},

from config. But if I remove "manga" subcategory, it works fine (1st execute without "manga" subcategory, 2nd with, witch trigger default directory template). image

But then I cant separate download full manga and download only chapter, because I have 2 different directory template for when I download whole manga and when only one chapter.

Hrxn commented 2 months ago

https://github.com/mikf/gallery-dl/blob/0bbdeaaa504541c1b9fb62396e228cd99fdff25f/gallery_dl/extractor/mangadex.py#L130-L136

Your URL here should definitely be manga subcategory.. Are you sure your config is correct?

dajotim937 commented 2 months ago

Are you sure your config is correct?

"mangadex": {
    "sleep": 0.2,
    "manga": {
        "directory": [
            "{manga} TEST TEST TEST",
            "c{chapter:>03}{chapter_minor} {title:?: //}"
        ]
    },
    "filename": "c{chapter:>03}{chapter_minor}_{page:>03}.{extension}"
},

image

Yes, I'm sure. And, like I said in first comment, it shows subcategory: chapter for https://mangadex.org/title/*. Which triggers problem, I guess. Because it's first time when I see such situation with gallery-dl, when -K shows child extactor subcategory instead of parent extractor subcategory.

Hrxn commented 2 months ago

What is your result of gallery-dl -E YourURL?

dajotim937 commented 2 months ago

image

Hrxn commented 2 months ago

Any post-processors active? Like metadata with modify or something? Post a verbose log of that input URL, you should see post-processors at the beginning.

dajotim937 commented 2 months ago

Any post-processors active? Like metadata with modify or something?

No, I don't have any global postprocessors, only couple specific for extractors. And I already showed my config for mangadex. image

I understand why you are asking so many "check if", but it's not my first bug report. For every other extractor subcategory settings triggers as intended and shows as intended in -K output. Bato example is above. It has same config as mangadex. And it shows correctly subcategory for manga and chapter. With same settings for mangadex (which doesn't matter because -K shows subcategory before any config) it shows same subcategory for manga and chapter.

And yes, I just updated from 1.27.2 to 1.273 and nothing changed.

Hrxn commented 2 months ago

Ah, yeah I see now. Should've read the entire thread, mikf mentioned it already.

Input URL is MangadexMangaExtractor but it then spawns MangadexChapterExtractor.

So you'd want to use the subcategory settings for "chapter"

Hrxn commented 2 months ago 
PS C:\Test> $URL
https://mangadex.org/title/0fa5d834-fdab-4ac4-b3c9-821f1580f991/
PS C:\Test> gallery-dl -v -o base-directory=. $URL
Debug  :  gallery-dl -> Version 1.27.3
Debug  :  gallery-dl -> Python 3.12.4 - Windows-10-10.0.19045-SP0
Debug  :  gallery-dl -> requests 2.32.3 - urllib3 2.2.2
Debug  :  gallery-dl -> Configuration Files ['%USERPROFILE%\\gallery-dl.conf']
Debug  :  gallery-dl -> Starting DownloadJob for 'https://mangadex.org/title/0fa5d834-fdab-4ac4-b3c9-821f1580f991/'
Debug  :  mangadex -> Using MangadexMangaExtractor for 'https://mangadex.org/title/0fa5d834-fdab-4ac4-b3c9-821f1580f991/'
Debug  :  urllib3.connectionpool -> Starting new HTTPS connection (1): api.mangadex.org:443
Debug  :  urllib3.connectionpool -> https://api.mangadex.org:443 "GET /manga/0fa5d834-fdab-4ac4-b3c9-821f1580f991/feed?order%5Bvolume%5D=asc&order%5Bchapter%5D=asc&contentRating%5B%5D=safe&contentRating%5B%5D=suggestive&contentRating%5B%5D=erotica&contentRating%5B%5D=pornographic&translatedLanguage%5B%5D=en&includes%5B%5D=scanlation_group&offset=0 HTTP/11" 200 None
Debug  :  urllib3.connectionpool -> https://api.mangadex.org:443 "GET /manga/0fa5d834-fdab-4ac4-b3c9-821f1580f991?includes%5B%5D=artist&includes%5B%5D=author HTTP/11" 200 1654
Debug  :  mangadex -> Using MangadexChapterExtractor for 'https://mangadex.org/chapter/2d3cdc35-488e-4e39-ac29-b5aecf17227c'
Debug  :  mangadex -> Using download archive 'C:\Apps\xmetadata\gallery-dl\archive\gallery-dl.archive.mangadex.db'
Debug  :  mangadex -> Active postprocessor modules: [ZipPP]
Debug  :  urllib3.connectionpool -> https://api.mangadex.org:443 "GET /at-home/server/2d3cdc35-488e-4e39-ac29-b5aecf17227c HTTP/11" 200 317
Debug  :  mangadex -> Sleeping 0.20 seconds (download)
Debug  :  urllib3.connectionpool -> Starting new HTTPS connection (1): cmdxd98sb0x3yprd.mangadex.network:443
Debug  :  urllib3.connectionpool -> https://cmdxd98sb0x3yprd.mangadex.network:443 "GET /data/be68711a0aeb9f80bacfaf4981c67396/1-026d402bb2a53e88b1a326bb5e27f73663cdc7244302e0b22ca5c95aa6ad5e1a.png HTTP/11" 200 1461152
.\Mangadex_TEST_LOL_TEST\Company_and_Private_Life\c001\Company_and_Private_Life_c001_001.png
Debug  :  mangadex -> Sleeping 0.20 seconds (download)
Debug  :  urllib3.connectionpool -> https://cmdxd98sb0x3yprd.mangadex.network:443 "GET /data/be68711a0aeb9f80bacfaf4981c67396/2-740f5829757d8a8f0b35ec04c33429728227c8ee3562914cd7be23ef9c8364d1.png HTTP/11" 200 1518513
.\Mangadex_TEST_LOL_TEST\Company_and_Private_Life\c001\Company_and_Private_Life_c001_002.png
Debug  :  mangadex -> Using MangadexChapterExtractor for 'https://mangadex.org/chapter/667f30dd-c099-439a-bdd2-9ba388acf1e6'
Debug  :  mangadex -> Using download archive 'C:\Apps\xmetadata\gallery-dl\archive\gallery-dl.archive.mangadex.db'
Debug  :  mangadex -> Active postprocessor modules: [ZipPP]
Debug  :  urllib3.connectionpool -> https://api.mangadex.org:443 "GET /at-home/server/667f30dd-c099-439a-bdd2-9ba388acf1e6 HTTP/11" 200 None
Debug  :  mangadex -> Sleeping 0.20 seconds (download)
Debug  :  urllib3.connectionpool -> https://cmdxd98sb0x3yprd.mangadex.network:443 "GET /data/bf1d90358d9042ac12fc13214b09a0b6/1-b53b0cc6393841be91fb198e16222dfba6ebfcf0ba6afcb3108763812b53db3a.png HTTP/11" 200 1596877
.\Mangadex_TEST_LOL_TEST\Company_and_Private_Life\c002\Company_and_Private_Life_c002_001.png
Debug  :  mangadex -> Sleeping 0.20 seconds (download)
Debug  :  urllib3.connectionpool -> https://cmdxd98sb0x3yprd.mangadex.network:443 "GET /data/bf1d90358d9042ac12fc13214b09a0b6/2-4e34868a8492b6494fe75bb3cca18556a8ed9dc38e196c5acf72dfa2c33b72d5.png HTTP/11" 200 1619783
.\Mangadex_TEST_LOL_TEST\Company_and_Private_Life\c002\Company_and_Private_Life_c002_002.png
Debug  :  mangadex -> Using MangadexChapterExtractor for 'https://mangadex.org/chapter/aaa3de94-902a-4152-8531-cbae4ed42d45'
Debug  :  mangadex -> Using download archive 'C:\Apps\xmetadata\gallery-dl\archive\gallery-dl.archive.mangadex.db'
Debug  :  mangadex -> Active postprocessor modules: [ZipPP]
Debug  :  urllib3.connectionpool -> https://api.mangadex.org:443 "GET /at-home/server/aaa3de94-902a-4152-8531-cbae4ed42d45 HTTP/11" 200 None
Debug  :  mangadex -> Sleeping 0.20 seconds (download)
Debug  :  urllib3.connectionpool -> https://cmdxd98sb0x3yprd.mangadex.network:443 "GET /data/4920412a7ea2ed7ba94734d56608414b/1-10a572aadd608b3a8ad60c0dee802e2be33385490b8d37a92409c893b6ce74b6.png HTTP/11" 200 1478760
.\Mangadex_TEST_LOL_TEST\Company_and_Private_Life\c003\Company_and_Private_Life_c003_001.png
KeyboardInterrupt
PS C:\Test>
mikf commented 2 months ago

You need to enable category-transfer for mangadex to have it behave like most other manga/chapter extractors. This option is usually enabled by default for such sites, but not for mangadex since it doesn't inherit from MangaExtractor and kind of does its own thing.

dajotim937 commented 2 months ago

Oh, I see. It's because it uses api instead of parsing, okay.

Then feel free to close if you don't need this issue for tracking/todo purposes.