search

mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.9k stars 976 forks source link

[question] How would I get cursor, I have tried verbose flag but I dont know what sting from it is cursor and none of them match length of one I got from gallery-dl in other previous operation #2516

Closed gudgoi closed 2 years ago

gudgoi commented 2 years ago

[urllib3.connectionpool][debug] https://instagram.fraj5-1.fna.fbcdn.net:443 "GET /v/t51.2885-15/278715296_486001249866909_6880197711280608909_n.jpg?stp=dst-jpg_e35_p1080x1080&_nc_ht=instagram.fraj5-1.fna.fbcdn.net&_nc_cat=109&_nc_ohc=oRHGmnih1LMAX-2ZCGZ&edm=AImE7P0BAAAA&ccb=7-4&oh=00_AT-VpdySuX7nvij9PSztOLZSsZ3jripccNpYFzvSOCL5jg&oe=626708BF&_nc_sid=688e34 HTTP/1.1" 200 73544 ./gallery-dl/instagram/sinbadsgallery/2820108622551147677.jpg

What among this is cursor? Thank you for help

rautamiekka commented 2 years ago

What do you mean by 'cursor' ?

AlttiRi commented 2 years ago

You will see the cursor on the "redirect to login page" event in the log.

Also with -v on each fetching the API response, on pagination. So, the cursor is logged at the program start and once for each 50 downloaded images. https://github.com/mikf/gallery-dl/blob/f8230dde4318be8c08cda057e1fb34b64fbe4507/gallery_dl/extractor/instagram.py#L390

Cursor: ABCD123...qweAQWE==

Look better at the log and you will find it.

Then you can use the cursor to continue downloading with the last position:

-o cursor=ABCD123...qweAQWE==
gudgoi commented 2 years ago

@AlttiRi

Ohh, I thought we get cursor for every post, I am getting list indices must be integer error (#2383 , #2435) even when I use download archive and I think it is because Instagram has to ping ig for every post and then it blocks further requests, having cursor may mitigate the issue I believe, Thank you very much.

rautamiekka commented 2 years ago

Ah, 'cursor' in this context is something I was 100% ignorant of.