mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.7k stars 953 forks source link

More instagram woes. #3011

Closed left1000 closed 2 years ago

left1000 commented 2 years ago

I got writepages of needing to have a certain age, logged in again, got

{"message":"Please wait a few minutes before you try again.","require_login":true,"status":"fail"}.

In both cases verbose wasn't as helpful as the write pages.

Waited several hours, now I'm back to

{"title":"Restricted profile","description":"You must be 18 years old or over to see this profile","user_avatar_url":"https://instagram.fagc3-1.fna.fbcdn.net/v/t51.2885-19/282245474_404562138200075_729994540798515822_n.jpg?stp=dst-jpg_s150x150&_nc_ht=instagram.fagc3-1.fna.fbcdn.net&_nc_cat=111&_nc_ohc=cK1nvLCKlOsAX_RFzRo&edm=ABfd0MgBAAAA&ccb=7-5&oh=00_AT98aGCgW9c0l8JtyYR3WzhyazUW75Jqk_m_B7Z1SNpZ4A&oe=6342B727&_nc_sid=7bff83","username":"PAGETRYINGTORIP"}

I swapped from login to cookies didn't help if anything made it worse.

got this on another nsfw page.

{"data":{"user":{"biography":"2 fat 2 furious \nbad at responding to messages blah blah blah []},"blocked_by_viewer":false,"restricted_by_viewer":null,"country_block":false,"external_url":example istgram url would go here","edge_followed_by":{"count":33080},"fbid":"17841444711142365","followed_by_viewer":false,"edge_follow":{"count":262},"follows_viewer":false,"full_name":"username","group_metadata":null,"has_ar_effects":false,"has_clips":true,"has_guides":false,"has_channel":false,"has_blocked_viewer":false,"highlight_reel_count":0,"has_requested_viewer":false,"hide_like_and_view_counts":false,"id":"44738922371","is_business_account":false,"is_eligible_to_view_account_transparency":false,"is_professional_account":false,"is_supervision_enabled":false,"is_guardian_of_viewer":false,"is_supervised_by_viewer":false,"is_supervised_user":false,"is_embeds_disabled":false,"is_joined_recently":false,"guardian_id":null,"business_address_json":null,"business_contact_method":"UNKNOWN","business_email":null,"business_phone_number":null,"business_category_name":null,"overall_category_name":null,"category_enum":null,"category_name":null,"is_private":true,"is_verified":false,"edge_mutual_followed_by":{"count":0,"edges":[]},"profile_pic_url":"blahblahblah blah 2b","requested_by_viewer":false,"should_show_category":false,"should_show_public_contacts":false,"state_controlled_media_country":null,"location_transparency_country":null,"transparency_label":null,"transparency_product":"STATE_CONTROLLED_MEDIA","username":"usernameeeeeeeeeeeeeee","connected_fb_page":null,"pronouns":[],"edge_felix_video_timeline":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]},"edge_owner_to_timeline_media":{"count":39,"page_info":{"has_next_page":false,"end_cursor":""},"edges":[]},"edge_saved_media":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]},"edge_media_collections":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]},"edge_related_profiles":{"edges":[]}}},"status":"ok"}

feels like i've got safe browser on, but when i go to my webbrowser I can view all this nsfw stuff fine it's only gallery-dl running into trouble....

no where on https://www.instagram.com/accounts/edit/ does it even seem like I can input a birth date at all.

they must've changed the nsfw code because non-nsfw pages work fine in gallery-dl and nsfw pages work fine in my webbrowser.

afterdelight commented 2 years ago

whats your browser and gallery-dl version? could you provide an example link? did you try to use --cookies option?

left1000 commented 2 years ago

gallery-dl 1.23.2 I tried both -o browser=firefox and normal, been browsing in brave. I tried using gallery-dl 1.22 but of course that didn't help. I added cookies (text file) to the gallery.conf file alongside login instead of login and went back to just login.

random example of nsfw page https://www.instagram.com/demora.avarice/

as far as I can tell from the write-pages dumps gallery-dl is being asked to confirm my age, but instagram profiles don't appear to even have ages listed on them anymore.

afterdelight commented 2 years ago

Worked for me, I can download her posts. I use --cookies option on cmd batch not on config file.

afterdelight commented 2 years ago

Could you show you command and verbose logs?

left1000 commented 2 years ago

The commands and verbose logs are the same as over here https://github.com/mikf/gallery-dl/issues/2876 or like these https://github.com/mikf/gallery-dl/issues/2900 which is why I moved on straight to write pages.

PS D:\gallery-dl> .\gallery-dl -v --cookies="D:\gallery-dl\patreon.com_cookies.txt" https://www.instagram.com/demora.avarice/
[gallery-dl][debug] Version 1.23.2 - Executable
[gallery-dl][debug] Python 3.7.9 - Windows-10-10.0.19041
[gallery-dl][debug] requests 2.28.1 - urllib3 1.26.12
[gallery-dl][debug] Starting DownloadJob for 'https://www.instagram.com/demora.avarice/'
[instagram][debug] Using InstagramUserExtractor for 'https://www.instagram.com/demora.avarice/'
[instagram][debug] Using InstagramPostsExtractor for 'https://www.instagram.com/demora.avarice/posts/'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): i.instagram.com:443
[urllib3.connectionpool][debug] https://i.instagram.com:443 "GET /api/v1/users/web_profile_info/?username=demora.avarice HTTP/1.1" 200 36
[instagram][error] An unexpected error occurred: TypeError - 'NoneType' object is not subscriptable. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[instagram][debug]
Traceback (most recent call last):
  File "gallery_dl\job.pyc", line 82, in run
  File "gallery_dl\extractor\instagram.pyc", line 61, in items
  File "gallery_dl\extractor\instagram.pyc", line 375, in posts
  File "gallery_dl\extractor\instagram.pyc", line 696, in user_id
TypeError: 'NoneType' object is not subscriptable

This stuff seemed useless which is why I went right to --write-pages but maybe I've overlooked a mistake. (using cookies via conf file or via command flag shouldn't make a difference, unless of course in one case I mess up the syntax and in the other case I don't.)

Also the write-pages feedback makes it clear that it thinks something is wrong with my account, so if another user tries it, it would work. The problem is I'd have no clue how to fix my account as age is no longer part of instagram profiles at all.

mikf commented 2 years ago

I get the same error when I'm not logged in, so there is a high possibility that your login session has expired.

$ gallery-dl -o cookies=null --write-pages https://www.instagram.com/demora.avarice/
[instagram][error] An unexpected error occurred: TypeError - 'NoneType' object is not subscriptable. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .

$ cat 01_https_i.instagram.com_api_v1_users_web_profile_info_username_demora.avarice.txt 
{"data":{"user":null},"status":"ok"}

username and password login most likely doesn't really work anymore. I'd recommend exported cookies from a private browser session with "cookies-update": false. That's what I'm currently using.

mikf commented 2 years ago

Waited several hours, now I'm back to

IG thinks you are not logged in with whatever cookies gdl is sending.

got this on another nsfw page.

  • "is_private":true
  • "followed_by_viewer":false

Probably the same issue. You cannot access a private profile without following it, I think, and invalid/expired cookies cause your follow status to not get recognized.

afterdelight commented 2 years ago

The commands and verbose logs are the same as over here #2876 or like these #2900 which is why I moved on straight to write pages.


PS D:\gallery-dl> .\gallery-dl -v --cookies="D:\gallery-dl\patreon.com_cookies.txt" https://www.instagram.com/demora.avarice/

You used patreon cookies instead of instagram cookies, that's why. ??????

I got TypeError: 'NoneType' object is not subscriptable error too when i'm not logged in

afterdelight commented 2 years ago

"cookies-update": false

what if i set cookies-update to true? will the cookies get expired?

left1000 commented 2 years ago

Wow, silly me, the files were listed next to each other and I copy pasted the wrong one. They were also next to each other sequentially in the conf file where I must've also messed it up.

Of course it just flat out doesn't work at all with the correct username password, which was what most attempts used.

mikf commented 2 years ago

username and password login most likely doesn't really work anymore

I looked into this and it was indeed completely broken. Took me a while to figure out a way to get Instagram to cooperate. Using a mobile User-Agent header worked. (https://github.com/mikf/gallery-dl/commit/827ab0a62d75f460df63aa8676545eef24e8d0f0)

afterdelight commented 2 years ago

nice job