search

ScriptSmith / instamancer

Scrape Instagram's API with Puppeteer
http://adamsm.com/instamancer
MIT License
398 stars 61 forks source link

[BUG] Locations are not supported any more #17

Closed goooseman closed 5 years ago

goooseman commented 5 years ago

Describe the bug Instagram has made /explore/locations to be available only to logged-in users.

https://www.***.com/explore/locations/212988663 - here is the route, which is used by Instamancer. If you will open this route in your browser - it will ask to log in.

To Reproduce instamancer location 212988663

OR

npm test -- -t "Library Classes"

Expected behavior Location photos to be found

Output

 ● Library Classes › location

    expect(received).toBe(expected) // Object.is equality

    Expected: 10
    Received: 0
goooseman commented 5 years ago

There is nothing we can do, unfortunately.

I can create a PR to remove this functionality and the tests, if you approve it.

ScriptSmith commented 5 years ago

Damn. Thanks for spotting this. I'll give it 24 hours, then I'll remove it myself.

ScriptSmith commented 5 years ago

The build history matches the timeline of this: https://www.businessinsider.com.au/startup-hyp3r-saving-instagram-users-stories-tracking-locations-2019-8?r=US&IR=T

karimkairo commented 5 years ago

Hi, does this mean that instamancer can no longer scrape location data or do we just need to login; i.e. I wont be able to run this via instaphyte: instaphyte location 212988663 -c 200 --filetype csv

ScriptSmith commented 5 years ago

Because instamancer and instaphyte are intended for public data scraping, they cannot support location scraping because it is only available through signed-in accounts.

I've checked and the methods used by instamancer and instaphyte would both work with signed in accounts. They're released under the MIT license so you are allowed to copy both of these projects and modify them to work with signed-in accounts, however I cannot condone or support such usage.

karimkairo commented 5 years ago

Really appreciate the fast response. I understand that both scripts use public data, is there an option (or non-developer) setting I can implement to add in the login functionality, with the basic end result I used to get a month back before the IG restrctions?

On Sun, Aug 18, 2019 at 3:36 PM Adam Smith notifications@github.com wrote:

Because instamancer and instaphyte are intended for public data scraping, they cannot support location scraping because it is only available through signed-in accounts.

I've checked and the methods used by instamancer and instaphyte would both work with signed in accounts. They're released under the MIT license so you are allowed to copy both of these projects and modify them to work with signed-in accounts, however I cannot condone or support such usage.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ScriptSmith/instamancer/issues/17?email_source=notifications&email_token=AM52A6LG6YW7Q5P5LYQWHK3QFFF5RA5CNFSM4IK3XDT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4RAGPA#issuecomment-522322748, or mute the thread https://github.com/notifications/unsubscribe-auth/AM52A6P5PR37747L5FNJ243QFFF5RANCNFSM4IK3XDTQ .

ScriptSmith commented 5 years ago

Currently there is no way of doing this without writing some code yourself. For instaphyte I think the only difference would be that you'd need to use cookies when requesting data from the API here.

Surprisingly I couldn't find other tools that use the same API for locations as instaphyte so I don't think I can point you somewhere else.

In the future I'd consider adding a plugin system to instamancer allowing developers to listen for events that the scraper dispatches and implement extra functionality. However I think I'd need some other use-cases to consider adding that

ScriptSmith commented 5 years ago

Actually, instaloader does support signed-in location scraping. I'm not sure if it uses the same method that instaphyte did, but it should work.

karimkairo commented 5 years ago

Thanks Adam Cant personally confirm instaloader. But managed to get signed in location scraping from instagram-scraper: instagram-scraper -u USERNAME -p PASSWORD --location LOCATION_ID -m MAX_COUNT -t none --media-metadata Ill revert back if I give instaloader a shot, and thanks again Adam

On Sun, Aug 18, 2019 at 4:02 PM Adam Smith notifications@github.com wrote:

Actually, instaloader does support signed-in location scraping. I'm not sure if it uses the same method that instaphyte did, but it should work.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ScriptSmith/instamancer/issues/17?email_source=notifications&email_token=AM52A6MKY5LPSUWZBYQVULTQFFI6HA5CNFSM4IK3XDT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4RAV3A#issuecomment-522324716, or mute the thread https://github.com/notifications/unsubscribe-auth/AM52A6NRU7VLYQZZRCG5SRLQFFI6HANCNFSM4IK3XDTQ .

ScriptSmith commented 5 years ago

No worries, happy to help