Closed code4hub closed 4 years ago
Obviously Instagram doesn't want any machine to scrape their content.
I've experienced the same error and investigated a bit. They redirect with 302 to a login page, instead of displaying the public content.
Here are some observations I've made :
Somehow Instagram is flagging each query (based on IP address, cookies or such) as being machine run or not.
I don't really see a solution for that issue.
Thanks a lot @lucianosantana for your comment, I'm thinking maybe adding a real-looking User-Agent could do the job, but that's usually not that simple. Looking at this picture, we can see that the request returned a 200 even without the cookie. Removing the User-Agent does nothing tho, but it does not cost anything to add it, I'll do that asap.
@EdouardCourty thanks for your reply. I believe that won't be enough. As I've mentioned in my previous message, I experience the issue also when accessing the profile URL using an incognito browser window, which sends a real browser user agent.
Probably Instagram uses multiple flags to identify a robot:
I actually just tested modifying the library myself and running the script tunneling it through my server's IP and I get the same result.
In any case, I'd be glad to help with testing if you need a hand.
I pushed an updated version, feel free to update user-instagram to its latest version with npm !
Also, I'm not experiencing this issue at all, tested on friends computers, at work too, and I couldn't recreate your case, what is the username you're trying to fetch ?
All my tests sent me valid data
Hi @EdouardCourty
Thank you for your patience.
Here are a couple of tests I've made.
1 - Requests tunneling connection via server A : https://monosnap.com/file/0dvgpbVwKFxEJNNKUMC5N0VmYBhFAH
2 - No connection tunneling: https://monosnap.com/file/P4kOm02pjASha50XNKkelUwIdcVern
I've also made another test with a server B and the results are the same as server A. Both servers are in Digital Ocean. My guess is that Instagram somehow manage to identify that those machines are servers (therefore not final Instagram users) and try to force a login.
Maybe you could adjust the library so it has the option to authenticate before requesting Instagram's feed.
Regarding this :
I'm not experiencing this issue at all, tested on friends computers, at work too, and I couldn't recreate your case Were any of this computers a server ?
I'm also curious to know more details about the context of the issue appearance for @code4hub, since he/she was the one opening the issue.
Hi, I just started using this module and I don't get this error when I'm using it on an ip like my home or my phone, but when I use it on my vps, I get it, so I tried using the same headers as my browser do (yes, even the cookies) and it works fine yet on my computer etc but not my vps so I'm thinking that instagram are blacklisting ip adresses, but I'm wondering what if you actually login? would it allow you to use the api ? (I have not tried yet, it seems just oh so complicated that I don't know where to start)
instagram are blacklisting ip adresses That's what I believe too. If you login, then it works fine regardless of the IP address.
so (I don't know how to, or if it is possible) it would mean that in case "that error" happens, in the best case, there's an arg in the function like username and password, and if then it would login ? I don't know if it's possible or not in javascript ... or else a code that would allow to login but yet, I don't know if it's even possible .... Oh and also, I'm wondering what ip adress is it using to "blacklist" cuz' what if they're only blacklisting ipv4 ? then ipv6 would maybe mean that it's ok ?
A simple explanation would be that VPS providers have a certain IP range they use, and these IPs may be banned.
I also think of it but how can we go around, using a proxy in the code to show in another country ?
You could use a proxy sure, but the module does not provide any method of implementing that yet, I could add this feature soon tho, it's quite simple.
i'd like if you can, and if possible add a way to use the user and password for proxy (like the one in squid)
Getting "UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'user' of undefined" with below code