jsanahuja / jquery.instagramFeed

Instagram Feed without access token. Not using the Instagram API
MIT License
313 stars 164 forks source link

Stopped working #103

Closed luizping closed 3 years ago

luizping commented 3 years ago

I think IG changed the policy. Now to see a user page, you have to be logged in. So when you run the script, the GET returns the IG Login Page content.

web-burn commented 3 years ago

Currently the fix with proxy host: "https://instagram-feed-proxy.herokuapp.com/?url=https://www.instagram.com/'" doesn't work.

Unfortunately, that is to be expected. IG is hard to cheat. But caching still does the trick. It is not perfect but quite acceptable.

@m0nclous, do you have any new findings?

If i will use a value "1440" for the "cache_time" then single IP will not be banned? If i understood right then this option allows to decrease a quantity of requests from each IP. Is this right?

mahotilo commented 3 years ago

As I understand it. IG wants you to subscribe and log in. So, it limits the number of interactions of an unregistered user with images and profiles (usernames). It doesn't matter if it is several images in one profile or just one image in several profiles. The script acts as unregistered user, so your IP is the subject of ban. This is not the IP address of the server hosting the page with the script. You should notice that other users are still able to see script output despite you are banned.

Thus, if you use a big cache time, you protect yourself from being banned when interacting with the same profile in the same page. But, if you run multiple scrips with different profiles (username) within a short period of time, you will still be banned, no mater how big the cache time is. Or not :) The question is how many IG data updates (get()'s) will you make.

m0nclous commented 3 years ago

@mahotilo, any news about fix? Currently the fix with proxy host: "https://instagram-feed-proxy.herokuapp.com/?url=https://www.instagram.com/'" doesn't work.

image Yeah, they still blocked the session after a while

m0nclous commented 3 years ago

@m0nclous, do you have any new findings?

I implemented (locally) what I said above, but I got the same problem - Instagram blocks requests to

https://www.instagram.com/web/search/topsearch/?query=<username>

after about 6 requests

BUT! Instagram doesn't block it

https://instagram.com/graphql/query/?query_id=17888483320059182&variables={"id":"<user_id>","first":20,"after":null}

Thus, we can avoid errors if we set the target profile using ID, but to completely solve the issue, we need to find a way to get ID based on username without blocking.

m0nclous commented 3 years ago

We can receive data through https://www.instagram.com/<username>/?__a=1 but this path is blocked after about 4 requests

mahotilo commented 3 years ago

but this path is blocked after about 4 requests

At least we can create an instruction on how to get this ID manually.

m0nclous commented 3 years ago

@mahotilo https://instagram.com/graphql/query/?query_id=17888483320059182&variables={"id":"<user_id>","first":20,"after":null} Doesn't return profile info, such as profile_pic_url/username/biography

What to do with it?

mahotilo commented 3 years ago

@m0nclous Hmm. Very intriguing. While I was looking for an answer to your question I found this article

https://medium.com/@carloshenriquereis_17318/how-to-get-data-from-a-public-instagram-profile-edc6704c9b45

  1. author describes the same problem and the same approach to solving
  2. he gets missed user data from request https://www.instagram.com/username/?__a=1
  3. he gets more than 12 images with offset in request :)

His demo is at the end of https://carlos-henreis.github.io/

m0nclous commented 3 years ago

@mahotilo Yes, I read that too. The request is blocking just like the others We can cache this information for a long time, but this is not a solution to the problem, since caching is not a solution for many users, if you look at this issue

mahotilo commented 3 years ago

I believe that hiding user account data is a part of IG's policy. So there is little chance of finding a long-lasting trick.

But for many users, this script is just a handy way to get an easy-to-update picture gallery on the site. I have even hidden a user info in some projects. So this might be an option:

Personally, I think caching is a great solution. The only real alternative is to use IG API.

web-burn commented 3 years ago

Hello,

The plugin stopped work.

Screenshot_1
jsanahuja commented 3 years ago

It looks like they have added a CORS restriction again but only in some of their servers. That is why some requests are success and some are CORS blocked. We will test and implement the graphql solution soon

web-burn commented 3 years ago

@jsanahuja many thanks! Can you say as soon new solution will be ready? (approximate time of course). Sorry for the indiscreet question.

jsanahuja commented 3 years ago

Unfortunately GraphQL API is also affected by the arbitrary CORS issues.

Release 2.1.1 implementing GraphQL (with user_id parameter) is ready. We have also implemented request retries (limited by max_tries parameter) to avoid arbitrary CORS errors.

web-burn commented 3 years ago

@jsanahuja It is should updated plugin version to 2.1.1 on my site? Is this fix the CORS issue?

jsanahuja commented 3 years ago

@jsanahuja It is should updated plugin version to 2.1.1 on my site? Is this fix the CORS issue?

CORS issue is not something we can fix client-side. As it is arbitrary (only some of their cluster servers have a CORS restriction) we have implemented a policy of retries. You should definitely update to the 2.1.1 but the real solution would be using a proxy and the graphQL (user_id parameter)

colinchadwick commented 3 years ago

It isn’t working anymore.

romanklabal commented 3 years ago

So since the idiots at IG introduced CORS, there is simply no way to reliably load the last 12 IG posts based on the username any longer?

The only solution is to use graphQL with user_id, which we.. can not get programatically and have to use manual workarounds to get it first?

mahotilo commented 3 years ago

They want to know personally everyone who opens their site :)

colinchadwick commented 3 years ago

Actually, I've just found a (very tricky, but working) way to bypass the IG CORS policy – use this as the host: 'host': "https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=https://www.instagram.com/"

piotrmerton commented 3 years ago

Actually, I've just found a (very tricky, but working) way to bypass the IG CORS policy – use this as the host: 'host': "https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=https://www.instagram.com/"

Thanks, it works!

iennetastic commented 3 years ago

Actually, I've just found a (very tricky, but working) way to bypass the IG CORS policy – use this as the host: 'host': "https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=https://www.instagram.com/"

This works for me too! Thank you! <3

mahotilo commented 3 years ago

'host': "https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=https://www.instagram.com/"

Indeed, magic works. But, if I try to open the generated URL (host + username) in the browser, I get the message We're sorry... ... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.

So the question is, should I just not open this URL directly in the browser, or will we eventually get banned by Google this time?

colinchadwick commented 3 years ago

I would say it’s unlikely to get banned by Google, I‘ve seen other use this at a fairly high scale. I just hope that this doesn’t get banned by IG.

introzen commented 3 years ago

Actually, I've just found a (very tricky, but working) way to bypass the IG CORS policy – use this as the host: 'host': "https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=https://www.instagram.com/"

Works like a charm! Cheers!

m0nclous commented 3 years ago

@mahotilo

But, if I try to open the generated URL (host + username) in the browser, I get the message We're sorry... ... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.

Google checks the referer header

image

image

mahotilo commented 3 years ago

@m0nclous So as of now, this is kind of a permanent solution! Can we call it the Google proxy? :)

jimbokes commented 3 years ago

The proxy works thanks but I’m finding extra margin now between image rows - can I fix this?

m0nclous commented 3 years ago

Actually, I've just found a (very tricky, but working) way to bypass the IG CORS policy – use this as the host: 'host': "https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=https://www.instagram.com/"

No longer appears to be working.

image

@introzen This works if you make an ajax request or add a referer header manually https://github.com/jsanahuja/jquery.instagramFeed/issues/103#issuecomment-788854136

Creative-developers commented 3 years ago

https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=

This is not working for following link https://www.instagram.com/graphql/query/?query_hash=" + queryHash + "&variables=" + variables;

jsanahuja commented 3 years ago

Upgrade to 3.0.0 and use this https://github.com/jsanahuja/jquery.instagramFeed/issues/103#issuecomment-788260968


https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=

This is not working for following link https://www.instagram.com/graphql/query/?query_hash=" + queryHash + "&variables=" + variables;

Right. We are not able to use GraphQL anymore unless we use a real proxy.

Mikhail-MM commented 3 years ago

@jsanahuja @introzen

This looks almost like we are spoofing/tricking IG by manufacturing the header - I would not be surprised if the solution goes down again, no?

web-burn commented 3 years ago

@jsanahuja i noticed this error now. The instagram feed doesn't displayed. Browser firefox. I use the latest version 3.0.0.

Screenshot_1
mahotilo commented 3 years ago

@web-burn Is not it related to https://github.com/jsanahuja/jquery.instagramFeed/issues/117?

web-burn commented 3 years ago

@mahotilo Sometimes i see the error on real hosting too. Also i noticed the next error when i change username in plugin settings:

Screenshot_1 Screenshot_2
mahotilo commented 3 years ago

Sometimes i see the error on real hosting too.

I do not have it here. Could you describe a condition to reproduce?

Also i noticed the next error when i change username in plugin settings:

It can be if parse_caption returns false. It should not be an isolated error, just a consequence of an early problem.

web-burn commented 3 years ago

@mahotilo I noticed this error only for some instagram accounts. Please try the next user name: modern_presta

Update: The error occur when instagram account doesn't have bio.

m0nclous commented 3 years ago

@jsanahuja i noticed this error now. The instagram feed doesn't displayed. Browser firefox. I use the latest version 3.0.0.

Screenshot_1

This happens when Google responds with an error, because in this case it does not send the header "Access-Control-Allow-Origin" To understand the reason - you need to reproduce the problem and see the answer from Google manually.

I think this solution is still crude and more logging needs to be added.

image

image

mahotilo commented 3 years ago

@mahotilo I noticed this error only for some instagram accounts. Please try the next user name: modern_presta

Update: The error occur when instagram account doesn't have bio.

I have checked, there is the same error here. But, it has happened to me before too. It is related to the first VIDEO in the row.

m0nclous commented 3 years ago

@mahotilo I noticed this error only for some instagram accounts. Please try the next user name: modern_presta Update: The error occur when instagram account doesn't have bio.

I have checked, there is the same error here. But, it has happened to me before too. It is related to the first VIDEO in the row.

This happens when parse_caption() dont finds a caption and returns false in escape_string where the string is expected. I will fix it.

image

image

mahotilo commented 3 years ago

We should definitely have created a separate new issue for this :)

m0nclous commented 3 years ago

We should definitely have created a separate new issue for this :)

Can you do it please?

mahotilo commented 3 years ago

Please, let us continue to discuss this error here https://github.com/jsanahuja/jquery.instagramFeed/issues/120

jsanahuja commented 3 years ago

@jsanahuja i noticed this error now. The instagram feed doesn't displayed. Browser firefox. I use the latest version 3.0.0.

Screenshot_1

This happens when Google responds with an error, because in this case it does not send the header "Access-Control-Allow-Origin" To understand the reason - you need to reproduce the problem and see the answer from Google manually.

I think this solution is still crude and more logging needs to be added.

image

image

@m0nclous this is because you are running the script without a web server. The solution is setting the 'Referer' header but web browsers won't allow us to do that with Javascript for security reasons

thomashdk commented 3 years ago

ALso gettings this. Using 3.0.0 whit the "fix" host

image

mahotilo commented 3 years ago

@thomashdk, I can only quote for you

this is because you are running the script without a web server

thomashdk commented 3 years ago

On my site I get this ?

image

mahotilo commented 3 years ago

@thomashdk, can you see the script output for the demo page https://www.sowecms.com/demos/jquery.instagramFeed/index.html?

thomashdk commented 3 years ago

image hmm no ? @mahotilo

@thomashdk, can you see the script output for the demo page https://www.sowecms.com/demos/jquery.instagramFeed/index.html?

mahotilo commented 3 years ago

@thomashdk Hmm, it works for me here image