mrafee113
commented
2 years ago Hi again.
I have narrowed the problem even more than before.So, here's the update.
I tried reproducing the manual request again like this:
import requests
session = requests.Session()
session.verify = False
session.headers.update({
'Connection': 'Keep-Alive',
'Accept': '*/*',
'Accept-Encoding': 'gzip,deflate',
'Accept-Language': 'en-US',
'User-Agent': 'Instagram 245.0.0.18.108 Android (30/11.0.0; 440dpi; 1080x2400; Xiaomi; Redmi Note 9 Pro; joyeuse; qcom; en_US; 314665256)',
'Authorization': client.authorization
})
r = session.get("https://www.instagram.com/explore/tags/art/", params={'__a': 1, '__d': 'dis'})
and when returned, r.url did not have '/login/' in it. and r.json() did not raise any exceptions.
But when I replaced user agent with PublicRequestMixin's session.headers.update's user agent, the response behaved like I explained in my original bug report.
So I have this guess that apparently the request is using the original header set within PublicRequestMixin.init and not the one provided through settings. If that is the case, (i couldn't debug the original request sent by instagrapi) then it can be concluded that instagram public api will behave according to the user agent; if it thinks you're a phone user, it will send you json data, else otherwise.
But that is not all. When I looked into the fixed request, there was a lot of data. I didn't know where to look, so I traced the code. It seems like even if the user-agent problem is fixed, still the code will raise an exception. And that is KeyError: graphql which will be raised in PublicRequestMixin.public_a1_request.
Now, here's a wild insight. Despite me not knowing what processes actually went into designing this method, I have a guess. You see, I did a little research on instagram related hashtags, and this article had section called Use Instagram’s Related Hashtags feature. In there I saw a picture that showed that whilst seeing a hashtags top medias, there also are related hashtags presented to the app user. Take note that this article was from July, 19, 2022. But when I looked at my own phone, I did not find anything related. So my wild guess is that maybe instagram has removed this feature. Or... My user agent (meaning my phone model) is being discriminated against by instagram (haha). IDK.
Describe the bug A clear and concise description of what the bug is.
After the client is ready to use (loginned with user/pass and/or cookies with settings properly configured), calling client.hashtag_related_hashtags("art") raises ClientLoginRequired due to JSONDecodeError. I have tracked down the code, and figured that the exception is being raised in PublicRequestMixin._send_public_request; the exception is being called by the requests module when the code tries to run response.json() in line 132.
To Reproduce Provide a piece of code to reproduce the problem. cl = Client() cl.load_settings(path) cl.login(user=user, pass=pass)
art_related_hashtags = cl.hashtag_related_hashtags("art")
Traceback Show your full traceback so that it is clear where exactly the error occurred.
Traceback (most recent call last): File ".../python3.10/site-packages/requests/models.py", line 971, in json return complexjson.loads(self.text, kwargs) File ".../python3.10/json/init.py", line 346, in loads return _default_decoder.decode(s) File ".../python3.10/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File ".../python3.10/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File ".../python3.10/site-packages/instagrapi/mixins/public.py", line 132, in _send_public_request self.last_public_json = response.json() File ".../python3.10/site-packages/requests/models.py", line 975, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File ".../python3.10/site-packages/IPython/core/interactiveshell.py", line 3398, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "", line 1, in <cell line: 1>
art_related_hashtags = cl.hashtag_related_hashtags('art')
File ".../python3.10/site-packages/instagrapi/mixins/hashtag.py", line 129, in hashtag_related_hashtags
data = self.public_a1_request(f"/explore/tags/{name}/")
File ".../python3.10/site-packages/instagrapi/mixins/public.py", line 173, in public_a1_request
response = self.public_request(
File ".../python3.10/site-packages/instagrapi/mixins/public.py", line 76, in public_request
raise e # Stop retries
File ".../python3.10/site-packages/instagrapi/mixins/public.py", line 74, in public_request
return self._send_public_request(url, kwargs)
File ".../python3.10/site-packages/instagrapi/mixins/public.py", line 138, in _send_public_request
raise ClientLoginRequired(e, response=response)
instagrapi.exceptions.ClientLoginRequired: Expecting value: line 1 column 1 (char 0)
Expected behavior A clear and concise description of what you expected to happen.
The only expectation I had was that some data must've been returned. But instead I got an exception. Honsetly I have no idea how this data should even be structured. I tried reproducing it manually (because of public api), and got a 200 response, but it does not contain the expected json format.
code reproduced
r = requests.get('https://www.instagram.com/explore/tags/art/', params={'__a': 1, '__d': 'dis'}')The output of r.text was a full html containing css and js of page https://www.instagram.com/explore/tags/art/.Screenshots If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Additional context Add any other context about the problem here. I tried to track down the url path of "https://www.instagram.com/explore/tags/{name}/", but I reached a dead-end since there was no trace of such a thing neither in the page itself, nor Instagram Basic Display Api, and also not in Instagram GraphQL Api. I'm actually not sure where you got these paths from, except by sniffing the app itself.