search

ping / instagram_private_api

A Python library to access Instagram's private API.
MIT License
2.96k stars 613 forks source link

Fetch comments within an interval with no rate limit #277

Closed luizamazo closed 4 years ago

luizamazo commented 4 years ago

Please follow the guide below

Before submitting an issue, make sure you have:

Which client are you using?

Describe your Question/Issue:

Please make sure the description is worded well enough to be understood with as much context and examples as possible.

I'm building a bot that is going to scrape a few things from a specific IG account. I'm writing in javascript and I'm using a childprocess to access this lib as well as others in python. My problem is: I need to request for new info within an interval (I put 35s) and it works fine with stories/other stuff (from other libs) but I'm using this one to request for comments and this same interval hits rate_limit_error after some minutes. I'm logged in with cookies. I tried to fix this by calling for the function in alternative times (35s yes, next 35s no, and it goes like this) but somehow, it still gives rate limit after 15 minutes, and I need to do this with no limit, basically 24/7. What I also tried: to alternate between two accounts, making requests randomly through them (didn't work, one started to require challenge everytime). I also tried to give a random time of sleep each time the condition was true and it was time for the api to call for comments but still no success.

I also want to know if I'm doing the logging in thing the right away. Even though I stored in cookies, should I still use Client(user,pass)? I'm new in python, I don't know if I should load the session from a file or something.

I would appreciate some help, I really need it. Thanks in advance.

Paste the output of python -V here: Python 3.8.1 Code:

# Example code that will produce the error reported

from instagram_private_api import Client, ClientCompatPatch 
MEDIA_ID = xxxx
api = Client(user_name, password, auto_patch=True)

def commentsMaster():
    comments = api.media_n_comments(MEDIA_ID, n=50)
    file_path = './instagram/comments/comments [' + MEDIA_ID + '].json'
    write_json(file_path, comments)
    searchAndWriteRelevantComments(MEDIA_ID)
    [...]

Error/Debug Log:

"C:\Users\luiza\AppData\Local\Programs\Python\Python38\lib\urllib\http://request.py", line 649, in http_error_default  
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "http://comments.py", line 27, in <module>
    api = Client(user_name, password, auto_patch=True)
  File "C:\Users\luiza\AppData\Local\Programs\Python\Python38\lib\site-packages\instagram_private_api\http://client.py", line 
208, in __init__
    self.login()
  File "C:\Users\luiza\AppData\Local\Programs\Python\Python38\lib\site-packages\instagram_private_api\endpoints\http://accounts.py", line 49, in login
    login_response = self._call_api(
  File "C:\Users\luiza\AppData\Local\Programs\Python\Python38\lib\site-packages\instagram_private_api\http://client.py", line 
527, in _call_api
    ErrorHandler.process(e, error_response)
  File "C:\Users\luiza\AppData\Local\Programs\Python\Python38\lib\site-packages\instagram_private_api\http://errors.py", line 
135, in process
    raise ClientError(error_msg, http_error.code, error_response)
instagram_private_api.errors.ClientError: Bad Request: rate_limit_error
dvingerh commented 4 years ago

Saving and reusing a session with a cookie can be done via the example code provided https://github.com/ping/instagram_private_api/blob/master/examples/savesettings_logincallback.py

As for the rate limit, there's not much you can do to avoid it when you aren't mimicking standard user behavior for an extended period of time.

The general rule of thumb is, the greater you deviate from a "regular" user's behavior, the greater the risk of getting into trouble with IG. Use the longest interval that is sufficient for your needs and not the shortest interval you think you can get away with. Be reasonable. Don't be spammy/abusive.

luizamazo commented 4 years ago

I see. So I should try to put a longer interval so it wont get rate limits right, I guess that's the only way.

As for the logging in thing, what I did was: I ran a file named login.py that had the content from the link you suggested me, with the command thats written there, and I received "all ok" and the time the cookie was going to expire and all. Would I need to load the session on my "comments.py" code, like instaloader, as an example? Or should I keep using Client(user,pass), I'm a bit confused if using Client(user, pass) I would be logging in everytime and not with the cookie created. Sorry if the question is dumb, I'm a beginner in this. Thank you in advance.

dvingerh commented 4 years ago

See L64-82 https://github.com/ping/instagram_private_api/blob/master/examples/savesettings_logincallback.py#L64-L82 To reuse the cookie file you need to pass its path to the script with --settings

e.g. python3 savesettings_logincallback.py --username johndoe --password grapefruits --settings cookie.json You'll still pass username and password to the script so that a new cookie file can be generated in case the existing one has expired.

Or should I keep using Client(user,pass)

Yes, just implement and refactor the example code for your project instead of writing your own methods to handle authentication

luizamazo commented 4 years ago

Thank you for your help with the login, I finally understood how to do it :) Also, for the comments, I managed to get them without hitting rate limits by putting a larger interval, even though I'm having some trouble with fetching comments when its IGTV post. Some IGTV post work, some doesn't, it says media is unavailable, I'll investigate better. Thank you again for the help!