voussoir / timesearch

The subreddit archiver
BSD 3-Clause "New" or "Revised" License
172 stars 7 forks source link

received 400 HTTP response #11

Closed kidzrockboom closed 3 years ago

kidzrockboom commented 3 years ago

Sorry if this isn't detailed enough, I'm new to writing Issues but when I try to run the code I get this error and I have tried reinstalling and checked the bot.py info and checked google and tried to read the code and still cant figure out what is wrong can someone please help. (venv) C:\Users\richa\PycharmProjects\timesearch>python timesearch.py get_submissions -r aww Thank you Jason Baumgartner of Pushshift.io! Traceback (most recent call last): File "timesearch.py", line 424, in raise SystemExit(main(sys.argv[1:])) File "timesearch.py", line 411, in main return betterhelp.subparser_main( File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\voussoirkit\betterhelp.py", line 204, in subparser_main return subparser_betterhelp(parser, main_docstring, sub_docstrings)(main)(argv) File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\voussoirkit\betterhelp.py", line 184, in wrapped return main(argv) File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\voussoirkit\betterhelp.py", line 203, in main return args.func(args) File "timesearch.py", line 334, in get_submissions_gateway get_submissions.get_submissions_argparse(args) File "C:\Users\richa\PycharmProjects\timesearch\timesearch_modules\get_submissions.py", line 99, in get_submissions_argparse return get_submissions( File "C:\Users\richa\PycharmProjects\timesearch\timesearch_modules\get_submissions.py", line 73, in get_submissions for chunk in submissions: File "C:\Users\richa\PycharmProjects\timesearch\timesearch_modules\common.py", line 78, in generator_chunker for item in generator: File "C:\Users\richa\PycharmProjects\timesearch\timesearch_modules\pushshift.py", line 243, in supplement_reddit_data live_copies = list(common.r.info(ids)) File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\praw\reddit.py", line 631, in generator for result in self.get(API_PATH["info"], params=params): File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\praw\reddit.py", line 566, in get return self._objectify_request(method="GET", params=params, path=path) File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\praw\reddit.py", line 667, in _objectify_request self.request( File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\praw\reddit.py", line 849, in request return self._core.request( File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\sessions.py", line 328, in request return self._request_with_retries( File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\sessions.py", line 226, in _request_with_retries response, saved_exception = self._make_request( File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\sessions.py", line 183, in _make_request response = self._rate_limiter.call( File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\rate_limit.py", line 33, in call kwargs["headers"] = set_header_callback() File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\sessions.py", line 281, in _set_header_callback self._authorizer.refresh() File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\auth.py", line 254, in refresh self._request_token( File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\auth.py", line 155, in _request_token response = self._authenticator._post(url, **data) File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\auth.py", line 38, in _post raise ResponseException(response) prawcore.exceptions.ResponseException: received 400 HTTP response

voussoir commented 3 years ago

Hi,

I know that debugging oauth can be a pain because the 400s can be mysterious.

Here's where I get the variables to begin with:

image

Now, in this script it tells you to do some export praw_client_id=..., but I actually don't like dealing with environment variables, so an alternative is to just edit the script itself, and add client_id and client_secret arguments to the praw.Reddit:

image

When I run that script I get this:

image

It works:

image

I hope that helps a bit.

kidzrockboom commented 3 years ago

Thank you for responding. This was very helpful I was able to see the point of the error. It happened when I tried to run r.user.me()

import bot r = bot.login() r.user.me()

Traceback (most recent call last): File "", line 1, in File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\praw\models\user.py", line 155, in me user_data = self._reddit.get(API_PATH["me"]) File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\praw\reddit.py", line 566, in get return self._objectify_request(method="GET", params=params, path=path) File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\praw\reddit.py", line 667, in _objectify_request self.request( File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\praw\reddit.py", line 849, in request return self._core.request( File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\sessions.py", line 328, in request return self._request_with_retries( File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\sessions.py", line 226, in _request_with_retries response, saved_exception = self._make_request( File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\sessions.py", line 183, in _make_request response = self._rate_limiter.call( File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\rate_limit.py", line 33, in call kwargs["headers"] = set_header_callback() File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\sessions.py", line 281, in _set_header_callback self._authorizer.refresh() File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\auth.py", line 254, in refresh self._request_token( File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\auth.py", line 155, in _request_token response = self._authenticator._post(url, **data) File "C:\Users\richa\PycharmProjects\timesearch\venv\lib\site-packages\prawcore\auth.py", line 38, in _post raise ResponseException(response) prawcore.exceptions.ResponseException: received 400 HTTP response

voussoir commented 3 years ago

Yes, the error occurs there because that's the moment where the r object needs to actually log in to reddit in order to figure out who it is. Something's wrong with your refresh token so the authorization failed at that moment.

It is difficult for me to know exactly which part of your bot.py file is messed up, but I have demonstrated that the process works. So I would suggest deleting the bot.py that you have and starting from scratch. Here are some additional thoughts:

kidzrockboom commented 3 years ago

Sorry, this is probably a dumb question but where am I copying and pasting the refresh token to?

voussoir commented 3 years ago

Into bot.py, APP_REFRESH

image

By the way, CONTACT_INFO is used when downloading from pushshift because stuck_in_the_matrix requests it. You should put an email or something there. The USERAGENT could be "/u/kidzrockboom using timesearch".

kidzrockboom commented 3 years ago

I tried everything you suggested and then some, would it be possible to mail you my bot.py file so you can test it and see what the issue is?. I am running Praw 7.3.0 and I don't think that is the issue either, I have been on this all day yesterday and today and I'm stumped brother.

voussoir commented 3 years ago

Sure, email to ethan@voussoir.net

kidzrockboom commented 3 years ago

YAY IT worked with your method. I just followed the video. Thank you so much Man I appreciate it so much

voussoir commented 3 years ago

Great, I'm glad it worked 👍