westonplatter
commented
5 years ago @wccramer I see you closed this issue. Did it work for you?
Closed viaConBodhi closed 5 years ago
westonplatter
commented
5 years ago @wccramer I see you closed this issue. Did it work for you?
viaConBodhi
commented
5 years ago @westonplatter No...I messed up my prior post and started a new. Found the following thread somewhere else on this which means your code most likely is affected for others (like me). Looks like the recent RH update has affected others too but not all. Nice work on the code base though...been very helpful up till when login stopped working. Let me know if you find a solution as I'm keeping an eye out for those with more skills to help provide some options...no pun intended.
westonplatter
commented
5 years ago @wccramer thanks for the context. Won't have time to look at it for a while. Happy to look at a PR.
viaConBodhi
commented
5 years ago @westonplatter If/when I get can get something together I'll send your way.
halessi
commented
5 years ago Also having this error as of a few hours ago, everything was working well last night. (~ 12 hours ago)
Chenzoh12
commented
5 years ago Same as @halessi. Worked perfectly yesterday and last night and then as of this morning I can not access anything.
halessi
commented
5 years ago @wccramer Any chance you could reopen this? Or keeping consolidated to #86 ?
viaConBodhi
commented
5 years ago Checkout the link I posted above for the thread from Jamonek. There are some folks who have a solution but the code is mostly in C#/Java but there is some python. No fully updated code base but looks like you should be able to build out a solution with what has been provided via the discussions. RH has added SMS/email verification so login has changed. Looks like they are rolling it out in phases though.
Chenzoh12
commented
5 years ago Agh yes, I saw that thread as well. I could not figure out my next steps exactly once I received the the SMS verification code. If anybody has any idea how to clear that hurdle please do let me know. I will keep my eyes on that thread as well.
On Wed, May 1, 2019 at 12:42 PM wccramer notifications@github.com wrote:
Checkout the link I posted above for the thread from Jamonek/Robinhood#176 https://github.com/Jamonek/Robinhood/issues/176. There are some folks who have a solution but the code is mostly in C#/Java but there is some python. No fully updated code base but looks like you should be able to build out a solution with what has been provided via the discussions. RH has added SMS/email verification so login has changed. Looks like they are rolling it out in phases though.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/westonplatter/fast_arrow/issues/85#issuecomment-488337056, or mute the thread https://github.com/notifications/unsubscribe-auth/AG3DHZIE6SOE57A6JR2XKT3PTHB6ZANCNFSM4HJE7IYQ .
viaConBodhi
commented
5 years ago @Chenzoh12 Haven't tested but it looks like you can opt to have an email sent then, if using Google API, use some reg X and email filtering to capture the # and add to the variable. A few steps but seems like it should work.
halessi
commented
5 years ago @Chenzoh12
Utilizing what they did over at the Robinhood repo linked earlier, I replaced the first few lines of login_oauth2 with:
`if self.device_token == None:
self.device_token = self.GenerateDeviceToken()
data = {
"grant_type": "password",
"scope": "internal",
"client_id": CLIENT_ID,
"expires_in": 86400,
"password": password,
"username": username,
"challenge_type": 'sms',
"device_token": self.device_token
}`with GenerateDeviceToken (also written in the other thread) being:
`def GenerateDeviceToken():
rands = []
for i in range(0,16):
r = random.random()
rand = 4294967296.0 * r
rands.append((int(rand) >> ((3 & i) << 3)) & 255)
hexa = []
for i in range(0,256):
hexa.append(str(hex(i+256)).lstrip("0x").rstrip("L")[1:])
id = ""
for i in range(0,16):
id += hexa[rands[i]]
if (i == 3) or (i == 5) or (i == 7) or (i == 9):
id += "-"
return id`And I get an SMS text. Going to swap to email and see if I can somehow do what @wccramer suggested.
viaConBodhi
commented
5 years ago @halessi thanks for that! Let me know if it works out...caught up on day job stuff so I can't play with it now.
halessi
commented
5 years ago Alright, I've managed to get it working off of what was said in the other thread. The initial post for authenticating returns a "challenge", of which it's necessary to save the challenge id. If you have that, you can use the verification code retrieved via the below function (note, user & pass are gmail login info) if you submitted the original post with {'challenge_type': 'email'}:
`def fetch_mfa_code(self):
username, password = user, pass
obj = imaplib.IMAP4_SSL('imap.gmail.com')
obj.login(username, password)
obj.select()
cutoff = (datetime.today() - timedelta(minutes = 2)).strftime('%d-%b-%Y')
typ, data = obj.search(None,
'(SINCE %s) (FROM "notifications@robinhood.com")' % (cutoff,))
email_ids = data[0].decode().split(' ')
email = obj.fetch(email_ids[-1], '(UID BODY[TEXT])')
parsed = email[1][0][1].decode().split('r')[9]
mfa_code = [int(s) for s in re.findall(pattern = r'\d+', string = parsed)]
assert len(str(mfa_code[0])) == 6, print('ERROR: failed to parse mfa code.')
return mfa_code[0]`Use this verification code alongside the saved challenge id to submit another post:
`if mfa_code is not None and id is not None:
url = 'https://api.robinhood.com/challenge/{}/respond/'.format(id)
payload = {'response' : mfa_code}
res = self.post(url, payload = payload, retry = False)`Res['status'] should be validated, and you're good! I can make a PR if @westonplatter wants, but I'm certain my solution isn't exactly the best approach.
viaConBodhi
commented
5 years ago Proper! good work @halessi ! Looks like you got the regX figured. Can't wait to try this. Not sure if there are too many other approaches around this SMS/email thing so looks like you got a working model without having to manually enter numbers from a SMS.
viaConBodhi
commented
5 years ago Ran into issues with gmail running the code above but looks like should work. When/If you all get a google API email setup (not hard and plenty of online support) the code below should help you parse the text from RH. The code still needs to filter based on datetime (creating an input by capturing the datetime prior to RH sending the email then filtering gmail based on that time) but the code basically outputs a data frame on outputDF so you can filter based on time. Make sure when you create the credentials.json file (the file google provides you with all the password things) you save it using this name as it is referenced in the code. The google API is not hard to setup but may take some time depending on your level. Once setup it is pretty straight forward and code is reusable (which is what I've used below).
`''' For the API varificationSP...Function still needs to take in a datetime to determine a file that is later then the datetime input so that a record can be accessed that is the most recent and return the number which is being provided below Uses the Google Email API to take in emails from Robinhood, checks a listing of prior processed emails, and returns a data frame with emails that have not been processed.
'''
from future import print_function import pickle import os.path from googleapiclient.discovery import build from google_auth_oauthlib.flow import InstalledAppFlow from google.auth.transport.requests import Request import dateutil.parser as parser import base64 import email from apiclient import errors import pandas as pd import re
SCOPES = ['https://www.googleapis.com/auth/gmail.readonly']
def spitDigits(): creds = None
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server()
# Save the credentials for the next run
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('gmail', 'v1', credentials=creds)
query = "from:notifications@robinhood.com"
# Call the Gmail API
results = service.users().messages().list(userId='me', q=query, labelIds = ['INBOX']).execute()
labels = results.get('labels', [])
messages = results.get('messages', [])
outputDF = pd.DataFrame()
if not messages:
print("No messages found.")
else:
for message in messages:
msg = service.users().messages().get(userId='me', id=message['id']).execute()
payld = msg['payload'] # get payload of the message
headr = payld['headers'] # get header of the payload
subject_dict = { }
received_dict = { }
time_dict = { }
sender_dict = { }
messageId_dict = { }
for one in headr: # getting the Subject
if one['name'] == 'Subject':
msg_subject = one['value']
subject_dict['Subject'] = msg_subject
#print(subject_dict)
else:
pass
for two in headr: # getting the date
#print(two)
if two['name'] == 'Date':
msg_date = two['value']
date_parse = (parser.parse(msg_date))
received_dict['Received'] = str(date_parse)
else:
pass
for three in headr: # getting the Sender
if three['name'] == 'From':
msg_from = three['value']
sendList = []
sendList.append(msg_from)
sender_dict['Sender'] = sendList
#print(sender_dict)
else:
pass
#this returns the entire message
for pay in msg['payload']['parts']:
msg_str = base64.urlsafe_b64decode(pay['body']['data'].encode('UTF-8'))
messageId_dict['Message ID'] = msg['id']
d= {'messageID':msg['id'],'subject':msg_subject,'email_Received':date_parse \
,'sender':sendList,'snippet':msg['snippet'],'message':msg_str}
df = pd.DataFrame(data=d)
df['subject'] = df['subject'].astype(str).str.lower()
df['sender'] = df['sender'].astype(str).str.lower()
df['snippet'] = df['snippet'].astype(str).str.lower()
df['message'] = df['message'].astype(str).str.lower()
outputDF = outputDF.append(df)
outputDF = outputDF[outputDF['subject']=='your email verification code']
#print(outputDF)
pattern = "\<h3\>\d\d\d\d\d\d\<\/h3\>"
sub = ""
for i in re.findall(pattern, outputDF['message'].iloc[0]):
sub = re.sub(r'\<h3\>|\<\/h3\>','',i)
return(sub)`Just noticed for some reason the first part of my code is pasting funny...I'll try to get something cleaned up on my profile for easy access for those that may have trouble pulling/editing.
Chenzoh12
commented
5 years ago I am fairly new to coding especially with Python/ Python3, but I have found that you will always want the last email from RH and using gmail API quickstart I am able to get the challenge response from my gmail using the below:
def gmail_mfa_code():
creds = None
user_id = 'email@gmail.com'
query = 'from:notifications@robinhood.com'
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server()
# Save the credentials for the next run
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('gmail', 'v1', credentials=creds)
response = service.users().messages().list(userId=user_id, q=query).execute()
messages = []
if 'messages' in response:
messages.extend(response['messages'])
while 'nextPageToken' in response:
page_token = response['nextPageToken']
response = service.users().messages().get(userId=user_id, q=query, pageToken=page_token).execute()
messages.extend(response['messages'])
message_id = messages[0]['id']
message = service.users().messages().get(userId=user_id, id=message_id).execute()
challenge_response = message['snippet'][94:100]
print(challenge_response)
return challenge_responseHowever, I am struggling with posting that challenge code to clear the actual challenge. Any advice?
viaConBodhi
commented
5 years ago @Chenzoh12 @halessi Chenzoh12...finally got some time to catch up on this but seems I'm still getting the 400 message but I'm also getting emailed the pin only with a delay which is weird. I can't grab the "challenge" without the right return so I can't finish the build. Also...for some reason it worked once and then stopped which is even more weird. I think it may be related to the way I'm posting the "data" that has been suggested. One of comments from the other feed notes HTTP/2 is needed to make the post. Are you all using another library to make the posts as it looks like the requests library does not support HTTP/2.
halessi
commented
5 years ago @wccramer @Chenzoh12 Below is my code for the post, where I currently have the logic for dealing with authentication. I've also shared the code I have for GenerateDeviceToken and fetching the mfa_code, SEE MY POST ABOVE. As it stands, this works every time. I added the while loop because Robinhood is remarkably inconsistent in how long it takes to send the 2FA email out (sometimes < 1 second, others > 10).
NOTE: if you want to fetch the mfa_code using my function above, you must configure your Google account settings to allow for less secure connections. Otherwise you can fiddle around with the Google API, I just couldn't be bothered. Associated Robinhood 2FA with a non-important Gmail so if it's ever compromised as a result of changing security settings, no biggie.
ALSO: it is important you configure the "data" dictionary in login_oauth2 to be the following:
`if self.device_token == None:
self.device_token = self.GenerateDeviceToken()
data = {
"grant_type": "password",
"scope": "internal",
"client_id": CLIENT_ID,
"expires_in": 86400,
"password": password,
"username": username,
"challenge_type": 'email',
"device_token": self.device_token
}`As for how submitting the challenge works, you have to save the challenge_id from the original post and insert it:
url = 'https://api.robinhood.com/challenge/{}/respond/'.format(res.json()['challenge']['id'])
and also add it to the headers:
headers['X-ROBINHOOD-CHALLENGE-RESPONSE-ID'] = challenge_res.json()['id'].
Obviously having the 2FA logic inside post is less than ideal -- I'll move it and make a PR soon.
`def post(self, url=None, payload=None, retry=True):
'''
Execute HTTP POST
'''
headers = self._gen_headers(self.access_token, url)
attempts = 1
while attempts <= HTTP_ATTEMPTS_MAX:
try:
res = requests.post(url, headers=headers, data=payload,
timeout=15, verify=self.certs)
res.raise_for_status()
if res.headers['Content-Length'] == '0':
return None
else:
return res.json()
except requests.exceptions.RequestException as e:
attempts += 1
if res.status_code in [400, 429]:
if res.json()['challenge']:
''' HANDLE CHALLENGE 2FA '''
url = 'https://api.robinhood.com/challenge/{}/respond/'.format(res.json()['challenge']['id'])
validated = False
while validated == False:
time.sleep(3.5) # wait until the email has been sent
challenge_res = requests.post(url, headers = headers,
data = {'response' : self.fetch_mfa_code()}, timeout = 15, verify = self.certs)
try:
headers['X-ROBINHOOD-CHALLENGE-RESPONSE-ID'] = challenge_res.json()['id']
validated = True
except KeyError:
validated = False
''' TRY TO FETCH REFRESH/ACCESS TOKENS '''
url = "https://api.robinhood.com/oauth2/token/"
challenge_res = requests.post(url, headers = headers,
data = payload, timeout = 15, verify = self.certs)
#print('Response from submitting 2nd challenge code: {}'.format(json.dumps(challenge_res.json(), indent = 4)))
assert challenge_res is not None, print('Multi-factor challenge failed.')
return challenge_res.json()
else:
raise e
elif retry and res.status_code in [403]:
self.relogin_oauth2()`
pratik-r
commented
5 years ago @halessi Thanks for sharing your solution. Could you post a copy of your entire client.py file? Would make it a lot easier to debug.
halessi
commented
5 years ago @pratik-r Find below. Haven't had time to fix it yet, but it gets stuck in the while loop ~25% of the time (whoops).
Another cool (and necessary!) enhancement would involve storing the device_token that is validated for use in future logins. I believe they said it remains valid for ~24 hours over in the other Robinhood thread.
import imaplib
import os
import random
import time
import re
import json
from datetime import datetime, timedelta
import requests
from fast_arrow.exceptions import AuthenticationError, NotImplementedError
from fast_arrow.resources.account import Account
from fast_arrow.resources.user import User
from fast_arrow.util import get_last_path
CLIENT_ID = "c82SH0WZOsabOXGP2sxqcj34FxkvfnWRZBKlBjFS"
HTTP_ATTEMPTS_MAX = 2
class Client(object):
def __init__(self, **kwargs):
self.options = kwargs
self.account_id = None
self.account_url = None
self.access_token = None
self.refresh_token = None
self.mfa_code = None
self.device_token = None
self.scope = None
self.authenticated = False
self.certs = os.path.join(os.path.dirname(__file__), 'ssl_certs/certs.pem')
def authenticate(self):
'''
Authenticate using data in `options`
'''
if "username" in self.options and "password" in self.options:
self.login_oauth2(self.options["username"], self.options["password"], self.options.get('mfa_code'))
elif "access_token" in self.options and "refresh_token" in self.options:
self.access_token = self.options["access_token"]
self.refresh_token = self.options["refresh_token"]
self.__set_account_info()
else:
self.authenticated = False
return self.authenticated
def get(self, url=None, params=None, retry=True):
'''
Execute HTTP GET
'''
headers = self._gen_headers(self.access_token, url)
attempts = 1
while attempts <= HTTP_ATTEMPTS_MAX:
try:
res = requests.get(url, headers=headers, params=params, timeout=15, verify=self.certs)
res.raise_for_status()
return res.json()
except requests.exceptions.RequestException as e:
attempts += 1
if res.status_code in [400]:
raise e
elif retry and res.status_code in [403]:
self.relogin_oauth2()
def post(self, url=None, payload=None, retry=True):
'''
Execute HTTP POST
'''
headers = self._gen_headers(self.access_token, url)
attempts = 1
while attempts <= HTTP_ATTEMPTS_MAX:
try:
res = requests.post(url, headers=headers, data=payload,
timeout=15, verify=self.certs)
res.raise_for_status()
if res.headers['Content-Length'] == '0':
return None
else:
return res.json()
except requests.exceptions.RequestException as e:
attempts += 1
if res.status_code in [400, 429]:
if res.json()['challenge']:
''' HANDLE CHALLENGE 2FA '''
url = 'https://api.robinhood.com/challenge/{}/respond/'.format(res.json()['challenge']['id'])
validated, trys = False, 0
while validated == False:
trys += 1; print('Trys: {}.'.format(trys))
time.sleep(5) # wait until the email has been sent
challenge_res = requests.post(url, headers = headers,
data = {'response' : self.fetch_mfa_code()}, timeout = 15, verify = self.certs)
try:
headers['X-ROBINHOOD-CHALLENGE-RESPONSE-ID'] = challenge_res.json()['id']
validated = True
except KeyError:
print(json.dumps(challenge_res.json(), indent = 4))
validated = False
''' TRY TO FETCH REFRESH/ACCESS TOKENS '''
url = "https://api.robinhood.com/oauth2/token/"
challenge_res = requests.post(url, headers = headers,
data = payload, timeout = 15, verify = self.certs)
#print('Response from submitting 2nd challenge code: {}'.format(json.dumps(challenge_res.json(), indent = 4)))
assert challenge_res is not None, print('Multi-factor challenge failed.')
return challenge_res.json()
else:
raise e
elif retry and res.status_code in [403]:
self.relogin_oauth2()
def _gen_headers(self, bearer, url):
'''
Generate headders, adding in Oauth2 bearer token if present
'''
headers = {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"Accept-Language": "en;q=1, fr;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36",
}
if bearer:
headers["Authorization"] = "Bearer {0}".format(bearer)
if url == "https://api.robinhood.com/options/orders/":
headers["Content-Type"] = "application/json; charset=utf-8"
return headers
def GenerateDeviceToken():
rands = []
for i in range(0,16):
r = random.random()
rand = 4294967296.0 * r
rands.append((int(rand) >> ((3 & i) << 3)) & 255)
hexa = []
for i in range(0,256):
hexa.append(str(hex(i+256)).lstrip("0x").rstrip("L")[1:])
id = ""
for i in range(0,16):
id += hexa[rands[i]]
if (i == 3) or (i == 5) or (i == 7) or (i == 9):
id += "-"
return id
def fetch_mfa_code(self):
username, password = user, pass
obj = imaplib.IMAP4_SSL('imap.gmail.com')
obj.login(username, password)
obj.select()
cutoff = (datetime.today() - timedelta(minutes = 2)).strftime('%d-%b-%Y')
typ, data = obj.search(None,
'(SINCE %s) (FROM "notifications@robinhood.com")' % (cutoff,))
email_ids = data[0].decode().split(' ')
email = obj.fetch(email_ids[-1], '(UID BODY[TEXT])')
parsed = email[1][0][1].decode().split('r')[9]
mfa_code = [int(s) for s in re.findall(pattern = r'\d+', string = parsed)]
#assert len(str(mfa_code[0])) == 6
if len(str(mfa_code[0])) != 6: mfa_code = '0' + str(mfa_code) # HACK: fix
return mfa_code[0]
def login_oauth2(self, username, password, mfa_code=None, id=None):
'''
Login using username and password
'''
self.username = username
self.password = password
if self.device_token == None:
self.device_token = self.GenerateDeviceToken()
data = {
"grant_type": "password",
"scope": "internal",
"client_id": CLIENT_ID,
"expires_in": 86400,
"password": password,
"username": username,
"challenge_type": 'email',
"device_token": self.device_token
}
url = "https://api.robinhood.com/oauth2/token/"
res = self.post(url, payload = data, retry = False)
if res is None:
if mfa_code is None:
msg = "Client.login_oauth2(). Could not authenticate. Check username and password."
raise AuthenticationError(msg)
else:
msg = "Client.login_oauth2(). Could not authenticate. Check username and password, and enter a valid MFA code."
raise AuthenticationError(msg)
elif res.get('mfa_required') is True:
msg = "Client.login_oauth2(). Could not authenticate. MFA is required."
raise AuthenticationError(msg)
self.access_token = res["access_token"]
self.refresh_token = res["refresh_token"]
self.mfa_code = res["mfa_code"]
self.scope = res["scope"]
self.__set_account_info()
return self.authenticated
def __set_account_info(self):
account_urls = Account.all_urls(self)
if len(account_urls) > 1:
msg = "fast_arrow 'currently' does not handle multiple account authentication."
raise NotImplementedError(msg)
elif len(account_urls) == 0:
msg = "fast_arrow expected at least 1 account."
raise AuthenticationError(msg)
else:
self.account_url = account_urls[0]
self.account_id = get_last_path(self.account_url)
self.authenticated = True
def relogin_oauth2(self):
'''
(Re)login using the Oauth2 refresh token
'''
url = "https://api.robinhood.com/oauth2/token/"
data = {
"grant_type": "refresh_token",
"refresh_token": self.refresh_token,
"scope": "internal",
"client_id": CLIENT_ID,
"expires_in": 86400,
}
res = self.post(url, payload=data, retry=False)
self.access_token = res["access_token"]
self.refresh_token = res["refresh_token"]
self.mfa_code = res["mfa_code"]
self.scope = res["scope"]
def logout_oauth2(self):
'''
Logout for given Oauth2 bearer token
'''
url = "https://api.robinhood.com/oauth2/revoke_token/"
data = {
"client_id": CLIENT_ID,
"token": self.refresh_token,
}
res = self.post(url, payload=data)
if res == None:
self.account_id = None
self.account_url = None
self.access_token = None
self.refresh_token = None
self.mfa_code = None
self.scope = None
self.authenticated = False
return True
else:
raise AuthenticationError("fast_arrow could not log out.")@halessi thanks for passing this along...haven't run it yet but looks like you were pulling the 'challenge' from the 400 return which I didn't think was possible with that return . I'll let you know if I have challenges with this. I'm getting emails with access code around the same deltas you described so looks like, at least, that's consistent. Since I was getting 400 codes I thought something was wrong. If I can get this working I'm thinking I could use a stamp = datetime.now when the function initiates and this could be passed to the function for the email processing via a loop (as needed to deal with the email sending delta) so the filter can be based on the time the original function was called. Capturing the return challenge could also be passed to the email function to output a final send for complete login.
pratik-r
commented
5 years ago @halessi I just tried out your code, and I can confirm this works. I just had to change a few lines in the get_mfa_code function. Apparently my parser doesn't work the same way as yours. But it definitely works. Thanks again.
jwschmo
commented
5 years ago I got it working by adding a device ID from a computer that was already logged in....
Used the latest code from the master branch, with the below modification with no issues....
data = {
"grant_type": "password",
"scope": "internal",
"client_id": CLIENT_ID,
"device_token": DEVICE_ID,
"expires_in": 86400,
"password": password,
"username": username@jwschmo I just tried it using my desktop's device ID. I logged in manually through the website first, but I'm still getting the same 400 error. I got the device ID from my system settings. What device ID did you use?
halessi
commented
5 years ago The specific ID does not matter, hence the GenerateID function. Any ID in a proper format can be validated by submitting the mfa code to the aforementioned URL with the challenge ID (taken from the original request that was met with a 400 response).
The next step (which I think will take little code, just file formatting) is to save the validated DeviceToken and upon authenticating trying to use that, which will work as long as it was validated within the last 24hrs. It will require some conditioning, as I think we'd like the client to automatically redo the 2FA if it's been longer than 24hrs and the token is no longer validated (fails authentication).
Get Outlook for iOShttps://aka.ms/o0ukef
From: pratik-r notifications@github.com Sent: Saturday, May 4, 2019 8:23 PM To: westonplatter/fast_arrow Cc: Hugh Alessi (S); Mention Subject: Re: [westonplatter/fast_arrow] Login fails (#85)
This email originated outside Colorado College. Do not click links or attachments unless you know the content is safe.
@jwschmohttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjwschmo&data=02%7C01%7Ch_alessi%40coloradocollege.edu%7Cf58d8536a50b4006cf6508d6d100b0ab%7Ccfc7b13c12964387b3085de08fd13c99%7C1%7C0%7C636926198233743961&sdata=bnewYwHCbClb9jY5MxpqYUhYLVbf4006tEMLO4m77IE%3D&reserved=0 I just tried it using my desktop's device ID. I logged in manually through the website first, but I'm still getting the same 400 error. I got the device ID from my system settings. What device ID did you use?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fwestonplatter%2Ffast_arrow%2Fissues%2F85%23issuecomment-489381824&data=02%7C01%7Ch_alessi%40coloradocollege.edu%7Cf58d8536a50b4006cf6508d6d100b0ab%7Ccfc7b13c12964387b3085de08fd13c99%7C1%7C0%7C636926198233743961&sdata=iqOTz%2Fe2Y8LbbyMHwm8ZFFNRpVezom%2B8RxKiaS3kank%3D&reserved=0, or mute the threadhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIKCMQXIPFMVYAIZSPMJRCLPTZAKVANCNFSM4HJE7IYQ&data=02%7C01%7Ch_alessi%40coloradocollege.edu%7Cf58d8536a50b4006cf6508d6d100b0ab%7Ccfc7b13c12964387b3085de08fd13c99%7C1%7C0%7C636926198233753959&sdata=CGTOL5buniDHHZ4Q%2F6E9reaUAtfZzOCzJBdaotp26Fw%3D&reserved=0.
stewood
commented
5 years ago This is showing closed to me. Is that correct?
Edit-- Nevermind. I see #86 open. Just got confused with all the updates going here.
viaConBodhi
commented
5 years ago @halessi you rock...I was able to integrate the google API and now have access. I was stuck at the post even though I was getting an email with the code and now I'm catching the challenge. I owe you a beer. Good looking out.
westonplatter
commented
5 years ago First off, hats off to all the people who contributed to this conversation. Thanks for your comments, experimentation, and feedback. I sat down this morning to get things working for myself and leveraged so much of the above conversation. Thank you: @viaConBodhi @halessi @Chenzoh12 @pratik-r @jwschmo @stewood.
Here's my attempt to summarize the conversation and add my own findings (please, correct me if I'm missing something and I'll correct/edit this list),
device_token is present in the HTTP POST bodydevice_token has not been previously authenticated, Robinhood will require confirmation via Email or Sms.device_token confirmation process.My goal is to come up with a simple approach to handling the device_token confirmation process so that authentication "works out of the box" without issues for all users. Here are a couple approaches that came to mind. There are likely many more.
1. Adjust the fast_arrow/client.Client to interactively allow users to input the device_token confirmation code. Pros: minimal code changes to fast_arrow; relatively easy to code up. Cons: fast_arrow changes from being a client and becomes an interactive, shell client library.
2. Move authentication out of fast_arrow into a supporting library. Pros: reduces complexity and helps fast_arrow focus on doing relatively few things exceptionally well; really opens the door to providing detailed implementations for automated device_token fetching. Cons: adds another codebase to develop and upkeep.
My preference is option 2; I've got some free cycles the next couple days and will share a simple example library. Once the community has gotten a chance to give it a try, I'd love feedback. However, this proposed solution my not meet everyone's needs so totally open considering other options.
westonplatter
commented
5 years ago Reopening in order to track / keep notes on solving the device_token issue that keeps authentication from working right out of the box.
viaConBodhi
commented
5 years ago My vote is also for option 2. Authentication is a separate and now "more involved" function and depending on the use cases and interfaces people are using option 1 could be a painted corner. Option 2, at least from what I've seen, will still require more user configuration (email or SMS capture) which will relinquish the whole "turn-key" approach but them's the breaks to meet the security requirement. The email approach has been working for me but I still have a few small bugs that I'm willing to live with as I work on other more interesting items. Authentication is the first step in my workflow so if it fails I simply rerun and it works. Not production quality but worthy for what I've knocking out. Thanks again for continuing to support this library as I've found it very resourceful.
klepsydra
commented
5 years ago Regarding approach 2:
Could we attempt to re-use the same auth.data file already generated by anilshanbhag's RobinhoodShell (following auto-fetching its email verification code as needed)? So as to try to maintain the same login session across separate but similar RH projects:
{
"auth_token": "eyJ0eSUzI1NiJ9.eyJleHAiOjE1NjM4NDY5MTgsInRva2VuIjoiZzRhVFdzV0owc09HTVB0TUsXAiOiJKV1QiLCJhbGciOiJzdFg4b0pVT1lFNEp1IiwidXNlcl9pZCI6ImI4ZjgxNDg0LTYxODgtNDFiZi1iZGI4LTJlNzhjNTUxMWRlNCIsIm9wdGlvbnMiOnRydWUsImxldmVsMl9hY2Nlc3MiOnRydWV9.Wzt85kF2OWM0QrK7U74LCXMMsMrJJeAFNZ6Xj8sgDpg8i2zG4YYGA7nR3v3C_uQPdAqO8ttPBsRsqayBUqNet6LAKbgVwmvbOarsFM-iuYkLCZeuCftGxIEH7toJYsir835r6djXdso8GfZjLLCbu_vfou6dhH4Dmwq4Gwev3nefOxw7ns1dxZLvuyNM7neaXvu6eQrZckI-1TM_OzvxkfQUTYK-B3x1whIh0qplIcvLQVA9o2JUkCgV4UTmfr88-r0GyzRCGhIHqu0n-LC_RKKoQZifPJ6Fd7r0s4eXn3lIDAYxur4VPinxomMJJpexFb9C_HML0WSIlcchNduhQw",
"device_token": "8e48a02a-9dc4-7d0d-dbea-be40a1c9c206",
"refresh_token": "NoM3RJ0DIThKE3ZdzMDa0a9RbX988X"
}
@klepsydra funny you should mention that as I've been thinking about building out a small tool for this exact topic but have been caught up on other stuff and limping along with something I've hot-wired together. I've been authenticating first with the Fast_Arrow and then passing this auth into a modified "login" for the Jamonek Robinhood package as there are some functions in this library that are not in Fast_Arrow. This same approach may work for your use case or could be incorporated into more general Auth Method for Opt 2
Sorry I know this maybe out of scope of the Fast_Arrow but maybe still in scope with a general Auth Method for opt 2.
Passing along these keys is working for me like the following.
Auth with Fast_Arrow using client = Client(username="X", password="X") client.authenticate()
Pass the keys to Jamonek my_trader = Robinhood() logged_in = my_trader.login(client) logged_in
The Jamonek modify is below. I've included the entire logging method just encase.
` ###########################################################################
###########################################################################
def __init__(self):
self.session = requests.session()
self.session.proxies = getproxies()
self.headers = {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"Accept-Language": "en;q=1, fr;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5",
"Content-Type": "application/x-www-form-urlencoded; charset=utf-8",
"X-Robinhood-API-Version": "1.0.0",
"Connection": "keep-alive",
"User-Agent": "Robinhood/823 (iPhone; iOS 7.1.2; Scale/2.00)"
}
self.session.headers = self.headers
self.device_token = ""
self.challenge_id = ""
def login_required(function): # pylint: disable=E0213
""" Decorator function that prompts user for login if they are not logged in already. Can be applied to any function using the @ notation. """
def wrapper(self, *args, **kwargs):
if 'Authorization' not in self.headers:
self.auth_method()
return function(self, *args, **kwargs) # pylint: disable=E1102
return wrapper
def GenerateDeviceToken(self):
rands = []
for i in range(0,16):
r = random.random()
rand = 4294967296.0 * r
rands.append((int(rand) >> ((3 & i) << 3)) & 255)
hexa = []
for i in range(0,256):
hexa.append(str(hex(i+256)).lstrip("0x").rstrip("L")[1:])
id = ""
for i in range(0,16):
id += hexa[rands[i]]
if (i == 3) or (i == 5) or (i == 7) or (i == 9):
id += "-"
self.device_token = id
def get_mfa_token(self, secret):
intervals_no = int(time.time())//30
key = base64.b32decode(secret, True)
msg = struct.pack(">Q", intervals_no)
h = hmac.new(key, msg, hashlib.sha1).digest()
o = h[19] & 15
h = '{0:06d}'.format((struct.unpack(">I", h[o:o+4])[0] & 0x7fffffff) % 1000000)
return h
def login(self,*args):
for i in args:
testing = i.access_token
username = i.username
password = i.password
device_token = i.device_token
auth_token = i.access_token
refresh_token = i.refresh_token
self.auth_token = auth_token
self.refresh_token = refresh_token
self.headers['Authorization'] = 'Bearer ' + auth_token
def auth_method(self):
if self.qr_code:
payload = {
'password': self.password,
'username': self.username,
'grant_type': 'password',
'client_id': "c82SH0WZOsabOXGP2sxqcj34FxkvfnWRZBKlBjFS",
'scope': 'internal',
'device_token': self.device_token,
'mfa_code': self.get_mfa_token(self.qr_code)
}
try:
res = self.session.post(login(), data=payload, timeout=15)
data = res.json()
if 'access_token' in data.keys() and 'refresh_token' in data.keys():
self.auth_token = data['access_token']
self.refresh_token = data['refresh_token']
self.headers['Authorization'] = 'Bearer ' + self.auth_token
return True
except requests.exceptions.HTTPError:
raise RH_exception.LoginFailed()
else:
payload = {
'password': self.password,
'username': self.username,
'grant_type': 'password',
'client_id': "c82SH0WZOsabOXGP2sxqcj34FxkvfnWRZBKlBjFS",
'expires_in': '86400',
'scope': 'internal',
'device_token': self.device_token,
}
try:
res = self.session.post(login(), data=payload, timeout=15)
res.raise_for_status()
data = res.json()
if 'access_token' in data.keys() and 'refresh_token' in data.keys():
self.auth_token = data['access_token']
self.refresh_token = data['refresh_token']
self.headers['Authorization'] = 'Bearer ' + self.auth_token
return True
except requests.exceptions.HTTPError:
raise RH_exception.LoginFailed()
return False
def logout(self):
"""Logout from Robinhood
Returns:
(:obj:`requests.request`) result from logout endpoint
"""
try:
payload = {
'client_id': self.client_id,
'token': self.refresh_token
}
req = self.session.post(logout(), data=payload, timeout=15)
req.raise_for_status()
except requests.exceptions.HTTPError as err_msg:
warnings.warn('Failed to log out ' + repr(err_msg))
self.headers['Authorization'] = None
self.auth_token = None
return req`
westonplatter
commented
5 years ago @klepsydra yes, I think we could that.
Here's my current take on option 2, https://github.com/westonplatter/fast_arrow_auth/pull/2.
westonplatter
commented
5 years ago I've merged https://github.com/westonplatter/fast_arrow/pull/94 as a 1.0.0 release candidate. Code is hard to get exactly right, so expecting that the current approach doesn't hit all needs. Happy to work through details and make adjustments to get something that's solid and useable.
westonplatter
commented
5 years ago Closing this issue after release 1.0.0 in the wild. Please open another issue if you run into authentication issues.
upgraded to latest package today to see if that would fix and still having issues...all was running fine last Friday and today I'm getting a
400 Client Error: Bad Request for url: https://api.robinhood.com/oauth2/token/
log into Fast_Arrow
----> 3 client.authenticate() 4 #login_oauth2 5 # client = Client.login_oauth2(self,username=XXXXX, password=XXXXX) ~\AppData\Local\Continuum\anaconda3\envs\py3.6\lib\site-packages\fast_arrow\client.py in authenticate(self) 32 ''' 33 if "username" in self.options and "password" in self.options: ---> 34 self.login_oauth2(self.options["username"], self.options["password"], self.options.get('mfa_code')) 35 elif "access_token" in self.options and "refresh_token" in self.options: 36 self.access_token = self.options["access_token"]
~\AppData\Local\Continuum\anaconda3\envs\py3.6\lib\site-packages\fast_arrow\client.py in login_oauth2(self, username, password, mfa_code) 117 data['mfa_code'] = mfa_code 118 url = "https://api.robinhood.com/oauth2/token/" --> 119 res = self.post(url, payload=data, retry=False) 120 121 if res is None:
~\AppData\Local\Continuum\anaconda3\envs\py3.6\lib\site-packages\fast_arrow\client.py in post(self, url, payload, retry) 78 attempts += 1 79 if res.status_code in [400]: ---> 80 raise e 81 elif retry and res.status_code in [403]: 82 self.relogin_oauth2()
~\AppData\Local\Continuum\anaconda3\envs\py3.6\lib\site-packages\fast_arrow\client.py in post(self, url, payload, retry) 70 try: 71 res = requests.post(url, headers=headers, data=payload, timeout=15, verify=self.certs) ---> 72 res.raise_for_status() 73 if res.headers['Content-Length'] == '0': 74 return None
~\AppData\Local\Continuum\anaconda3\envs\py3.6\lib\site-packages\requests\models.py in raise_for_status(self) 938 939 if http_error_msg: --> 940 raise HTTPError(http_error_msg, response=self) 941 942 def close(self):