tedchou12 / webull

Unofficial APIs for Webull.
MIT License
597 stars 181 forks source link

Bypassing Image Verification to Login?! #367

Closed tradingwithme closed 1 year ago

tradingwithme commented 1 year ago

I am facing a conundrum: Is there anyone out there who can shed some light on this vexing issue? I am trying to resolve the verify picture verification, but alas, my attempts with the email address have been fruitless. Instead, I have been directed to use my phone number, yet every time I try, I am met with a frustrating error message on the Jupyter terminal. Can anyone help me navigate and find a solution to this problem?

image

image

Alternatively, I tried the "Workaround Login" method. However, it seems that the html document for Webull has changed since the last screenshot was taken by @tedchou12 . Therefore, the keywords are no longer searchable.

tedchou12 commented 1 year ago

Thanks for the feedback!! Using accesstoken should find it.

tradingwithme commented 1 year ago

Thanks for the feedback!! Using accesstoken should find it.

Thank you for your reply. I still have encountered another roadblock after attempting your suggestion.

I was able to obtain the 'access_token' information from the HAR-formatted JavaScript data file I downloaded from the "https://app.webull.com/paper" url.

Here is the data that I called to become read: image

Afterward, I ran the following code statement(s), but to no avail:

image

What would you suggest I do to resolve this issue?

tedchou12 commented 1 year ago

sure, let me check it latter today

tradingwithme commented 1 year ago

sure, let me check it latter today

Thanks so much. Please, keep me posted.

shaggy63 commented 1 year ago

Here is exactly what I did: open /usr/local/lib/python3.9/dist-packages/webull/endpoints.py, change the self.base_userfintech_url endpoint to: self.base_userfintech_url = 'https://nauser.webullfintech.com/api'

Open an incognito browser window, inspect. Login to webull. Look for the POST to the same url. Copy the deviceid and device name.

removed did.bin In my login code: wb = webull() wb._did = "DeviceIDHERE" wb.accountType = '2' data = wb.login('username@here', 'PASShere', device_name="DeviceNAMEHERE", save_token=True) print("\n\n\n\n"+ str(data) + "\n\n\n\n\n")

Then in my other code, I use this so I don't have to login on each run: wb = webull() wb._access_token = 'AccessToken_from_login_above'

This all may not be correct, I'm just trying to help since I was having the same problems and this resolved it.

tradingwithme commented 1 year ago

Here is exactly what I did: open /usr/local/lib/python3.9/dist-packages/webull/endpoints.py, change the self.base_userfintech_url endpoint to: self.base_userfintech_url = 'https://nauser.webullfintech.com/api'

Open an incognito browser window, inspect. Login to webull. Look for the POST to the same url. Copy the deviceid and device name.

removed did.bin In my login code: wb = webull() wb._did = "DeviceIDHERE" wb.accountType = '2' data = wb.login('username@here', 'PASShere', device_name="DeviceNAMEHERE", save_token=True) print("\n\n\n\n"+ str(data) + "\n\n\n\n\n")

Then in my other code, I use this so I don't have to login on each run: wb = webull() wb._access_token = 'AccessToken_from_login_above'

This all may not be correct, I'm just trying to help since I was having the same problems and this resolved it.

@tedchou12 & @shaggy63

Thank you for your reply. I still have encountered another roadblock after attempting your suggestion.

I have tried your suggestion as follows: image image image However, it printed the following message: image

What would you suggest I do to resolve this issue? Would an api login work instead because I have attempted that as well?

shaggy63 commented 1 year ago

Did you edit the actual endpoints.py file that was installed with this package?

tradingwithme commented 1 year ago

Did you edit the actual endpoints.py file that was installed with this package? @tedchou12 @shaggy63

Thank you for your reply. I am unsure what the following outputs imply. Could @tedchou12, @shaggy63, or anybody else tell if the following code statements are correct in terms of format:

I could not find the 'device_Name' element/field on the JSON document. Is it available on the HTTP page? If so, where?

image image image image

Please, let me know if there is anything else you could suggest to improve this code. Thanks for your help.

tedchou12 commented 1 year ago

@tradingwithme

截圖 2022-12-30 上午9 20 01

If you save the json in the right of the image as a json file, it should automatically load in the vars without you doing anything else:

from webull_package.webull import webull, paper_webull
import json

wb = paper_webull()

fh = open('webull_credentials.json', 'r')
credential_data = json.load(fh)
fh.close()
#
# print(credential_data)
wb._refresh_token = credential_data['refreshToken']
wb._access_token = credential_data['accessToken']
wb._token_expire = credential_data['tokenExpireTime']
wb._uuid = credential_data['uuid']

n_data = wb.refresh_login()
# print(n_data)

credential_data['refreshToken'] = n_data['refreshToken']
credential_data['accessToken'] = n_data['accessToken']
credential_data['tokenExpireTime'] = n_data['tokenExpireTime']

file = open('webull_credentials.json', 'w')
json.dump(credential_data, file)
file.close()

print(wb.get_account())
osaidz commented 1 year ago

Hi, I'm am facing the exact same issue on a new account.

Access token is only searchable on https://www.webull.com/center but refresh_token and token_expire are missing so I am unable to login using the api.

Could this be because I have not opened a full account (Non-US citizen)? I can however see quotes and level 2 data.

tedchou12 commented 1 year ago

@osaidz Are you able to find it here? https://app.webull.com/

osaidz commented 1 year ago

@osaidz Are you able to find it here? https://app.webull.com/

No

at

tradingwithme commented 1 year ago

@osaidz Are you able to find it here? https://app.webull.com/

No

at

Try typing the keyword 'access_token" on the search engine. It should return a result. The keywords that @tedchou12 has could appear different on your JSON screen. Try typing a part of the keyword to see the terms that the search results return.

osaidz commented 1 year ago

@tedchou12 Thanks,

It does return a lot of results all pointing to the same variable but I'm unable find this string mentioned in

https://github.com/tedchou12/webull/wiki/Workaround-for-Login

{"extInfo":{"userPwdFlag":"1","equipmentCheck":"success"},"accessToken":"dc_us1.188awefa4-00000000000000000000000000000000","uuid":"165dbaa7333aeawf312151ar1f1dc804b9b","refreshToken":"188a0awef2444-00000000000000000000000000000000","tokenExpireTime":"2021-02-14T13:35:35.780+0000","firstTimeOfThird":false,"registerAddress":169,"settings":{"id":94551253056757760,"userId":300431177,"regionId":169,"language":"en","focusMarketId":"2,3,4,5,6,14","theme":2,"increDecreColor":2,"fontSize":"M","portfolioDisplayMode":2,"portfolioNameNewline":1,"portfolioHoldingsDisplay":1,"portfolioIndexDisplay":1,"portfolioBulletin":1,"kdata":1,"refreshFrequency":1,"shock":0,"tickerPriceRemind":1,"orderDealRemind":1,"hotNews":1,"chartOption":2,"operateTime":"1969-12-31T00:00:00.000+0000","languageUpdateTime":"1970-01-01T00:00:00.000+0000","createTime":"2018-09-15T05:18:25.000+0000","updateTime":"2018-09-15T05:18:25.000+0000"}}

tradingwithme commented 1 year ago

Just found them for you:

"tokenExpireTime":"2021-02-14T13:35:35.780+0000" "refreshToken":"188a0awef2444-00000000000000000000000000000000" "accessToken":"dc_us1.188awefa4-00000000000000000000000000000000","uuid":"165dbaa7333aeawf312151ar1f1dc804b9b"

FYI: Do not share this kind of information. It's hackable.

osaidz commented 1 year ago

Just found them for you:

"tokenExpireTime":"2021-02-14T13:35:35.780+0000" "refreshToken":"188a0awef2444-00000000000000000000000000000000" "accessToken":"dc_us1.188awefa4-00000000000000000000000000000000","uuid":"165dbaa7333aeawf312151ar1f1dc804b9b"

FYI: Do not share this kind of information. It's hackable.

Thanks! but I copied this string from the original page here https://github.com/tedchou12/webull/wiki/Workaround-for-Login.

This is what I see. image

if I take part of that string and search it then I do not get any results.

image

tradingwithme commented 1 year ago

@tradingwithme

截圖 2022-12-30 上午9 20 01

If you save the json in the right of the image as a json file, it should automatically load in the vars without you doing anything else:

from webull_package.webull import webull, paper_webull
import json

wb = paper_webull()

fh = open('webull_credentials.json', 'r')
credential_data = json.load(fh)
fh.close()
#
# print(credential_data)
wb._refresh_token = credential_data['refreshToken']
wb._access_token = credential_data['accessToken']
wb._token_expire = credential_data['tokenExpireTime']
wb._uuid = credential_data['uuid']

n_data = wb.refresh_login()
# print(n_data)

credential_data['refreshToken'] = n_data['refreshToken']
credential_data['accessToken'] = n_data['accessToken']
credential_data['tokenExpireTime'] = n_data['tokenExpireTime']

file = open('webull_credentials.json', 'w')
json.dump(credential_data, file)
file.close()

print(wb.get_account())

@tedchou12 The keywords are different compared to mine because my account is based off the U.S. Therefore, the EXACT keywords cannot be found on JSON data. I found a search result that returned as "access_token" instead of "accessToken." Are there any other terms that I could find compatible to my JSON data?

tedchou12 commented 1 year ago

Sorry, I am not sure about that. May be it doesn't work for newer accounts that have not been fully activated?

Could this be because I have not opened a full account (Non-US citizen)? I can however see quotes and level 2 data.

How did you sign up?

tradingwithme commented 1 year ago

Just found them for you: "tokenExpireTime":"2021-02-14T13:35:35.780+0000" "refreshToken":"188a0awef2444-00000000000000000000000000000000" "accessToken":"dc_us1.188awefa4-00000000000000000000000000000000","uuid":"165dbaa7333aeawf312151ar1f1dc804b9b" FYI: Do not share this kind of information. It's hackable.

Thanks! but I copied this string from the original page here https://github.com/tedchou12/webull/wiki/Workaround-for-Login.

This is what I see. image

if I take part of that string and search it then I do not get any results.

image

Yup. The format for that entry matches mine: It's similar to mine. According to @shaggy63, only the access token information should suffice. Follow up with an API login and verify your access using 'wb.is_logged_in()' code statement to see if the boolean statement returns 'True.'

osaidz commented 1 year ago

Sorry, I am not sure about that. May be it doesn't work for newer accounts that have not been fully activated?

Could this be because I have not opened a full account (Non-US citizen)? I can however see quotes and level 2 data.

How did you sign up?

I signed up using my phone number then linked my email but I was not able to proceed further because it says I need to be a US resident. I'm based in the UK.

I can only view the markets and level 2 quotes.

tedchou12 commented 1 year ago

Unfortunately, I cant signup with another phone number (Ive only got one) May be https://github.com/tedchou12/webull/issues/367#issuecomment-1367686829 would work?

osaidz commented 1 year ago

@tedchou12 There might be a bug becasue wb.is_logged_in() is returning true, if I run the code below.

from webull import webull, StreamConn
wb = webull()
wb._access_token = 'some_token'
wb.is_logged_in()
tradingwithme commented 1 year ago

@tedchou12 There might be a bug becasue wb.is_logged_in() is returning true, if I run the code below.

from webull import webull, StreamConn
wb = webull()
wb._access_token = 'some_token'
wb.is_logged_in()

You might have to forgo implementing the 'wb.login' code statement. Instead, try 'wb.get_account_ID()' or 'wb.get_account' code statements to see if it would return anything. If so, then you have confirmed that you are indeed logged-in.

SweetNaterTater commented 1 year ago

Hey all,

I recently started playing with this package for trying to create an automated trading bot, for eventually having a bot that auto trades options by the barUpDn strategy on a paper account. I was also having issues with being able to login to my account, but you need to make sure you have selected "inspect". Then press network, so that you can begin recording any network activity that occurs when performing any actions on the web page. Then simply login to your account with your credentials, and entering your token(6 digit code for mfa). Then you should be able to find a fetched item that contains the token information.

osaidz commented 1 year ago

@tradingwithme Yes, I think I am logged in?

if I do wb.get_account()

I get

{'netLiquidation': '0.00', 'unrealizedProfitLoss': '0.00', 'unrealizedProfitLossRate': '0.0000', 'accounts': [], 'positions': [], 'openOrders': [], 'openOrderSize': 0, 'positions2': [], 'openOrders2': [], 'openIpoOrders': []}

if I do wb.refresh_login()

I get

{'msg': 'AccessToken is invalid',
 'traceId': 'ac4dcee7c7964191ae0566d2a0376f4c',
 'code': 'auth.token.req'}
tradingwithme commented 1 year ago

@tradingwithme Yes, I think I am logged in?

if I do wb.get_account()

I get

{'netLiquidation': '0.00', 'unrealizedProfitLoss': '0.00', 'unrealizedProfitLossRate': '0.0000', 'accounts': [], 'positions': [], 'openOrders': [], 'openOrderSize': 0, 'positions2': [], 'openOrders2': [], 'openIpoOrders': []}

if I do wb.refresh_login()

I get

{'msg': 'AccessToken is invalid',
 'traceId': 'ac4dcee7c7964191ae0566d2a0376f4c',
 'code': 'auth.token.req'}

I only know what I tried. You could try assigning the 'did' value to 'wb._did' to see if that would work.

I did not get the output after running the 'wb.get_account() ' code statement, as shown on your post. However, I obtained a value instead from implementing the following: print(wb.get_account()). You might not need to implement the wb.refresh_login() once you run 'wb.get_account()' because what it does, I presume, is that it would cause your login status to refresh itself.

Alternatively, you could try replicating what I did here to see if you get a similar result: https://github.com/tedchou12/webull/issues/367#issuecomment-1366922522.

Do you have any existing orders or whatnot?

tradingwithme commented 1 year ago

Hey all,

I recently started playing with this package for trying to create an automated trading bot, for eventually having a bot that auto trades options by the barUpDn strategy on a paper account. I was also having issues with being able to login to my account, but you need to make sure you have selected "inspect". Then press network, so that you can begin recording any network activity that occurs when performing any actions on the web page. Then simply login to your account with your credentials, and entering your token(6 digit code for mfa). Then you should be able to find a fetched item that contains the token information.

Does this refer to the access_token value available on the Inspect window?

osaidz commented 1 year ago

@tradingwithme Yes, I think I am logged in? if I do wb.get_account() I get {'netLiquidation': '0.00', 'unrealizedProfitLoss': '0.00', 'unrealizedProfitLossRate': '0.0000', 'accounts': [], 'positions': [], 'openOrders': [], 'openOrderSize': 0, 'positions2': [], 'openOrders2': [], 'openIpoOrders': []} if I do wb.refresh_login() I get

{'msg': 'AccessToken is invalid',
 'traceId': 'ac4dcee7c7964191ae0566d2a0376f4c',
 'code': 'auth.token.req'}

I only know what I tried. You could try assigning the 'did' value to 'wb._did' to see if that would work.

I did not get the output after running the 'wb.get_account() ' code statement, as shown on your post. However, I obtained a value instead from implementing the following: print(wb.get_account()). You might not need to implement the wb.refresh_login() once you run 'wb.get_account()' because what it does, I presume, is that it would cause your login status to refresh itself.

Alternatively, you could try replicating what I did here to see if you get a similar result: #367 (comment).

Do you have any existing orders or whatnot?

Thanks for helpful response. I'm using a jupyter notebook on vscode and have developed a bad habit of leaving out print statements while testing code because it does that for you automatically.

So if I do a print(wb.get_account_id()) I get None as a result. I'll try https://github.com/tedchou12/webull/issues/367#issuecomment-1366922522.

tradingwithme commented 1 year ago

I am using the Jupyter software from Julia.

osaidz commented 1 year ago

@tradingwithme I can confirm that api_login() works as I can see my alerts list but functions such as get_account() or get_detail() return errors.

{'msg': 'AccessToken is expire', 'traceId': 'e0aaf937bbef4e8bab8b14dfab451d34', 'code': 'auth.token.expire'}

{'msg': 'Failed to convert value of type \'java.lang.String\' to required type \'java.lang.Long\'; nested exception is java.lang.NumberFormatException: For input string: "None"', 'traceId': 'a7aa94bed56e4de697dba755c17adef4', 'code': '400', 'success': False}

SweetNaterTater commented 1 year ago

Hey all, I recently started playing with this package for trying to create an automated trading bot, for eventually having a bot that auto trades options by the barUpDn strategy on a paper account. I was also having issues with being able to login to my account, but you need to make sure you have selected "inspect". Then press network, so that you can begin recording any network activity that occurs when performing any actions on the web page. Then simply login to your account with your credentials, and entering your token(6 digit code for mfa). Then you should be able to find a fetched item that contains the token information.

Does this refer to the access_token value available on the Inspect window?

Good morning,

Yes it does, because I also was not getting the accesstoken for the webull credentials.json file. I was having issues because I was not successfully letting the inspect network capture my login attempt into webull itself. Once you have done this, you should be able to obtain the access token to bypass the mfa authentication and access either your main webull account or paper account. Hopefully this was helpful in some shape or form.

tradingwithme commented 1 year ago

@tradingwithme I can confirm that api_login() works as I can see my alerts list but functions such as get_account() or get_detail() return errors.

{'msg': 'AccessToken is expire', 'traceId': 'e0aaf937bbef4e8bab8b14dfab451d34', 'code': 'auth.token.expire'}

{'msg': 'Failed to convert value of type \'java.lang.String\' to required type \'java.lang.Long\'; nested exception is java.lang.NumberFormatException: For input string: "None"', 'traceId': 'a7aa94bed56e4de697dba755c17adef4', 'code': '400', 'success': False}

I see. What you did to get this result is correct; I was able to verify the output by implementing the code statements below: image image

You might need to consider restarting your cells and clearing all outputs on Jupyter before continuing. Also, you might need to refer to the accesstoken value to see if it had changed since the last time you obtained it.

Curious: How were you able to obtain the values for 'refreshtoken, 'tokenExpireTime', and 'uuid'?

osaidz commented 1 year ago

@tradingwithme

You might need to consider restarting your cells and clearing all outputs on Jupyter before continuing.

I think this was the issue. Just assigning the wb._access_token directly is sufficient to get quotes.

Yesterday, I was able to login once using wb.login(wb_user, wb_pass) it prints 'refreshtoken, 'tokenExpireTime', That's how I got the values for api_login(...).

wb.login() no longer works and results in "Fail to verify image verification code".