Chiang97912 / bet365.com

The scraper of bet365.com
MIT License
113 stars 47 forks source link

Unable to authenticate #9

Open otac0074 opened 4 years ago

otac0074 commented 4 years ago

i think this problem

bet365 update..

스크린샷 2020-03-03 오전 12 34 30

P + session_id + '????'

added

georgait commented 3 years ago

Finally can authenticate or not (without selenium 4)?

georgait commented 3 years ago

https://stackoverflow.com/questions/54617919/how-to-decrypt-bet365-socket-data/59392392#59392392

can you contact with me mate ?

Hello I am greek too. Do you have any progress on this?

HMaker commented 3 years ago

@victorratts13 Is that data from bet365's websocket connection?

HMaker commented 3 years ago

@tpikachu Haha. I think bet365 will change it soon, you wrote a script that takes the parameters and generates the token by its own? or you are evaluating the obfuscated code?

georgait commented 3 years ago

https://www.programmersought.com/article/39844800907/

It may helps someone to get the token. This token is required for placing bet too.

georgait commented 3 years ago

I didn't realize that... So as you mentioned above with selenium it is possible to make post request after connecting to the websocket server and getting the nst token. For placing bets required nst token not the uid

HMaker commented 3 years ago

I didn't realize that... So as you mentioned above with selenium it is possible to make post request after connecting to the websocket server and getting the nst token. For placing bets required nst token not the uid

What is that UID?

georgait commented 3 years ago

wss://premws-pt1.365lpodds.com/zap/?uid=5861835950535124

HMaker commented 3 years ago

wss://premws-pt1.365lpodds.com/zap/?uid=5861835950535124

Oh yes, that UID is the decimal part of Math.random(), it is not used for authentication.

georgait commented 3 years ago

X-Net-Sync-Term header requires nst token that appears in WS messages: commandnsts0uPXw==.ywWk3mMsexd0PAPFwPQPogXdD7JYuRaAiIJiMoe9rHc=SPTBK.

Using selenium 4 Dev Tools Network can get WS messages

HMaker commented 3 years ago

X-Net-Sync-Term header requires nst token that appears in WS messages: �command�nst�s0uPXw==.ywWk3mMsexd0PAPFwPQPogXdD7JYuRaAiIJiMoe9rHc=�SPTBK.

Using selenium 4 Dev Tools Network can get WS messages

Yes. I think NST is a shorthand for Net Sync Term.

iflp commented 3 years ago

@HMaker: Nice work getting there. A question: Which part of the NST token is the b64 encoded expiry time?

iflp commented 3 years ago

@HMaker Thanks for the thorough explanation! Probably easier to go through the whole auth flow again using an arbitrarily short expiry.

286844626 commented 3 years ago

@tpikachu Well I am able to use selenium driver to generate the token in headless mode, it's just needed to setup the driver to be undetectable. Also the obfuscated code includes the checking of the current environment, they try to figure out if it is being ran in a browser but not controlled by a bot, if it is controlled by a bot the resulting NST token will be invalid. It's needed to clean the obfuscated code to remove the browser environment checking.

https://github.com/286844626/bet3656-

iflp commented 3 years ago

@HMaker Not in my experience.. the PSTK expiry can be set to 12 hours in your settings, and I don't think the NST token has an expiry per se, once you successfully authenticate with a socket and get a response you should be good to use it for it's lifetime.

marc6691 commented 3 years ago

After open the websocket and send every second the message \x16\x006V' + EVENT_ID + '\x01' the websocket stop receiving messages about 40 seconds. Anybody has the same issue?

HMaker commented 3 years ago

What about creating the discord server? So that we can keep the secure discussion. So many observers... If anyone is going to create this, please ping me first

Yes, I was thinking about that too, we should make a discord server.

HMaker commented 3 years ago

After open the websocket and send every second the message \x16\x006V' + EVENT_ID + '\x01' the websocket stop receiving messages about 40 seconds. Anybody has the same issue?

You are sending repeated subscription messages? Messages starting with the byte 22 (\x16) means subscription. Generally bet365's frontend does batch subscription by joining several topics separated by commas.

iflp commented 3 years ago

I've created a discord for further discussion. Please join and indicate your github username: https://discord.gg/NsSRzJk

dochenaj commented 3 years ago

After open the websocket and send every second the message \x16\x006V' + EVENT_ID + '\x01' the websocket stop receiving messages about 40 seconds. Anybody has the same issue?

You are sending repeated subscription messages? Messages starting with the byte 22 (\x16) means subscription. Generally bet365's frontend does batch subscription by joining several topics separated by commas.

I was wondering if you can subscribe to multiple events at once or you'll need to unsubscribe before subscribing to the another.

\x16 for subscribe and \x17

marc6691 commented 3 years ago

After open the websocket and send every second the message \x16\x006V' + EVENT_ID + '\x01' the websocket stop receiving messages about 40 seconds. Anybody has the same issue?

You are sending repeated subscription messages? Messages starting with the byte 22 (\x16) means subscription. Generally bet365's frontend does batch subscription by joining several topics separated by commas.

I sent topics separated by commas and it closes in the same way. I think bet365 websocket has some mechanism to detect if I flood sending messages..?

After open the websocket and send every second the message \x16\x006V' + EVENT_ID + '\x01' the websocket stop receiving messages about 40 seconds. Anybody has the same issue?

You are sending repeated subscription messages? Messages starting with the byte 22 (\x16) means subscription. Generally bet365's frontend does batch subscription by joining several topics separated by commas.

I was wondering if you can subscribe to multiple events at once or you'll need to unsubscribe before subscribing to the another.

\x16 for subscribe and \x17

I haven't seen any unsubscribe messages at the moment..

dochenaj commented 3 years ago

@marc6691 If you were to visit multiple events on the browser, search the websocket stream for \x17, and you'll see that each time you move to a new event the old one was unsubscribed.

marc6691 commented 3 years ago

I share my code, but it stops receiving messages after a while. https://github.com/marc6691/bet365-websocket/blob/master/bet365.py

dochenaj commented 3 years ago

@marc6691 you need to connect to the other socket too. If you observe the websocket in Chrome Dev Tools, you'll notice it connects to two different sockets and sends the handshake code to both of them. I will share that code with you when I get to my computer.

dochenaj commented 3 years ago

I share my code, but it stops receiving messages after a while. https://github.com/marc6691/bet365-websocket/blob/master/bet365.py

Look at this and you'll figure it out. https://github.com/JoeBili/bet365-websocket-crawler/blob/master/bet365_websocket_spider.py

HMaker commented 3 years ago

Well bet365 changed the token code and broke its evaluation in nodejs, but it is still possible to generate the token with nodejs.

dochenaj commented 3 years ago

I discovered that today, and have been trying to figure out how to extract the new code. Strangely the old code pattern can still be calculated but it is doesn't get the websocket authorized. Have you solved the new issue?

HMaker commented 3 years ago

I discovered that today, and have been trying to figure out how to extract the new code. Strangely the old code pattern can still be calculated but it is doesn't get the websocket authorized. Have you solved the new issue?

I am able to do it. They changed the code a little but it is still possible to do the reverse engineering, extract only the needed token code and evaluate it with NodeJS.

4501771 commented 3 years ago

Hello, anyone managed this or any other bet365 live game scraper to get working? thanks in advance

velafrys commented 3 years ago

has anyone faced the problem of using chromedriver & selenium in the last couple of days? Infinity white loading screen during opening bet365.com i mean.

b0dnar commented 3 years ago

I have similar problem. @velafrys do you decided problem?

ihor2005 commented 3 years ago

Hi, have no problem to get in-live data via socket, but i have problem with decoding PRELIVE odds, they encode odds, like "OD=5(>;|"

can anybody help, please?

286844626 commented 3 years ago

Hi, have no problem to get in-live data via socket, but i have problem with decoding PRELIVE odds, they encode odds, like "OD=5(>;|"

can anybody help, please?

hi I also encountered the same problem, did you solve it?

ihor2005 commented 3 years ago

Hello, no

let keep us in touch :)

------ Original Message ------ From: "hilong" @.> To: "Chiang97912/bet365.com" @.> Cc: "ihor2005" @.>; "Comment" @.> Sent: 6/9/2021 10:49:30 AM Subject: Re: [Chiang97912/bet365.com] Unable to authenticate (#9)

Hi, have no problem to get in-live data via socket, but i have problem with decoding PRELIVE odds, they encode odds, like "OD=5(>;|"

can anybody help, please?

hi I also encountered the same problem, did you solve it?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Chiang97912/bet365.com/issues/9#issuecomment-857511781, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUJJUXCK7XRHDXVPBBMHQELTR4TJVANCNFSM4K7WZJ5A.

xino1010 commented 3 years ago

@ihor2005 would you mind to give some tips to get in-live via socket? Not all the details only the steps. Thank you

ihor2005 commented 3 years ago

my skype is cirle__50

------ Original Message ------ From: "Daniel Andújar" @.> To: "Chiang97912/bet365.com" @.> Cc: "ihor2005" @.>; "Mention" @.> Sent: 6/23/2021 4:58:24 PM Subject: Re: [Chiang97912/bet365.com] Unable to authenticate (#9)

@.*** https://github.com/ihor2005 would you mind to give some tips to get in-live via socket? Not all the details only the steps. Thank you

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Chiang97912/bet365.com/issues/9#issuecomment-866912398, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUJJUXGPQYZPOZBYWHXL7E3TUHZBBANCNFSM4K7WZJ5A.

-- This email has been checked for viruses by AVG. https://www.avg.com

dochenaj commented 3 years ago

Hi, have no problem to get in-live data via socket, but i have problem with decoding PRELIVE odds, they encode odds, like "OD=5(>;|"

can anybody help, please?

You figured this out yet?