Closed gleivas closed 3 years ago
Glad you enjoy this repo. X-Net-Sync-Term
is an header that bet365 has introduced today. I'm currently working on a solution, but at the moment I'm still far from it.
That explains why I am getting a 'soccerapi.api.base.NoOddsError' @gleivas Can you please explain how did you hardcode the header and where to make it work? Thanks
At the moment no solution about X-Net-Sync-Term header involving on http requests have been found. However this kind of token last for 15 minutes, so a "manual" solution exists.
https://www.bet365.it/SportsBook.API/web?...
and grab X-Net-Sync-Term
from the Requests headers
.This solution is temporary and rather cumbersome.
Hello. The header is complete by some parts of the previous request of the archive "https://www.bet365.es/#/HO/". In the response I have find some parts, not all, of the heather. Y have send 6 diferents request and I can find some parts of the header.
X-Net-Sync-Term: scEQYA==.Cqo4ETl/ZTDdI4nQ+3LzY9Q5S1o71S3oJrSbQT62jtY= 'kDKZq','Q+3LzY9Q5S1o71S3oJrSbQT62jtY=' 'vpQPu','YnF9BnT'+'xCL==','Cqo4ETl/ZTD
X-Net-Sync-Term: rsUQYA==.Cg7YgMDiCRrO2i+nB/r/ldyYIQMYaxGnEexbtkV5r+0= 'kDKZq','+nB/r/ldyYIQMYaxGnEexbtkV5r+0=' 'vpQPu','xBxQ'+'ZyuBXg==','Cg7YgMDiCR'
X-Net-Sync-Term: WscQYA==.bXw1aWdrt+KfTXc1DOdy/nw6W2vdPCk3DD5nf5zvhd4= 'kDKZq','nw6W2vdPCk3DD5nf5zvhd4=' 'vpQPu','Swr3yT'+'AsH4==','bXw1aWdrt+KfTXc1D'
X-Net-Sync-Term: KckQYA==.diMugj7IjpV9QnYDDiX+XuYsjgwZ1Zy3OaBwhKQ4nOI=
'RentW','aqnWE','kDKZq','pV9QnYDDiX+XuYsjgwZ1Zy3OaBwhKQ4nOI='
'vpQPu','zWt3F'+'r9Je0==','diMug'
X-Net-Sync-Term: XMoQYA==.Mixo9GRa+EKOGhTseeoVBfH1DyeJ0stROSOvhTYsP/0=
'UakHE','RentW','aqnWE','kDKZq','VBfH1DyeJ0stROSOvhTYsP/0='
'vpQPu','VSd/kS'+'HQQq==','Mixo9GRa+EKOGhT'
X-Net-Sync-Term: TNUQYA==.myYQcI9/rbYHlI666HxVhT1x/TfdpyI2PnUHAlpVfkg= UakHE','RentW','aqnWE','kDKZq','rbYHlI666HxVhT1x/TfdpyI2PnUHAlpVfkg=' 'vpQPu','Q1RQag'+'u6XQ==','myYQ'
only 3 characters left at the begin and four characters in the midle of the hearder. I'm working at all.
Hello everyone! I found some useful info in this thread: https://github.com/Chiang97912/bet365.com/issues/9
Seems like the key is changes with a message in one of the two web sockets, about every 15 minutes - you can easily trace which one (the slower one). It also seems to be encrypted with the B365SimpleEncrypt util found in the BLOB. I have not yet figured out how to get the initial key, but I'm guessing it's either via some command sent in the socket or hidden somewhere as @rubenaarro suggests. I hope this helps, @S1M0N38.
Any news colleagues?
I've found a place where this header added to the request and presumptive generation code, for that 365team uses NSTLib and some bitwise shifts or some encryption algorithm. Maybe someone understands this area better
If you have any news, please let me know..
soccerapi 0.7.1 fix this issue. Now when you perform the first request to bet365 a browser instance created which grab the X-Net-Sync-Term
and then the browser is close. This is pretty quick and once this header is obtain request are perform with http (in the usual way). This header lasts for 15 min and it is automatically renew every 10 min for good measure. All these things happen under the hood.
I decide against the reversing the algo that generate X-Net-Sync-Term
because I'm lazy and unskilled. Moreover this approach should be more robust even if something in the encryption algo is changed. 🖖
I have been working with copy bots for a long time, mainly with bet365, I know that bet365 encrypts silver for this base64 token request. this request has existed since last year, but only in POST requests.
@victorratts13 Have you found a solution to get it? It seems that Puppeteer is not working anymore. I think they have been blocking the access to get the data needed. Do you know how to solve it?
Seems like I can fetch second part of the token and strange first part.
Maybe you have Idea how to fetch correct first part?
Fetched TOKEN - ~rt%C2%85%C2%88pll.eqlmXPKFoRf1R2uuzNV8x7m8+GnmvTV61fMqhaBRXo4= Correct TOKEN - OCEVYA==.eqlmXPKFoRf1R2uuzNV8x7m8+GnmvTV61fMqhaBRXo4=
How do you get to this point? You run this after page load? Can you share the code? The first part (before the dot) is a Base64 encoded timestamp... I'm not sure though if it's just the current timestamp in the browser or using the server time param in sports-configuration... it's probably just the former.
I just run the self-called func from host page script and log the 'q' variable. You can unminify script that included in host page layout and execute anywhere The end of func looks like on the screen below
func params is different Every time , so no reason to copy func
If you can find any way to solve the first part, please, let me know) I've tried base64, but that's not worked properly
I'm not able to work atm, but here's the code which reverts the first part of the token to int (which is a timestamp). Later on in the code the timestamp is compared to the server time + offset. That's how I know it is precisely a timestamp.
The code for the ConvertBase64ToNumber function they use to decode the first part of the token:
var t;
!function (e) {
function t(e) {
return e - 0
}
function n(t, n) {
var i = Math.pow(10, n);
return ((t + e.Epsilon) * i + .5 << 0) / i
}
function i(t) {
return (100 * (t + e.Epsilon) + .5 << 0) / 100
}
function r(t) {
return (100 * (t + e.Epsilon) << 0) / 100
}
function o(e) {
return ~~e
}
function s(e) {
return e - e % 1
}
function a(e) {
var t,
n,
i,
r,
o = atob(e),
s = o.length,
a = new Uint8Array(new ArrayBuffer(s));
for (t = 0; s > t; t++)
a[t] = o.charCodeAt(t);
for (n = new ArrayBuffer(a.length), i = new DataView(n), t = 0, r = a.length; r > t; t++)
i.setUint8(a.length - 1 - t, a[t]);
return i.getUint32(0)
}
e.Epsilon = Math.pow(2, -52),
e.StringToNumber = t,
e.ToNDecimalPlaces = n,
e.To2DecimalPlaces = i,
e.RoundDownTo2DecimalPlaces = r,
e.StringToInteger = o,
e.Truncate = s,
e.ConvertBase64ToNumber = a
}
Then just split the token and call ConvertBase64ToNumber with it ... I still don't know what the use to generate it in the first place though. ... And the function to encode it to Base64, for that matter...
Conver back Just get the current time and add 15minutes and convert to epoch time
function ConvertEpochToBase64(e)
{
var tt = new DataView(new ArrayBuffer(4));
tt.setUint32(0, e);
var str = '';
for(var i = 3; i >=0; i--)
str += String.fromCharCode(tt.getUint8(i));
return btoa(str);
};
console.log(ConvertEpochToBase64(1611997496));
So maybe it's a decision
@vcherniatyn can you get the token without using webdrivers?
I think yes, but have several big NO... Seems like we need to send our token through web socket and something else... Also first-time token changed after several requests to API, it's about 10 seconds after page loading...
The question now is how to get the header of this request by using puppeteer or other method. If you check on network and XHR methods, the X-NET-SYNC-TERM is already there. I use to intercept this information using puppeteer but the Bet365 started to block it. You can get any information using Chrome, Firefox or Edge, but the question now is how to automate it again. Any suggestions in order to get the Puppeteer working again or using another method?
I used puppeteer/selenium+chrome(headless or not) + individual socks5 proxy and currently seems like it was automated..
Can you share the code with us? I’m getting problems to run it now. It’s just stopped on the first screen and keeps loading forever. The front-end is not retrieving the data to populate the components.
I use puppeteer with this args "--disable-blink-features=AutomationControlled", "--disable-gpu-rasterization", "--ignore-certificate-errors", "--disable-infobars", "--force-webrtc-ip-handling-policy", "--lang=en-GB"
page.EvaluateExpressionOnNewDocumentAsync("Object.defineProperty(navigator, 'webdriver', { get: () => undefined })").Wait(); page.SetJavaScriptEnabledAsync(true).Wait(); page.SetUserAgentAsync("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36").Wait();
code example:
Request request = null;
using (var page = browser.NewPageAsync().Result)
{
page.EvaluateExpressionOnNewDocumentAsync("Object.defineProperty(navigator, 'webdriver', { get: () => undefined })").Wait();
page.SetJavaScriptEnabledAsync(true).Wait();
page.SetUserAgentAsync("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36").Wait();
page.GoToAsync("https://www.bet365.com/#/AS/B2/").Wait();
request = page.WaitForRequestAsync((x) => x.Url.Contains("SportsBook.API/web?lid=1&zid=9&pd=%23AE%23B2%23K%5E%23&cid=195&cgid=1"), new WaitForOptions() { Timeout = 20_000 }).Result;
if (!string.IsNullOrWhiteSpace(request?.Headers["X-Net-Sync-Term"]))
{
_tokenHolder.SyncToken = request.Headers["X-Net-Sync-Term"];
_tokenHolder.UpdateTime = DateTime.Now;
_tokenHolder.Cookies = page.GetCookiesAsync().Result.ToList();
_logger.LogInformation("New token" + _tokenHolder.SyncToken);
}
else
{
_logger.LogInformation("New token is empty");
}
page.CloseAsync().Wait();
}
My algo- take the token and cookies through the puppeteer once at 10min and then use simple HTTP loader call web API to get data
My algo- take the token and cookies through the puppeteer once at 10min and then use simple HTTP loader call web API to get data
Oh ok, this is the common solution most people use, intercept request and extract the X-Net-Sync-Term
header. I thought you could extract the NST token without a browser or something that emulates it.
soccerapi 0.7.1 fix this issue. Now when you perform the first request to bet365 a browser instance created which grab the
X-Net-Sync-Term
and then the browser is close. This is pretty quick and once this header is obtain request are perform with http (in the usual way). This header lasts for 15 min and it is automatically renew every 10 min for good measure. All these things happen under the hood.I decide against the reversing the algo that generate
X-Net-Sync-Term
because I'm lazy and unskilled. Moreover this approach should be more robust even if something in the encryption algo is changed. 🖖
I have extracted this simple code from your soccerapi but it doesnt work properli. I only need the X-Net-Sync-Term to after use httprequest in Visual Basic.
async def Resultado(): browser = launch( headless=True, args=['--disable-blink-features=AutomationControlled'], ) page = (await browser.pages())[0] await page.setUserAgent( 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36' '(KHTML, like Gecko) Chrome/88.0.4324.96 Safari/537.36' ) await page.goto("https://www.bet365.es/#/AC/B1/C1/D13/E52115687/F2/") request = await page.waitForRequest(lambda r: 'SportsBook' in r.url) await browser.close() return request.headers['X-Net-Sync-Term']
xnet = Resultado() print (xnet)
Function Resultado returns: <coroutine object Resultado at 0x00000217A71759C0>. Not work properli but I don't no why. Can you help me?
Seems like I can fetch second part of the token and strange first part.
Maybe you have Idea how to fetch correct first part?
Fetched TOKEN - ~rt%C2%85%C2%88pll.eqlmXPKFoRf1R2uuzNV8x7m8+GnmvTV61fMqhaBRXo4= Correct
Seems like I can fetch second part of the token and strange first part.
Maybe you have Idea how to fetch correct first part?
Fetched TOKEN - ~rt%C2%85%C2%88pll.eqlmXPKFoRf1R2uuzNV8x7m8+GnmvTV61fMqhaBRXo4= Correct TOKEN - OCEVYA==.eqlmXPKFoRf1R2uuzNV8x7m8+GnmvTV61fMqhaBRXo4=
Good morning, I rewrite the text because the Spanish spell checker has changed several words: I wanted to say this: Hi. I have been intercepting many tokens manually. In OCEVYA, the V character changes sequentially every day. YA is probably month and year. When it changes, I'll know what it is, but for the moment it's always just YA. OCE is probably time. I can decode this part easily, but I don't know how to automatically capture the obtained token. If you teach me, with a brute force attack I can easily decipher everything and I will share.
Seems like I can fetch second part of the token and strange first part.
Maybe you have Idea how to fetch correct first part?
Fetched TOKEN - ~rt%C2%85%C2%88pll.eqlmXPKFoRf1R2uuzNV8x7m8+GnmvTV61fMqhaBRXo4= Correct TOKEN - OCEVYA==.eqlmXPKFoRf1R2uuzNV8x7m8+GnmvTV61fMqhaBRXo4=
Good morning, I rewrite the text because the Spanish spell checker has changed several words: I wanted to say this: Hi. I have been intercepting many tokens manually. In OCEVYA, the V character changes sequentially every day. YA is probably month and year. When it changes, I'll know what it is, but for the moment it's always just YA. OCE is probably time. I can decode this part easily, but I don't know how to automatically capture the fetched token. If you teach me, with a brute force attack I can easily decipher everything and I will share.
So now first part of token is not a problem.. It's just current time + 5 or 15 minute(timestamp) converted by this func
Conver back Just get the current time and add 15minutes and convert to epoch time
function ConvertEpochToBase64(e) { var tt = new DataView(new ArrayBuffer(4)); tt.setUint32(0, e); var str = ''; for(var i = 3; i >=0; i--) str += String.fromCharCode(tt.getUint8(i)); return btoa(str); }; console.log(ConvertEpochToBase64(1611997496));
So maybe it's a decision
But then we need to send token in websocket Then we need to generate second token...
@vcherniatyn why are you using websockets? If you can successfully get a token from the homepage (say by watching the headers on the api call) it lasts for 15 minutes. The websockets are just used for inplay data and will refresh the token when needed.
I investigate the requests from b365 and seems like the token will be activated through WebSocket... It's the first place when we send token to b365 If you just get token - it doesn't work properly
But that's just my guess .. I could be wrong
But then we need to send token in websocket Then we need to generate second token...
The websocket works to detect the dinamic odds. With the first token you can get the odds ussing request in python or httprequest in visual basic. But you need request if do you want know the if odds are changed. It's works properly. I have tried it. I request for odds every 2 minutes but I don't know how to connect to webshocket. Do you need inplay odds?
it lasts for 15 minutes.
Now behaviour is different...
I found out this myself.. after wondering why the bot was not working for the last couple of days... Is it reliable to fetch the token each 15min from the mainpage?
I found out this myself.. after wondering why the bot was not working for the last couple of days... Is it reliable to fetch the token each 15min from the mainpage?
Yes, it's reliable but the cuestion is if you are able to automate get this header. I can´t automate this. When I get manually the header y can get odds by request, in python and visual basic.
Preliminary way(not tested completely) Load https://www.bet365.com/#/HO/ - take script by Regex "(?'script'(function(){var\sa\=.+\=r\;}())\;})()\;)" Load https://www.bet365.com/defaultapi/sports-configuration - take serverTime, add 300
Need to get 'q' variable from the script can be returned here q.split('.')[1] - the second part of your token For get first part just use this func
function ConvertEpochToBase64(e)
{
var tt = new DataView(new ArrayBuffer(4));
tt.setUint32(0, e);
var str = '';
for(var i = 3; i >=0; i--)
str += String.fromCharCode(tt.getUint8(i));
return btoa(str);
};
ConvertEpochToBase64(serverTime + 300);
If you make these steps you can get the token
Why preliminary way? Because I cannot execute JS script through C# with code like this, executors can't parse the script, but maybe if you use JS - it would be simpler. But If I load the b365 in a browser and make all these steps manually - I get the same token as in request header
I hope this helps everyone!
PS You can also replace next terms in minified script ',r=new f();' -> ';return q.split('.')[1];' FIRST '(' symbol to 'var mainFunc = ' '}());})();' -> '}; return secondFunc(); }; return mainFunc();' '}}}(),function(){var bi=' -> '}}}();var secondFunc = function(){var bi=' So then for getting the second part just execute resulted script, manually I tried that on the js.do and it worked
Good night. I also cannot access the bet365 website after using the puppeteer. Is this temporary or permanent? How do I get the requests that the bet365 page makes and its headers?
I thank you for your help. =)
For get first part just use this func function ConvertEpochToBase64(e)
This doesnt work propeli:
should return: R4MgYA
You missed the final btoa conversion
For get first part just use this func function ConvertEpochToBase64(e)
This doesnt work propeli:
should return: R4MgYA
Thanks, now I can get the first part. I have traduced Java code to VisualBasic code and the first part works propeli.
Preliminary way(not tested completely) Load https://www.bet365.com/#/HO/ - take script by Regex "(?'script'(function(){var\sa=.+=r;}());})();)" Load https://www.bet365.com/defaultapi/sports-configuration - take serverTime, add 300
Need to get 'q' variable from the script can be returned here q.split('.')[1] - the second part of your token For get first part just use this func
function ConvertEpochToBase64(e) { var tt = new DataView(new ArrayBuffer(4)); tt.setUint32(0, e); var str = ''; for(var i = 3; i >=0; i--) str += String.fromCharCode(tt.getUint8(i)); return btoa(str); }; ConvertEpochToBase64(serverTime + 300);
If you make these steps you can get the token
Why preliminary way? Because I cannot execute JS script through C# with code like this, executors can't parse the script, but maybe if you use JS - it would be simpler. But If I load the b365 in a browser and make all these steps manually - I get the same token as in request header
I hope this helps everyone!
PS You can also replace next terms in minified script ',r=new f();' -> ';return q.split('.')[1];' FIRST '(' symbol to 'var mainFunc = ' '}());})();' -> '}; return secondFunc(); }; return mainFunc();' '}}}(),function(){var bi=' -> '}}}();var secondFunc = function(){var bi=' So then for getting the second part just execute resulted script, manually I tried that on the js.do and it worked
Can you generate the second part? I have seen then code changes usually. The function to calculate "q" changes. How do you solve it?
Hey guys, the solution with puppeteer worked for me using proxy. If you are willing to pay a little bit to it works I suggest luminati proxy, is very easy to use it with puppeteer, I changed the _get_token
function to:
async def _get_token(self):
browser = await launch(
headless=True,
args=['--disable-blink-features=AutomationControlled', '--proxy-server=zproxy.lum-superproxy.io:22225'],
)
page = (await browser.pages())[0]
await page.setUserAgent(
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36'
'(KHTML, like Gecko) Chrome/88.0.4324.96 Safari/537.36'
)
await page.authenticate({
'username': 'My Luminati Username',
'password': 'My Luminati Password'
})
await page.goto("https://www.bet365.com")
request = await page.waitForRequest(lambda r: 'SportsBook' in r.url)
await browser.close()
return request.headers['x-net-sync-term']
Although I'm still looking for a better way to do it without browser or proxy this is a very good temporary solution
Hey guys, the solution with puppeteer worked for me using proxy. If you are willing to pay a little bit to it works I suggest luminati proxy, is very easy to use it with puppeteer, I changed the
_get_token
function to:async def _get_token(self): browser = await launch( headless=True, args=['--disable-blink-features=AutomationControlled', '--proxy-server=zproxy.lum-superproxy.io:22225'], ) page = (await browser.pages())[0] await page.setUserAgent( 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36' '(KHTML, like Gecko) Chrome/88.0.4324.96 Safari/537.36' ) await page.authenticate({ 'username': 'My Luminati Username', 'password': 'My Luminati Password' }) await page.goto("https://www.bet365.com") request = await page.waitForRequest(lambda r: 'SportsBook' in r.url) await browser.close() return request.headers['x-net-sync-term']
Although I'm still looking for a better way to do it without browser or proxy this is a very good temporary solution
Which subscription model do you use with luminati? How much does it cost/month?
Hi guys,
I just try to decrypt the D_
token following some of the things that have been exposed in this issue. The two parts of the token generated seems correct, but when I send to the websocket it does not complete the handshake (the SPTBK_D23
message is not received in wss://pshudws.365lpodds.com
)
session_id = await get_session_id() #request https://www.bet365.com/defaultapi/sports-configuration
server_time = await get_server_time(session_id) #request https://www.bet365.com/defaultapi/sports-configuration
nst_token = await get_first_part_token(server_time) + "." + await get_second_part_token()
"#\x03P\x01__time,S_{},D_{}\x00".format(session_id, nst_token)
on get_first_part_token(server_time)
just use function
function ConvertEpochToBase64(e)
{
var tt = new DataView(new ArrayBuffer(4));
tt.setUint32(0, e);
var str = '';
for(var i = 3; i >=0; i--)
str += String.fromCharCode(tt.getUint8(i));
return btoa(str);
};
ConvertEpochToBase64(serverTime + 300);
on get_second_part_token()
request https://www.bet365.com/#/HO/
and returns q.split('.')[1]
I just upload the code I am using: https://github.com/marc6691/bet365-websocket/blob/master/bet365.py
How long is the token valid for? How many request? When I try fetching it each 10s I still get a new token it seems. edit: now got banned... should have realised that I should not test updating that often haha..
Does anyone want to cooperate with bet365's automatic profit project?
Does BET365 run this project themselves? If so, can you please share the link?
Does anyone want to cooperate with bet365's automatic profit project?
Does BET365 run this project themselves? If so, can you please share the link?
Hi, I think everibody here is in a bet365 project. If you want talking about, my telegram is @ComadrejaHipertrofida. Greetings
bet365 now blocking puppeteer. Who trying bypass this?
@caffreysbb use this docker to generate the Bet365 X-Net-Sync-Term header https://github.com/S1M0N38/soccerapi-server
On April 10th 2021 this solution works
Hello guys, a bit late, but I found this interesting discussion, where @incapdns exposes a solution that could let us get rid of the browser emulation thing. The only downside is that it's written in js (and it can't be otherwise since it executes the js code provided by bet365). On the other side we could get much less overhead. In this scenario I could see two approaches:
node script.js
just to get the token via stdoutHello guys, I need your help, is their's a way to get the X-Net-Sync-Term using PHP (guzzle?) I also studied the reverse engineer but no luck, maybe you can help me. Thanks
Reverse the generation process of X-Net-Sync-Term it's quite tricky but doable in javascript. For an easy solution for getting a valid X-Net-Sync-Term try https://github.com/S1M0N38/soccerapi-server . It's a docker container that you have to run in back ground to which you can ask X-Net-Sync-Term by a simple http request. New X-Net-Sync-Terms are generated every 10 minutes.
Hello, does anyone here have a working method to extract that token in 2022? I think they've changed it so that same tokens cant be used for different requests.
Hello @S1M0N38, some weeks ago I tested the soccerapi for Bet365 and it worked fine. Today I tried again but the requests made to get the odds had no response resulting in
NoOddsError
. I looked in my browser how Bet365 does that request and I noticed that in the browser they had a different header calledX-Net-Sync-Term
, when I added this header hard coded with the same value as my browser(ImoQYA == .8CwDX5TgJEH5wFCEu1sxh4h1RjAoyXeM6gjjlcGuetw =
) the api worked again.Do you know how this header work? I didn't found much about it. I don't know either how long this hard coded header will work.
Thanks a lot for your help, your repository is awesome!