Chiang97912 / bet365.com

The scraper of bet365.com
MIT License
112 stars 48 forks source link

X-Net-Sync-Term Header Fixed !! #18

Open incapdns opened 3 years ago

incapdns commented 3 years ago

I would like to share the method used to capture the "X-Net-Sync-Term" header of the HTTP request. Currently the Bet365 index page does not have "boot.nsu (str1, str2)", they have changed the way the website is displayed.

Encrypted initial token: image

Method used to capture: [Javascript]

let generate = (param) => {
    let arr;

    if(typeof param == 'string')
        arr = eval('[' + param.split('var a=[')[1].split('];')[0] + ']');
    else if(Array.isArray(param))
        arr = param
    else
        return false

    let n = arr[397];

    let q = arr[158] + '.' + arr[375] + arr[182] + arr[294];

    let parts = q.split('.')

    let compressed = decodeURIComponent(parts[0])

    let decompress = function(o) {
        o = decodeURIComponent(o);

        let s = +atob(n) * -1;

        s = s % 64;

        let r = '';

        for (var t = 0; t < o.length; t++) {
            var u = o.charCodeAt(t)
              , v = String.fromCharCode((u + s) % 256);

            r += v;
        }

        return r;
    }

    return decompress(compressed) + '.' + parts[1];
}

The parameter can be the entire HTML returned by the Bet365 website, or an array, if you choose to pass an array, the array must be the one shown in the image.

Ps: The above function will return the token (encrypted) of the page, which in this case is: idS4bd==.KavFYZJwIEwXZ1nkv6zdXI/FqYJYQiHNxQDhvhnLP1E= with this token, you will need to decode

Method used to decode or encode (encode only to explication) function:

class Bet365 {
    mapLen = 64;

    charMap = [["A", "d"], ["B", "e"], ["C", "f"], ["D", "g"], ["E", "h"],
        ["F", "i"], ["G", "j"], ["H", "k"], ["I", "l"], ["J", "m"], ["K", "n"], ["L", "o"],
        ["M", "p"], ["N", "q"], ["O", "r"], ["P", "s"], ["Q", "t"], ["R", "u"], ["S", "v"],
        ["T", "w"], ["U", "x"], ["V", "y"], ["W", "z"], ["X", "a"], ["Y", "b"], ["Z", "c"],
        ["a", "Q"], ["b", "R"], ["c", "S"], ["d", "T"], ["e", "U"], ["f", "V"], ["g", "W"],
        ["h", "X"], ["i", "Y"], ["j", "Z"], ["k", "A"], ["l", "B"], ["m", "C"], ["n", "D"],
        ["o", "E"], ["p", "F"], ["q", "0"], ["r", "1"], ["s", "2"], ["t", "3"], ["u", "4"],
        ["v", "5"], ["w", "6"], ["x", "7"], ["y", "8"], ["z", "9"], ["0", "G"], ["1", "H"],
        ["2", "I"], ["3", "J"], ["4", "K"], ["5", "L"], ["6", "M"], ["7", "N"], ["8", "O"],
        ["9", "P"], ["\n", ":|~"], ["\r", ""]];

    encrypt( txt ){
        let result = "";

        for( let i = 0; i < txt.length; i++ ){
            let c = txt.substr(i, 1)

            for( let j = 0; j < this.mapLen; j++ ){
                if( c == this.charMap[j][0] ){
                    c = this.charMap[j][1];
                    break;
                }
            }
            result += c;
        }

        return result;
    }

    decrypt( txt ){
        let result = "";

        for(let i = 0; i < txt.length; i++){
            let c = txt.substr(i, 1)

            for( let j = 0; j < this.mapLen; j++ ){
                if( ":" == c && ":|~" == substr( txt, i, 3 ) ){
                    c = "\n";
                    i+=2;
                    break;
                }
                if( c == this.charMap[j][1] ){
                    c = this.charMap[j][0];
                    break;
                }
            }
            result += c;
        }

        return result;
    }
}

let tool = new Bet365

Decrypted token: FAcuYA==.4XSpij3T2oThjrKHSwWAh2/pNi3iaF17UanESEK59ro=

Websocket: image

After sending the token in the initial request, which looks something like this: "_time,S{PSTK},D_{DecodedToken}"

In this example: {PSTK} = 62E1FE57F3BF449681C3E380BE1D986A000003 {DecodedToken} = FAcuYA==.4XSpij3T2oThjrKHSwWAh2/pNi3iaF17UanESEK59ro=

Importante note:

  1. Sometimes you will receive responses from the WebSocket containing the message: "_SPTBK_D" and "==.", When you receive this, your requests should be updated to tool.decrypt (received token), example in the image above: tool.decrypt('RtA4bd==.qBNfyjxIInIdGMbjt/2ayaTyB8IoL8XKAzJU5uf4D6t=')

  2. You should always preserve the characters sent on the Websocket, for this reason it is recommended that you intercept messages sent by the websocket through Base64, an example: The initial request is: atob("IwNQAV9fdGltZSxTX1BTVEssRF9ERUNPREVEX1RPS0VOAA==")

image

With this I have the intact and safe initial message, and only need to execute:

let initReq = atob("IwNQAV9fdGltZSxTX1BTVEssRF9ERUNPREVEX1RPS0VOAA==")

initReq = initReq.replace('PSTK', PSTK)
initReq = initReq.replace('DECODED_TOKEN', DecodedToken)
incapdns commented 3 years ago

Let me know if it worked, if you find any difficulty checking / testing please comment, I will be waiting for updates.

I'm sorry for my bad english, i'm using google translator.

incapdns commented 3 years ago

Update: The number of elements in the array was updated in 15 minutes, so I deduced that the index page was dynamically generated by the backend. I'll have to think of another method :( Or use puppeter

HMaker commented 3 years ago

Good, how you did the reverse engineering of that obfuscated code?

incapdns commented 3 years ago

1: First step: image

1: VM:22:3
Code: l = o.customRequestHeaders;

2: Break point in "customRequestHeaders". Note: o.customRequestHeaders is set in function "n.prototype.load = function(e, o)" where n is callback to xmlhttprequest image

3: Get the caller of n.prototype.load in call stack Note: X-Net-Sync-Term is return of function t(), see: g = t && t() image

4: Get function t content Note: t fuction is n(){return m;}, so the content of X-Net-Sync-Term is equal to "return m;" image

5: Get the parent function that creates "m" variable: Code:

(function() {
    var C = function(c, d) {
        return b(c - '0x151', d);
    }
      , d = {};
    d[C('0x154')] = function(o, p) {
        return o < p;
    }
    ,
    d[C('0x15b')] = function(o, p) {
        return o + p;
    }
    ,
    d[C('0x181')] = C('0x158'),
    d[C('0x176')] = C('0x152'),
    d[C('0x175')] = function(o, p) {
        return o in p;
    }
    ,
    d[C('0x17e')] = C('0x17f'),
    d[C('0x18b')] = function(o, p) {
        return o % p;
    }
    ,
    d[C('0x167')] = function(o, p) {
        return o * p;
    }
    ,
    d[C('0x18a')] = function(o, p) {
        return o(p);
    }
    ,
    d[C('0x160')] = C('0x172'),
    d[C('0x166')] = C('0x15e'),
    d[C('0x195')] = C('0x163'),
    d[C('0x182')] = C('0x19a'),
    d[C('0x169')] = C('0x198'),
    d[C('0x177')] = C('0x162'),
    d[C('0x16c')] = C('0x188'),
    d[C('0x17d')] = C('0x15a'),
    d[C('0x153')] = C('0x15c'),
    d[C('0x18c')] = C('0x197'),
    d[C('0x16b')] = C('0x168'),
    d[C('0x186')] = C('0x165'),
    d[C('0x180')] = C('0x192'),
    d[C('0x15f')] = C('0x170'),
    d[C('0x15d')] = C('0x185'),
    d[C('0x17c')] = C('0x178'),
    d[C('0x16e')] = C('0x18e'),
    d[C('0x174')] = C('0x171'),
    d[C('0x156')] = C('0x16a'),
    d[C('0x155')] = C('0x184'),
    d[C('0x17b')] = function(o) {
        return o();
    }
    ;
    var e = d
      , f = ''
      , g = [e[C('0x160')], e[C('0x166')], e[C('0x195')], C('0x193'), e[C('0x182')], e[C('0x169')], e[C('0x177')], e[C('0x16c')], e[C('0x17d')], C('0x16f'), e[C('0x153')], e[C('0x18c')], e[C('0x16b')], e[C('0x186')], e[C('0x180')], e[C('0x15f')], e[C('0x15d')], C('0x17a'), C('0x194'), e[C('0x17c')], e[C('0x16e')], e[C('0x174')], C('0x191'), C('0x157'), e[C('0x156')], C('0x183'), e[C('0x155')]]
      , h = function() {
        var D = function(c, d) {
            return C(c - '0x378', d);
        }
          , o = '';
        for (var p = 0x0, q = g; e[D('0x4cc')](p, q[D('0x4e5')]); p++) {
            var r = q[p]
              , s = window[e[D('0x4d3')](e[D('0x4d3')](e[D('0x4f9')], r), D('0x511'))]
              , t = s[D('0x501')][e[D('0x4ee')]];
            if (!t)
                continue;
            var u = Object[D('0x507')](Object[D('0x4c9')](t));
            for (var v in u) {
                var w = u[v]
                  , x = Object[D('0x4c9')](t[w])
                  , y = Object[D('0x507')](x);
                for (var z in y) {
                    var A = y[z];
                    if (e[D('0x4ed')](A, Object))
                        continue;
                    if (x[A] && x[A]()) {
                        var B = x[A]();
                        o = B[0x0],
                        f = B[0x1];
                        break;
                    }
                }
                if (o)
                    break;
            }
            delete s[D('0x501')][e[D('0x4ee')]];
            if (o)
                break;
        }
        return o;
    }
      , i = function(o) {
        var E = function(c, d) {
            return C(c - -'0x40', d);
        }
          , p = e[E('0x13e')][E('0x150')]('|')
          , q = 0x0;
        while (!![]) {
            switch (p[q++]) {
            case '0':
                s = e[E('0x14b')](s, 0x40);
                continue;
            case '1':
                var r = '';
                continue;
            case '2':
                var s = e[E('0x127')](+e[E('0x14a')](atob, f), -0x1);
                continue;
            case '3':
                o = decodeURIComponent(o);
                continue;
            case '4':
                for (var t = 0x0; t < o[E('0x12d')]; t++) {
                    var u = o[E('0x14d')](t)
                      , v = String[E('0x124')](e[E('0x14b')](e[E('0x11b')](u, s), 0x100));
                    r += v;
                }
                continue;
            case '5':
                return r;
            }
            break;
        }
    }
      , j = e[C('0x17b')](h)
      , k = j[C('0x190')]('.')
      , l = e[C('0x18a')](i, k[0x0])
      , m = [l, '.', k[0x1]][C('0x196')]('')
      , n = function() {
        return m;
    };
}());

Final consideration: You can see that the function that generates the token is being returned via ajax at this URL: https://www.bet365.com/Api/1/Blob?33,sports,gen5base/560/|WebConsoleLib/323/S|SitePreferencesLib/31/|NavLib/73/S|ScrollerLib/42/S|HeaderModule/427/SL|PodLoaderModule/220/S|GridLoaderLib/75/|WebConsoleModule/725/SL through window.eval.

You also notice that variable names like "n", function D, E, C are random, and the contents of the "var a = [" array are also dynamic

If you analyze the function, you will see that the Token is generated by parts of this array returned in the index (home page https://www.bet365.com/), because on the home page there is the "var a = [", the problem that I discovered is that the positions of the elements are dynamic, that is: a [100], a [134] etc.

incapdns commented 3 years ago

I managed to bypass the protection of Bet365 where the content (script of the initial page) is generated dynamically, instead of deciphering the positions of the array with reverse engineering, I just used "eval" in the array, and used the function contained in Bet365 (image and code above), to generate the token.

I tested this code for 9 days, and so far it is functional. I believe I was able to discover a definitive form that does not need much work

Code working: https://pastebin.com/NjGGCFrX

incapdns commented 3 years ago

Put the code (pastebin) in file "code.js", and use this in NodeJS:

const jsdom = require("jsdom")
const { JSDOM } = jsdom

const code = fs.readFileSync('./code.js').toString()

let dom =  new JSDOM('', { 
    pretendToBeVisual: false, 
    runScripts: "outside-only",
    url: "https://www.bet365.com/",
    referrer: "https://www.bet365.com/",
})
dom.window.eval(code)
dom.window.generate(bet365).then(console.log)

Note: You don't need to recreate the "dom" instance to reuse the "generate" function, just use it normally, if you need to extract the token from several Bet365 (example: Several F5 tokens), just call the function:

window.window.generate(Bet365HTML).then(token => {
    console.log ('The token is:', token)
})

The generating time is quite short, around 1 ~ 5 milliseconds *(ms)

incapdns commented 3 years ago

Important: Domjs new JSDOM ('' ", {url: 'https://www.bet365.com'}) is not loading Bet365, just simulating window.origin and window.referer to be www.bet365.com.

JSDOM is also not a headless browser, it is lighter than a headless browser, as it only simulates the DOM elements like "document.create" or "HTMLAnchorElement.prototype" through the 100% javascript code implementation.

Note: You can also use this code without NodeJS directly in the browser when creating a page like "test.html" and using the " Githubissues.

  • Githubissues is a development platform for aggregating issues.