MITM server blocks recaptcha requests

[Baker68]

const Hero = require('@ulixee/hero-playground');

(async () => {
    const hero = new Hero({
        showChrome: true,
        showChromeInteractions: true,
        disableDevtools: true,
        disableMitm: false
    });

    await hero.goto('https://patrickhlauke.github.io/recaptcha/');
    /** new WS feature test */
    hero.activeTab.on('resource', (e) => {
        console.log(e);
        console.log('---------------------------');
    })
    await hero.waitForPaintingStable();
    // await hero.close();
})();

As you can see, the request was canceled. If I set disableMitm: true the request won't be blocked.

---

The second issue is that if you set disableMitm: true ; the hero.activeTab.on('resource'... won't be called but the following is shown in console :

2022-08-16T20:37:37.863Z ERROR [hero-core/node_modules/@unblocked-web/agent/lib/WebsocketMessages] CaptureWebsocketMessageError.UnregisteredResource {
  event: {
    resourceId: undefined,
    message: '{"online":[]}',
    isFromServer: true,
    lastCommandId: 3,
    timestamp: 1660682257862.168
  },
  context: { sessionId: 'L1HrZGvpbRG8WeLg-8REM' },
  sessionId: 'L1HrZGvpbRG8WeLg-8REM',
  sessionName: 'default-session'
}

Or maybe this is not an issue, but it makes sens (to me) to be passed to the callback since it's there.

[blakebyrnes]

Thanks @Baker68. We should probably have a big caveat with the disableMitm flag. I exposed that because we had a user heavily requesting it, but it's definitely not a hardened feature - it also opens you up to the detections the MITM is there to prevent in the first place. But... that said, what you're doing is part of why it's there. It's meant for testing.

I'm trying to get out this release with the web socket fix in it (the other ticket you logged). I'll be curious if that fixes this issue.

[Baker68]

@blakebyrnes since #141 is closed , I will post here ; I've modified node_modules/@unblocked-web/agent-mitm/lib/MitmProxy.js and added {allowHTTP1: true} to http2.createSecureServer constructor and test it against https://libwebsockets.org/testserver/ and several other websites with similar functionality and it works, WS Client is now connecting to the intended host server.

However, I would like to point out that the first time when you visit https://www.piesocket.com/websocket-tester the WS connection fails , but manages to connect upon page reload, here's the PoC :

const Hero = require('@ulixee/hero-playground');

const clickConnect = async (hero) => {
    const connectButton = hero.document.querySelector('button[type="submit"]');
    await hero.interact({move: connectButton}, {click: connectButton});
}

const sleep = ms => new Promise(r => setTimeout(r, ms));

(async () => {
    const hero = new Hero({
        showChrome: true,
        showChromeInteractions: true,
        disableDevtools: false,
        disableMitm: false
    });

    await hero.goto('https://www.piesocket.com/websocket-tester'); // https://www.piesocket.com/websocket-tester ; https://libwebsockets.org/testserver/
    /** new WS feature test */
    hero.activeTab.on('resource', async (e) => {
        console.log(e);
        console.log('---------------------------');
    })
    await hero.waitForPaintingStable();
    await clickConnect(hero);
    await sleep(2000);
    await hero.activeTab.reload();
    await hero.waitForPaintingStable();
    await clickConnect(hero);
    // await hero.close();
})();

[blakebyrnes]

@Baker68 I'm not seeing either of these issues with the 2.0.0-alpha.10 release. Could you see if you are seeing them in the newest version?

[blakebyrnes]

@Baker68 Should I close this issue?

[Baker68]

@blakebyrnes ; Yes, you can close it ; It connects now ; I was not using the correct version. Sorry. But I will keep testing this vs. other sites and I will keep you up to date with the tests.