Example about using HTTP proxy on http2?

[sinisterdev]

I was trying to use the got module with an http proxy (tried fiddler atm) but I can't find an example, they're all using https, isn't http supported?

[szmarczak]

The API is all the same for all proxies.

https://github.com/szmarczak/http2-wrapper/blob/0773c56c4068231a266b46e70491a26a36828323/examples/proxies/h2-over-h1.js#L4-L6

Just use Http2OverHttp instead (along with http:// as the url option) and you're good to go.

[sinisterdev]

I already tried that, and sadly using burp suite, the request returns a state 200 but does not get logged in my proxy, here's the code:

process.env["NODE_TLS_REJECT_UNAUTHORIZED"] = 0;
const http2 = require("http2-wrapper");

const httpsProxy = new http2.proxies.Http2OverHttps({
  proxyOptions: {
    url: "http://localhost:8080",

    rejectUnauthorized: false,
  },
});
(async () => {
  let res = await got.get("https://www.google.com/", {
    followRedirect: false,
    http2: true,
    agent: {
      https: httpsProxy,
    },
    headers: {
      accept: "application/json, text/javascript, */*; q=0.01",
      referer: "https://www.google.com/",
      "user-agent":
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.104 Safari/537.36",
      "X-Requested-With": "XMLHttpRequest",
    },
  });
  console.log(res.statusCode);
})();

[sinisterdev]

The API is all the same for all proxies.

https://github.com/szmarczak/http2-wrapper/blob/0773c56c4068231a266b46e70491a26a36828323/examples/proxies/h2-over-h1.js#L4-L6

Just use Http2OverHttp instead (along with http:// as the url option) and you're good to go.

Also, using this example, still with burp suite or using fiddler proxy, nothing happens, nothing at all shows in the console, nor in the proxy.

const http2 = require("http2-wrapper");

const agent = new http2.proxies.Http2OverHttps({
  proxyOptions: {
    url: "http://localhost:8080",

    // For demo purposes only!
    rejectUnauthorized: false,
  },
});

const request = http2.request(
  {
    hostname: "httpbin.org",
    protocol: "https:",
    path: "/anything",
    method: "POST",
    headers: {
      "content-length": 6,
    },
    agent,
  },
  (response) => {
    console.log("statusCode:", response.statusCode);
    console.log("headers:", response.headers);

    const body = [];
    response.on("data", (chunk) => {
      body.push(chunk);
    });
    response.on("end", () => {
      console.log("body:", Buffer.concat(body).toString());
    });
  }
);

request.on("error", console.error);

request.write("123");
request.end("456");

[sinisterdev]

Same, by using http2 on agent options with got package, nothing gets printed in the console, and nothing shows up in the proxy.

const got = require("got");
const http2 = require("http2-wrapper");

const httpsProxy = new http2.proxies.Http2OverHttp({
  proxyOptions: {
    url: "http://localhost:8080",

    // For demo purposes only!
    rejectUnauthorized: false,
  },
});
(async () => {
  let res = await got.get("https://www.google.com/", {
    followRedirect: false,
    http2: true,
    agent: {
      https: httpsProxy,
      http: httpsProxy,
      http2: httpsProxy,
    },
    headers: {
      accept: "application/json, text/javascript, */*; q=0.01",
      referer: "https://www.google.com/",
      "user-agent":
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.104 Safari/537.36",
      "X-Requested-With": "XMLHttpRequest",
    },
  });
  console.log(res.statusCode);
})();

[sinisterdev]

After doing some more tests using Charles Proxy (supports http2), seems like its silently exiting when It can't handshake properly, for example. sending a GET request to https://www.ipify.org/ the first CONNECT request returns a status 200

However, when I try on other websites ( example google.com ) the handshake fails, returning a status 503

And, when I get this error during the handshake, the script silently exits, throwing no errors, any idea how to solve this?

[szmarczak]

got@11.8.2 uses an old version of http2-wrapper. You need to

const http2wrapper = require('http2-wrapper');

got(..., {
    http2: true,
    request: http2wrapper.auto
});

[szmarczak]

I just tried the fiddler proxy and everything is working as expected:

[sinisterdev]

I just tried the fiddler proxy and everything is working as expected:

Are you able to see the response body?

[szmarczak]

[szmarczak]

If I enable HTTPS decryption, then the request fails:

Error: Session closed without receiving a SETTINGS frame
    at ClientHttp2Session.<anonymous> (/home/szm/Desktop/http2-wrapper/source/agent.js:443:22)
    at Object.onceWrapper (node:events:484:28)
    at ClientHttp2Session.emit (node:events:378:20)
    at emitClose (node:internal/http2/core:1038:8)
    at processTicksAndRejections (node:internal/process/task_queues:81:21) {
  code: 'HTTP2WRAPPER_NOSETTINGS'
}

[sinisterdev]

If I enable HTTPS decryption, then the request fails:

Error: Session closed without receiving a SETTINGS frame
    at ClientHttp2Session.<anonymous> (/home/szm/Desktop/http2-wrapper/source/agent.js:443:22)
    at Object.onceWrapper (node:events:484:28)
    at ClientHttp2Session.emit (node:events:378:20)
    at emitClose (node:internal/http2/core:1038:8)
    at processTicksAndRejections (node:internal/process/task_queues:81:21) {
  code: 'HTTP2WRAPPER_NOSETTINGS'
}

But, in the code, the got response does it contain a body? because it usually does not print anything for me, I am trying to access the website html ( response body )

[szmarczak]

'use strict';
const http2 = require('../../source'); // Note: using the local version
const got = require('got');

const agent = new http2.proxies.Http2OverHttp({
    proxyOptions: {
        url: 'http://localhost:8866',

        // For demo purposes only!
        rejectUnauthorized: false
    }
});

got('https://httpbin.org/anything', {
    http2: true,
    request: http2.auto,
    agent: {
        http2: agent
    },
    https: {
        rejectUnauthorized: false
    }
}).then(response => console.log(response.body), console.error);

If I enable HTTPS decryption inside Fiddler, I get

Error [ERR_STREAM_DESTROYED]: Cannot call end after a stream was destroyed
    at new NodeError (node:internal/errors:329:5)
    at errorBuffer (node:internal/streams/writable:527:14)
    at processTicksAndRejections (node:internal/process/task_queues:80:21) {
  code: 'ERR_STREAM_DESTROYED'
}

Because the http2 request fails too early.

[sinisterdev]

I understand now, thank you for helping me out, maybe they should add this to official got documentation for http2 proxies. Thank you

[szmarczak]

No problem. I have no idea why enabling HTTPS decryption in Fiddler makes the request fail though.

[sinisterdev]

I think its because fiddler uses a self signed certificate so the TLS handshake fails due to invalid cert?

[szmarczak]

No, I have set rejectUnauthorized to false.

[szmarczak]

Oh, I know what happens. I'm pretty sure Fiddler doesn't support HTTP/2 when HTTPS decrypting is on.

[szmarczak]

Yeah, socket.alpnProtocol shows null instead of h2.