search

moll / node-mitm

Intercept and mock outgoing Node.js network TCP connections and HTTP requests for testing. Intercepts and gives you a Net.Socket, Http.IncomingMessage and Http.ServerResponse to test and respond with. Super useful when testing code that hits remote servers.
Other
641 stars 48 forks source link

node-mitm with Electron #43

Closed Devhercule closed 7 years ago

Devhercule commented 7 years ago

Hi, I'm developing an Electron application,

How can i use your MITM with my application, Have you any example?

Thank you

moll commented 7 years ago

Hey,

What do you want to do with it? Test the network requests of your app?

Devhercule commented 7 years ago

Yess, I want to use a MITM proxy to listen requests and display them (display all requests sent by my electron app) with possibility to change them also (add headers...) I want to do this usin a MITM proxy outside the browser..

moll commented 7 years ago

Use in any Node.js based environment should be identical to use in... Node.js. :) Is something in the "Using" example list not working under Electron?

Devhercule commented 7 years ago

I was inspired by your example to integrate MITM proxy in my electron app

here is the code:

const electron = require('electron')

const {app, BrowserWindow, session, ipcMain} = require('electron')
const path = require('path')
const url = require('url')
const { URL } = require('url');
const protocol = electron.protocol

var Mitm = require("mitm")
var mitm = Mitm()

var Http = require('http');

let win

function createWindow () {

    var screenElectron = electron.screen;
    var mainScreen = screenElectron.getPrimaryDisplay();
    var dimensions = mainScreen.size;

  // Create the browser window.
  win = new BrowserWindow({width: dimensions.width, height: dimensions.height})

    //clear cache
    win.webContents.session.clearCache(function(){});
    win.webContents.clearHistory();

mitm.on('request', function(req, res) {

  console.log('----------------------------------------------------------------------------')
  console.log(req)
})

    session.defaultSession.webRequest.onSendHeaders({urls: ['<all_urls>']}, (details) => {
        Http.get(details.url)
    })

  // Open the DevTools.
  win.webContents.openDevTools()

win.loadURL('http://www.rtl.fr/')

  // Emitted when the window is closed.
  win.on('closed', () => {
    win = null
  })
}

app.on('ready', createWindow)

// Quit when all windows are closed.
app.on('window-all-closed', () => {
  if (process.platform !== 'darwin') {
    app.quit()
  }
})

app.on('activate', () => {
  if (win === null) {
    createWindow()
  }
})

here is a little part of the result : 

----------------------------------------------------------------------------
IncomingMessage {
  _readableState:
   ReadableState {
     objectMode: false,
     highWaterMark: 16384,
     buffer: BufferList { head: null, tail: null, length: 0 },
     length: 0,
     pipes: null,
     pipesCount: 0,
     flowing: null,
     ended: false,
     endEmitted: false,
     reading: false,
     sync: true,
     needReadable: false,
     emittedReadable: false,
     readableListening: false,
     resumeScheduled: false,
     defaultEncoding: 'utf8',
     ranOut: false,
     awaitDrain: 0,
     readingMore: true,
     decoder: null,
     encoding: null },
  readable: true,
  domain: null,
  _events: {},
  _eventsCount: 0,
  _maxListeners: undefined,
  socket:
   Socket {
     connecting: false,
     _hadError: false,
     _handle:
      InternalSocket {
        _readableState: [Object],
        readable: true,
        domain: null,
        _events: [Object],
        _eventsCount: 3,
        _maxListeners: undefined,
        _writableState: [Object],
        writable: true,
        allowHalfOpen: true,
        remote: [Object],
        owner: [Circular],
        onread: [Function: onread],
        reading: true },
     _parent: null,
     _host: null,
     _readableState:
      ReadableState {
        objectMode: false,
        highWaterMark: 16384,
        buffer: [Object],
        length: 0,
        pipes: null,
        pipesCount: 0,
        flowing: true,
        ended: false,
        endEmitted: false,
        reading: false,
        sync: false,
        needReadable: true,
        emittedReadable: false,
        readableListening: false,
        resumeScheduled: false,
        defaultEncoding: 'utf8',
        ranOut: false,
        awaitDrain: 0,
        readingMore: false,
        decoder: null,
        encoding: null },
     readable: true,
     domain: null,
     _events:
      { end: [Object],
        finish: [Function: onSocketFinish],
        _socketEnd: [Function: onSocketEnd],
        drain: [Object],
        timeout: [Function: bound socketOnTimeout],
        data: [Function: bound socketOnData],
        error: [Function: bound socketOnError],
        close: [Object],
        resume: [Function: onSocketResume],
        pause: [Function: onSocketPause] },
     _eventsCount: 10,
     _maxListeners: undefined,
     _writableState:
      WritableState {
        objectMode: false,
        highWaterMark: 16384,
        needDrain: false,
        ending: false,
        ended: false,
        finished: false,
        decodeStrings: false,
        defaultEncoding: 'utf8',
        length: 0,
        writing: false,
        corked: 0,
        sync: true,
        bufferProcessing: false,
        onwrite: [Function: bound onwrite],
        writecb: null,
        writelen: 0,
        bufferedRequest: null,
        lastBufferedRequest: null,
        pendingcb: 0,
        prefinished: false,
        errorEmitted: false,
        bufferedRequestCount: 0,
        corkedRequestsFree: [Object] },
     writable: true,
     allowHalfOpen: false,
     destroyed: false,
     _bytesDispatched: 0,
     _sockname: null,
     _writev: null,
     _pendingData: null,
     _pendingEncoding: '',
     server: null,
     _server: null,
     parser:
      HTTPParser {
        '0': [Function: parserOnHeaders],
        '1': [Function: parserOnHeadersComplete],
        '2': [Function: parserOnBody],
        '3': [Function: parserOnMessageComplete],
        '4': [Function: bound onParserExecute],
        _headers: [],
        _url: '',
        _consumed: false,
        socket: [Circular],
        incoming: [Circular],
        outgoing: null,
        maxHeaderPairs: 2000,
        onIncoming: [Function: bound parserOnIncoming] },
     on: [Function: socketOnWrap],
     _paused: false,
     read: [Function],
     _consuming: true,
     _httpMessage:
      ServerResponse {
        domain: null,
        _events: [Object],
        _eventsCount: 1,
        _maxListeners: undefined,
        output: [],
        outputEncodings: [],
        outputCallbacks: [],
        outputSize: 0,
        writable: true,
        _last: false,
        upgrading: false,
        chunkedEncoding: false,
        shouldKeepAlive: false,
        useChunkedEncodingByDefault: true,
        sendDate: true,
        _removedHeader: {},
        _contentLength: null,
        _hasBody: true,
        _trailer: '',
        finished: false,
        _headerSent: false,
        socket: [Circular],
        connection: [Circular],
        _header: null,
        _headers: null,
        _headerNames: {},
        _onPendingData: [Function: bound updateOutgoingData],
        _sent100: false,
        _expect_continue: false,
        req: [Circular] } },
  connection:
   Socket {
     connecting: false,
     _hadError: false,
     _handle:
      InternalSocket {
        _readableState: [Object],
        readable: true,
        domain: null,
        _events: [Object],
        _eventsCount: 3,
        _maxListeners: undefined,
        _writableState: [Object],
        writable: true,
        allowHalfOpen: true,
        remote: [Object],
        owner: [Circular],
        onread: [Function: onread],
        reading: true },
     _parent: null,
     _host: null,
     _readableState:
      ReadableState {
        objectMode: false,
        highWaterMark: 16384,
        buffer: [Object],
        length: 0,
        pipes: null,
        pipesCount: 0,
        flowing: true,
        ended: false,
        endEmitted: false,
        reading: false,
        sync: false,
        needReadable: true,
        emittedReadable: false,
        readableListening: false,
        resumeScheduled: false,
        defaultEncoding: 'utf8',
        ranOut: false,
        awaitDrain: 0,
        readingMore: false,
        decoder: null,
        encoding: null },
     readable: true,
     domain: null,
     _events:
      { end: [Object],
        finish: [Function: onSocketFinish],
        _socketEnd: [Function: onSocketEnd],
        drain: [Object],
        timeout: [Function: bound socketOnTimeout],
        data: [Function: bound socketOnData],
        error: [Function: bound socketOnError],
        close: [Object],
        resume: [Function: onSocketResume],
        pause: [Function: onSocketPause] },
     _eventsCount: 10,
     _maxListeners: undefined,
     _writableState:
      WritableState {
        objectMode: false,
        highWaterMark: 16384,
        needDrain: false,
        ending: false,
        ended: false,
        finished: false,
        decodeStrings: false,
        defaultEncoding: 'utf8',
        length: 0,
        writing: false,
        corked: 0,
        sync: true,
        bufferProcessing: false,
        onwrite: [Function: bound onwrite],
        writecb: null,
        writelen: 0,
        bufferedRequest: null,
        lastBufferedRequest: null,
        pendingcb: 0,
        prefinished: false,
        errorEmitted: false,
        bufferedRequestCount: 0,
        corkedRequestsFree: [Object] },
     writable: true,
     allowHalfOpen: false,
     destroyed: false,
     _bytesDispatched: 0,
     _sockname: null,
     _writev: null,
     _pendingData: null,
     _pendingEncoding: '',
     server: null,
     _server: null,
     parser:
      HTTPParser {
        '0': [Function: parserOnHeaders],
        '1': [Function: parserOnHeadersComplete],
        '2': [Function: parserOnBody],
        '3': [Function: parserOnMessageComplete],
        '4': [Function: bound onParserExecute],
        _headers: [],
        _url: '',
        _consumed: false,
        socket: [Circular],
        incoming: [Circular],
        outgoing: null,
        maxHeaderPairs: 2000,
        onIncoming: [Function: bound parserOnIncoming] },
     on: [Function: socketOnWrap],
     _paused: false,
     read: [Function],
     _consuming: true,
     _httpMessage:
      ServerResponse {
        domain: null,
        _events: [Object],
        _eventsCount: 1,
        _maxListeners: undefined,
        output: [],
        outputEncodings: [],
        outputCallbacks: [],
        outputSize: 0,
        writable: true,
        _last: false,
        upgrading: false,
        chunkedEncoding: false,
        shouldKeepAlive: false,
        useChunkedEncodingByDefault: true,
        sendDate: true,
        _removedHeader: {},
        _contentLength: null,
        _hasBody: true,
        _trailer: '',
        finished: false,
        _headerSent: false,
        socket: [Circular],
        connection: [Circular],
        _header: null,
        _headers: null,
        _headerNames: {},
        _onPendingData: [Function: bound updateOutgoingData],
        _sent100: false,
        _expect_continue: false,
        req: [Circular] } },
  httpVersionMajor: 1,
  httpVersionMinor: 1,
  httpVersion: '1.1',
  complete: false,
  headers: { host: 'media.rtl.fr', connection: 'close' },
  rawHeaders: [ 'Host', 'media.rtl.fr', 'Connection', 'close' ],
  trailers: {},
  rawTrailers: [],
  upgrade: false,
  url: '/cache/MtB7C_urPiSdugbi27RYQQ/280v187-2/online/image/2017/0704/7789207694_les-membres-du-conseil-economique-social-et-environnemental-le-mediateur-de-la-republique-sont-reunis-en-assemblee-pleniere
-le-16-novembre-2010-a-paris.jpg',
  method: 'GET',
  statusCode: null,
  statusMessage: null,
  client:
   Socket {
     connecting: false,
     _hadError: false,
     _handle:
      InternalSocket {
        _readableState: [Object],
        readable: true,
        domain: null,
        _events: [Object],
        _eventsCount: 3,
        _maxListeners: undefined,
        _writableState: [Object],
        writable: true,
        allowHalfOpen: true,
        remote: [Object],
        owner: [Circular],
        onread: [Function: onread],
        reading: true },
     _parent: null,
     _host: null,
     _readableState:
      ReadableState {
        objectMode: false,
        highWaterMark: 16384,
        buffer: [Object],
        length: 0,
        pipes: null,
        pipesCount: 0,
        flowing: true,
        ended: false,
        endEmitted: false,
        reading: false,
        sync: false,
        needReadable: true,
        emittedReadable: false,
        readableListening: false,
        resumeScheduled: false,
        defaultEncoding: 'utf8',
        ranOut: false,
        awaitDrain: 0,
        readingMore: false,
        decoder: null,
        encoding: null },
     readable: true,
     domain: null,
     _events:
      { end: [Object],
        finish: [Function: onSocketFinish],
        _socketEnd: [Function: onSocketEnd],
        drain: [Object],
        timeout: [Function: bound socketOnTimeout],
        data: [Function: bound socketOnData],
        error: [Function: bound socketOnError],
        close: [Object],
        resume: [Function: onSocketResume],
        pause: [Function: onSocketPause] },
     _eventsCount: 10,
     _maxListeners: undefined,
     _writableState:
      WritableState {
        objectMode: false,
        highWaterMark: 16384,
        needDrain: false,
        ending: false,
        ended: false,
        finished: false,
        decodeStrings: false,
        defaultEncoding: 'utf8',
        length: 0,
        writing: false,
        corked: 0,
        sync: true,
        bufferProcessing: false,
        onwrite: [Function: bound onwrite],
        writecb: null,
        writelen: 0,
        bufferedRequest: null,
        lastBufferedRequest: null,
        pendingcb: 0,
        prefinished: false,
        errorEmitted: false,
        bufferedRequestCount: 0,
        corkedRequestsFree: [Object] },
     writable: true,
     allowHalfOpen: false,
     destroyed: false,
     _bytesDispatched: 0,
     _sockname: null,
     _writev: null,
     _pendingData: null,
     _pendingEncoding: '',
     server: null,
     _server: null,
     parser:
      HTTPParser {
        '0': [Function: parserOnHeaders],
        '1': [Function: parserOnHeadersComplete],
        '2': [Function: parserOnBody],
        '3': [Function: parserOnMessageComplete],
        '4': [Function: bound onParserExecute],
        _headers: [],
        _url: '',
        _consumed: false,
        socket: [Circular],
        incoming: [Circular],
        outgoing: null,
        maxHeaderPairs: 2000,
        onIncoming: [Function: bound parserOnIncoming] },
     on: [Function: socketOnWrap],
     _paused: false,
     read: [Function],
     _consuming: true,
     _httpMessage:
      ServerResponse {
        domain: null,
        _events: [Object],
        _eventsCount: 1,
        _maxListeners: undefined,
        output: [],
        outputEncodings: [],
        outputCallbacks: [],
        outputSize: 0,
        writable: true,
        _last: false,
        upgrading: false,
        chunkedEncoding: false,
        shouldKeepAlive: false,
        useChunkedEncodingByDefault: true,
        sendDate: true,
        _removedHeader: {},
        _contentLength: null,
        _hasBody: true,
        _trailer: '',
        finished: false,
        _headerSent: false,
        socket: [Circular],
        connection: [Circular],
        _header: null,
        _headers: null,
        _headerNames: {},
        _onPendingData: [Function: bound updateOutgoingData],
        _sent100: false,
        _expect_continue: false,
        req: [Circular] } },
  _consuming: false,
  _dumped: false,
  res:
   ServerResponse {
     domain: null,
     _events: { finish: [Function: bound resOnFinish] },
     _eventsCount: 1,
     _maxListeners: undefined,
     output: [],
     outputEncodings: [],
     outputCallbacks: [],
     outputSize: 0,
     writable: true,
     _last: false,
     upgrading: false,
     chunkedEncoding: false,
     shouldKeepAlive: false,
     useChunkedEncodingByDefault: true,
     sendDate: true,
     _removedHeader: {},
     _contentLength: null,
     _hasBody: true,
     _trailer: '',
     finished: false,
     _headerSent: false,
     socket:
      Socket {
        connecting: false,
        _hadError: false,
        _handle: [Object],
        _parent: null,
        _host: null,
        _readableState: [Object],
        readable: true,
        domain: null,
        _events: [Object],
        _eventsCount: 10,
        _maxListeners: undefined,
        _writableState: [Object],
        writable: true,
        allowHalfOpen: false,
        destroyed: false,
        _bytesDispatched: 0,
        _sockname: null,
        _writev: null,
        _pendingData: null,
        _pendingEncoding: '',
        server: null,
        _server: null,
        parser: [Object],
        on: [Function: socketOnWrap],
        _paused: false,
        read: [Function],
        _consuming: true,
        _httpMessage: [Circular] },
     connection:
      Socket {
        connecting: false,
        _hadError: false,
        _handle: [Object],
        _parent: null,
        _host: null,
        _readableState: [Object],
        readable: true,
        domain: null,
        _events: [Object],
        _eventsCount: 10,
        _maxListeners: undefined,
        _writableState: [Object],
        writable: true,
        allowHalfOpen: false,
        destroyed: false,
        _bytesDispatched: 0,
        _sockname: null,
        _writev: null,
        _pendingData: null,
        _pendingEncoding: '',
        server: null,
        _server: null,
        parser: [Object],
        on: [Function: socketOnWrap],
        _paused: false,
        read: [Function],
        _consuming: true,
        _httpMessage: [Circular] },
     _header: null,
     _headers: null,
     _headerNames: {},
     _onPendingData: [Function: bound updateOutgoingData],
     _sent100: false,
     _expect_continue: false,
     req: [Circular] } }
----------------------------------------------------------------------------
IncomingMessage {
  _readableState:
   ReadableState {
     objectMode: false,

Questions :

1/ The method I used, is it the right one to use your module with Electron?

2/ How can I display the whole URL sent by the browser? (including scheme, domain...) I dont want to display all parameters displayed in my console.

3/ How can i Send original request Headers set by the browser with http.get?? because actually, only the url in sent.

Thank you

moll commented 7 years ago

You're using it right. However, in your exact example you seem to control the code that makes the HTTP request (by hooking into something like session.defaultSession.webRequest.onSendHeaders). If you wanted to add headers to that request, you could just add those headers before the call to Http.get. Or is that callback in Electron not used for all requests the browser-y window makes? I've never used Electron.

As for getting the URL out of that request, it'd be:

(req.connection.encrypted ? "https" : "http") + "://" + req.headers.host + req.url

But maybe you can solve your entire wish to intercept Electron requests by finding a callback Electron already has for that purpose and adding your headers before calling Http.get. That'd be a far better solution that using this testing library for it. :)

Devhercule commented 7 years ago

Thank you for your reply

I tried to use Http.request instead of Http.get, but nothing is displayed !!!

https://nodejs.org/api/http.html#http_http_request_options_callback

Can this help me sending headers to proxy?

There is another big problem with this... The same request is sent two times (one by session.defaultSession.webRequest.onSendHeaders and another one by Http.get) The goal is to send the request only one time (only via the MITM proxy) Is that possible???

Is there a possibility to intercept requests without Http.get? to avoid multiple request send

moll commented 7 years ago

I guess you could try not sending the request through onSendHeaders. :) You sure that Electron API is meant for you to intercept those requests and not just inspect their headers? Try removing that onSendHeaders entirely and if Mitm.js doesn't intercept any requests, you'll have to go inspect how Electron does its requests. Maybe they don't go through Node.js at all by default.

Devhercule commented 7 years ago

after removing onSendHeaders, no request is intercepted

This is my code, nothing is diplayed, requests are not intercepted by mitm proxy !!!! Yet the website is well displayed

const electron = require('electron')

const {app, BrowserWindow, session, ipcMain} = require('electron')
const path = require('path')
const url = require('url')
const { URL } = require('url');
const protocol = electron.protocol

var Mitm = require("mitm")
var mitm = Mitm()

var Http = require('http');

let win

function createWindow () {

    var screenElectron = electron.screen;
    var mainScreen = screenElectron.getPrimaryDisplay();
    var dimensions = mainScreen.size;

  // Create the browser window.
  win = new BrowserWindow({width: dimensions.width, height: dimensions.height})

    //clear cache
    win.webContents.session.clearCache(function(){});
    win.webContents.clearHistory();

mitm.on('request', function(req, res) {

  console.log('----------------------------------------------------------------------------')
  console.log(req)
})

  // Open the DevTools.
  win.webContents.openDevTools()

win.loadURL('http://www.rtl.fr/')

  // Emitted when the window is closed.
  win.on('closed', () => {
    win = null
  })
}

app.on('ready', createWindow)

// Quit when all windows are closed.
app.on('window-all-closed', () => {
  if (process.platform !== 'darwin') {
    app.quit()
  }
})

app.on('activate', () => {
  if (win === null) {
    createWindow()
  }
})

Can you tell me how can i intercept the requests!!

Known that Electron use "loadURL" to load urls

moll commented 7 years ago

Sadly I can't. I have no experience with Electron. You might want to scour their documentation. Perhaps they have an easy way for you to intercept web requests they make, akin to you finding the onHeaders callback. Suffice to say that if Mitm.js doesn't pick them up in your code above, then they do not go through Node.js as regular Http.get-style requests do.

Devhercule commented 7 years ago

thank you anyway :)

moll commented 7 years ago

You're welcome. I'll close this issue as there's nothing actionable for me right now, but feel free to continue chatting. ;)

Devhercule commented 7 years ago

Hi moll,

I have a question about your method to get url (req.connection.encrypted ? "https" : "http") + "://" + req.headers.host + req.url

Can I change the scheme of an URL, I want to change "https" by "http", How can i do this?

Known that i want also to modifi host, it is possible via req.headers.host.

Thank you

moll commented 7 years ago

At that moment in code whatever changes you do to req have no effect on the actual request — the original request will never go through, after all. If you possibly want to mutate something, you could look at the mitm.on("connect") callback. Then you at least have a chance of the request actually hitting the Node internal Http module.

Devhercule commented 7 years ago

Thank you