FormData body is parsed as "[object FormData]" string

[drazik]

Describe the bug

When sending a POST request with a FormData body, the req.body contains a string "[object FormData]" and I can't get the data that was sent.

Environment

• msw: 0.26.2
• nodejs: 14.15.4
• npm: 6.14.11

Browser version is not relevant as I'm using msw in Jest tests context. Jest version is 26.6.3.

To Reproduce

1. Send a request from the application with POST method and a FormData object as the body

const form = document.querySelector(".my-form")

fetch("http://localhost/user/register", {
  method: "POST",
  body: new FormData(form)
})

2. Implement a POST handler in msw and log req.body:

import { rest } from "msw"

export const handlers = [
    rest.post("/user/register", (req, res, ctx) => {
    console.log(req.body, typeof req.body)

    return res(ctx.status(200))
  }),
]

3. See that you get [object FormData] string as the console.log result

Expected behavior

I expected req.body to be an object like it is in the browser.

[drazik]

After looking at what are the differences between the browser environment and my tests environment, I thought the problem may be that I'm using node-fetch as a fetch polyfill for the tests. But when using node-fetch in the browser, req.body is an object just like it should be. So I don't think this is the reason why I have this result in my tests.

[kettanaito]

Thank you for reporting this, @drazik! I'll put this into an integration test and investigate any possible failures.

[kettanaito]

@drazik, could you please take a look at the test in #600? I've got the FormData as the request body test passing successfully. The req.body is an object of serialized form fields. Perhaps you can spot some differences in your usage.

[drazik]

Thanks @kettanaito for investigating this.

The only difference I see is that you are using setupWorker, so it means you are in a browser context if I'm not mistaken. In my case, I use setupServer because I am in a Jest tests (so node js) context.

The tests I did in the browser showed that it works well in this environment. But in node environement req.body is a string.

[kettanaito]

Great observation! Let's add a NodeJS test as well...

[kettanaito]

Coming back to you with my findings.

FormData in a browser

As mentioned previously, I've got a basic in-browser test working well with constructing FormData and passing it as a request's body. MSW provides you the request body as serialized FormData (done by fetch), so you could access it in req.body[fieldName].

FormData in DOM-like environment

In a DOM-like environment (i.e. JSDOM), you need to use a FormData polyfill. Jest comes with one, but bear in mind that FormData polyfills are not always compatible with fetch polyfills you decide to use. For instance, this is invalid:

import fetch from 'node-fetch'

fetch('/user', {
  method: 'POST',
  // Types of `BodyInit` of `node-fetch` and Jest's `FormData` polyfill are incompatible. 
  body: new FormData(document.getElementById('user-form')
})

It is when you use an incompatible polyfill for your request client you get the [object FormData] text as your request body. MSW receives whichever request body your client produces, there's no parsing logic whatsoever (one of the benefits of Service Workers in a browser and native modules monkey-patching in NodeJS).

FormData in NodeJS

Unless you use a polyfill (i.e. form-data-node), there is no FormData in NodeJS. This makes sense, as you don't have DOM and thus forms to construct that data. Polyfills use a Map-like API to allow you to append/set form fields on the polyfilled FormData instance.

How to get the proper data in the request?

If you use the right FormData polyfill for your request issuing library you'll get the serialized form data in the request body:

import fetch from 'node-fetch'
import FormDataPolyfill from 'form-data'

const formData = new FormDataPolyfill()
formData.append('username', 'john.maverick')
formData.append('password', 'secret123')

fetch('/user', {
  method: 'POST',
  body: formData
})

I understand this is not ideal, as you're detaching your test from DOM/JSDOM to act as the source for data. I'd also suggest experimenting with fetch/FormData polyfills to find a suitable combination. Alternatively, consider moving such a test to an actual browser as that would give you the most confidence and you wouldn't have to deal with polyfills. You can use tools like Cypress for that, or in case you feel like it's an overkill to pull in a new testing framework you may consider tiny utilities like page-with that still run in an actual browser.

[JacobMGEvans]

Just for posterity this might help some people

// The following to functions workaround the fact that MSW does not yet support FormData in requests.
// We use the fact that MSW relies upon `node-fetch` internally, which will call `toString()` on the FormData object,
// rather than passing it through  or serializing it as a proper FormData object.
// The hack is to serialize FormData to a JSON string by overriding `FormData.toString()`.
// And then to deserialize back to a FormData object by monkey-patching a `formData()` helper onto `MockedRequest`.
FormData.prototype.toString = mockFormDataToString;
function mockFormDataToString(this: FormData) {
    return JSON.stringify({
        __formdata: Array.from(this.entries()),
    });
}
interface RestRequestWithFormData extends MockedRequest, RestRequest {
    formData(): Promise<FormData>;
}
(MockedRequest.prototype as RestRequestWithFormData).formData =
    mockFormDataFromString;
async function mockFormDataFromString(this: MockedRequest): Promise<FormData> {
    const { __formdata } = await this.json();
    expect(__formdata).toBeInstanceOf(Array);
    const form = new FormData();
    for (const [key, value] of __formdata) {
        form.set(key, value);
    }
    return form;
}

Edit: Currently working on getting Blobs to be supported in this hacky-fill lol

[JacobMGEvans]

Completed the Polyfill for Blob support AND patching in .formData() Patching .formData()

function mockFormDataToString(this: FormData) {
    const entries = [];
    for (const [key, value] of this.entries()) {
        if (value instanceof Blob) {
            const reader = new FileReaderSync();
            reader.readAsText(value);
            const result = reader.result;
            entries.push([key, result]);
        } else {
            entries.push([key, value]);
        }
    }
    return JSON.stringify({
        __formdata: entries,
    });
}

async function mockFormDataFromString(this: MockedRequest): Promise<FormData> {
    const { __formdata } = await this.json();
    expect(__formdata).toBeInstanceOf(Array);

    const form = new FormData();
    for (const [key, value] of __formdata) {
        form.set(key, value);
    }
    return form;
}

// The following two functions workaround the fact that MSW does not yet support FormData in requests.
// We use the fact that MSW relies upon `node-fetch` internally, which will call `toString()` on the FormData object,
// rather than passing it through or serializing it as a proper FormData object.
// The hack is to serialize FormData to a JSON string by overriding `FormData.toString()`.
// And then to deserialize back to a FormData object by monkey-patching a `formData()` helper onto `MockedRequest`.
FormData.prototype.toString = mockFormDataToString;
export interface RestRequestWithFormData extends MockedRequest, RestRequest {
    formData(): Promise<FormData>;
}
(MockedRequest.prototype as RestRequestWithFormData).formData =
    mockFormDataFromString;

Reader Polyfill

/*! Read blob sync in NodeJS. MIT License. Jimmy Wärting <https://jimmy.warting.se/opensource>
 * Special Thanks to Jimmy Wärting helping in https://github.com/nodejs/undici/issues/1830
 */
const { join } = require("path");
const {
    Worker,
    receiveMessageOnPort,
    MessageChannel,
} = require("worker_threads");

/**
 * blob-worker & read-file-sync are part of a polyfill to synchronously read a blob in NodeJS
 * this is needed for MSW FormData patching to work and support Blobs, serializing them to a string before recreating the FormData.
 */
function read(blob) {
    const subChannel = new MessageChannel();
    const signal = new Int32Array(new SharedArrayBuffer(4));
    signal[0] = 0;

    const path = join(__dirname, "blob-worker.cjs");

    const worker = new Worker(path, {
        transferList: [subChannel.port1],
        workerData: {
            signal,
            port: subChannel.port1,
            blob,
        },
    });

    // Sleep until the other thread sets signal[0] to 1
    Atomics.wait(signal, 0, 0);

    // Close the worker thread
    worker.terminate();

    return receiveMessageOnPort(subChannel.port2)?.message;
}

class FileReaderSync {
    readAsArrayBuffer(blob) {
        this.result = read(blob);
    }

    readAsDataURL(blob) {
        const ab = read(blob);
        this.result = `data:${blob.type};base64,${Buffer.from(ab).toString(
            "base64"
        )}`;
    }

    readAsText(blob) {
        const ab = read(blob);
        this.result = new TextDecoder().decode(ab);
    }

    // Should not be used, use readAsArrayBuffer instead
    // readAsBinaryString(blob) { ... }
}

exports.FileReaderSync = FileReaderSync;

Worker Thread for Reader Polyfill

/*! Read blob sync in NodeJS. MIT License. Jimmy Wärting <https://jimmy.warting.se/opensource>
 * Special Thanks to Jimmy Wärting helping in https://github.com/nodejs/undici/issues/1830
 */

const { workerData } = require("worker_threads");

const { signal, port, blob } = workerData;

blob.arrayBuffer().then((ab) => {
    // Post the result back to the main thread before unlocking 'signal'
    port.postMessage(ab, [ab]);
    port.close();

    // Change the value of signal[0] to 1
    Atomics.store(signal, 0, 1);

    // This will unlock the main thread when we notify it
    Atomics.notify(signal, 0);
});