Safari 13.1 now includes FormData but polyfill is still needed

[tomsaleeba]

It looks like Safari 13.1 (ships with iOS 13.4) now includes FormData in a Service Worker context. The problem I experienced is that when you create a Request with FormData as the body and try to call Request.arrayBuffer(), it will fail with the error: "The operation is not supported". This same behaviour also happens in the main/UI thread too, it's not just Service Workers.

This polyfill can fix it but the existing check finds the global FormData and the polyfill isn't used.

I've forked and patched it, which you can find here https://github.com/ternandsparrow/FormData/tree/safari13.1. It's not great as I was under time pressure to fix stuff in production. That's now running in production and working, so I'll call that a win. I think the ideal patch would use the native FormData but patch whatever else is required so the .arrayBuffer() call works. I haven't had time to dig into what bit that is yet.

So the first question I want to ask is: Is this issue in scope for this polyfill? Technically FormData doesn't need to be polyfilled but the use of it definitely does.

Assuming it is in scope, if the patch I've made is suitable, I'll PR that. If it's not, I'm happy to make a better patch. Happy to accept any input on how you'd like it done too.

When it comes to demo-ing this, you can use this snippet:

new Request('http://localhost', {method: 'POST', body: new FormData()}).arrayBuffer().then(()=>console.log('success')).catch(err=>console.error('Fail: ' + err.message))

This will work in Chrome as-is, no polyfill needed. It will work with Safari 13.0 (iOS 13.3) with the polyfill, but will fail in Safari 13.1 because the polyfill is not used.

I've also put together a test rig that you can find here: https://tomsaleeba.gitlab.io/fiddles/safari-capabilities/. Compare the results between the two versions of Safari.

There's some more talk about this lack of .arrayBuffer() support here: https://bugs.webkit.org/show_bug.cgi?id=212858.

[jimmywarting]

uh, at first i was like "What?! safari have had a complete formdata for a long time why is it still needed?"

but i got you! have done that kind of hack a lot myself things like await new Response(x).text() etc. the thing is safari don't have a way to de-serialize formdata back to formData. Good that there is an issue for that now!

I once used that method to fetch numeros of files from the server in one request instead of ziping them i just constructed a formdata response instead of a request

const res = await fetch(myfiles)
const fd = await res.formData()
for (const entry in fd.values()) {
  const img = new Image()
  img.src = URL.createObjectURL(entry)
}

It kinda feels out of the scope but then again, we have done some patches to sendBeacon, XHR and fetch but it have so far only been when formdata lacks some spec stuff, it have never been b/c of some external api is unable to handle the native formdata in some way. like in your case where you convert something the way you do.

one thing I fail to understand is how you can make a polyfilled formdata to work with Request/Response if you are using the polyfilled version. The native Request/Response constructor will not see it as a native FormData and it would just cast the object into a string:

await new Response(new FormDataPolyfill()).text() // "[object FormData]"

obviously you must be using fetch in the main thread and then what happens is our patched fetch version will call _blob() and make a request using a blob (and not a formdata instance), which you intercept with a service worker.

[jimmywarting]

enabling what you have done only solves a specific usecase where service worker intervene. so it feels a bit too specific

[jimmywarting]

If we would do something like this we would have to actually patch the Request/Response constructor in some way.

class Req extends Request {
  constructor(url, init) {
    init.body = init.body.toUpperCase() // convert body to blob using _blob
    super(url, init)
  }
}
const req = new Req('https://httpbin.org/post', { method:'post', body: 'x' })
const res = await fetch(req)
await res.json() // json.data === 'X' as uppercase

reason why i haven't done it is b/c IE don't support class extends

[jimmywarting]

you could replace Request with a custom function that converts the body but returns a native Request

const NativeRequest = globalThis.Request

globalThis.Request = function Request(...x) {
  (convert polyfill to blob...)

  return new NativeRequest(...x)
}

but then instanceof checks would fail

new Request() instanceof Request // false

[jimmywarting]

Symbol.hasInstance could be a solution doe

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Symbol/hasInstance

But IE don't support it either, but the hole fetch API don't even exist in IE either so...

[tomsaleeba]

Yeah, I'm working in a Service Worker context. I'm actually using Workbox's background sync feature and the problem happened when I added a request to the queue. The queue calls Request.arrayBuffer() when it saves the request to IndexedDB: https://github.com/GoogleChrome/workbox/blob/05e6bd631b9cd239b8e44eeff132533430e668cb/packages/workbox-background-sync/src/lib/StorableRequest.ts#L65

Thanks for pointing out where the magic happens; in the overridden fetch function. Workbox uses the fetch function so that's why forcing the polyfill solves my problem.

I am manually crafting the requests that I push onto the queue so I looked at manually converting to a Blob before I push them onto the queue:

    const body = await (() => {
      const isPolyfilledFormData = fd._blob && typeof fd._blob === 'function'
      if (isPolyfilledFormData) {
        return fd._blob()
      }
      // use native Response to generate blob
      return new Response(fd).blob()
    })()

If this worked, it would mean I could use the this package as-is without a patch but it doesn't seem viable because the native Request/Response with FormData in Safari 13.1 cannot convert to blob either. Basically all of these fail:

(async function() {await new Response(new FormData()).blob()})();
(async function() {await new Response(new FormData()).arrayBuffer()})();
(async function() {await new Request('z', {method: 'POST', body: new FormData()}).blob()})();
(async function() {await new Request('z', {method: 'POST', body: new FormData()}).arrayBuffer()})();

Like you say, selectively polyfilling just the use of FormData and not the object itself, so we still have the _blob() function and the code that calls it, is a good solution. I'm still trying to fully understand what you've propose. I'll post this comment and go have a play with a patch that uses your suggestions.

[tomsaleeba]

Here's a proof of concept using Symbol.hasInstance.

I've assumed that the _blob function would be lifted out of the polyfill so it can be called from the polyfilled FormData or run against the native Safari FormData.

I'm also not sure how this will be affected by your linked issue: https://github.com/babel/babel/issues/1966.

<!DOCTYPE html>
<html>

<body>
  <script charset="utf-8">
    function _blob() {
      // the extracted polyfill function
      return '(polyfill blob result)'
    }

    const NativeRequest = globalThis.Request
    globalThis.Request = class Request extends NativeRequest {
      constructor(input, init, ...x) {
        super(input, init, ...x)
        this._init = init
      }

      static[Symbol.hasInstance](instance) {
        return [NativeRequest, Request].includes(instance.constructor)
      }

      async blob() {
        const body = (this._init || {}).body
        if (body) {
          const isPolyfilled = body._blob && typeof body._blob === 'function'
          if (isPolyfilled) {
            return body._blob()
          }
          const isNativeOperationSupported = await (async () => {
            try {
              // TODO could just try `return super.blob()` and catch the failure with the polyfill
              await new NativeRequest('z', {
                method: 'POST',
                body: new FormData() // needs to refer to native implementation
              }).blob()
              return true
            } catch (err) {
              return false
            }
          })()
          if (!isNativeOperationSupported) {
            return _blob.apply(body)
          }
        }
        return super.blob()
      }
    }
    async function runChecks() {
      const fd = new FormData()
      fd.append('foo', 'bar')
      const r1 = new Request('localhost', {
        method: 'POST',
        body: fd
      })
      const r2 = new NativeRequest('a')
      console.log('Check instanceof works for everything', r1 instanceof Request, r2 instanceof Request)

      const nativeResult = await r1.blob()
      console.log('Check native blob() works', nativeResult)

      // simulate Safari 13.1 behaviour (assumes you're not running in Safari/webkit)
      NativeRequest.prototype.blob = function() {
        throw new Error('Forced "operation not supported behaviour"')
      }
      const r3 = new Request('localhost', {
        method: 'POST',
        body: fd
      })
      const partialPolyfillResult = await r3.blob()
      console.log('Check native FormData but polyfilled blob() works',
        partialPolyfillResult)

      // simulate fully polyfilled behaviour
      fd._blob = _blob
      const r4 = new Request('localhost', {
        method: 'POST',
        body: fd
      })
      const fullPolyfillResult = await r4.blob()
      console.log('Check fully polyfilled FormData works',
        fullPolyfillResult)
    }

    runChecks()
  </script>
</body>

</html>

[jimmywarting]

I'm also not sure how this will be affected by your linked issue: babel/babel#1966.

We can't extend native classes since it gets compiled down to es5 (prototyping with functions)

so super becomes transpiled down to Request.apply(Request, arguments) which gives you the error Uncaught TypeError: Failed to construct 'Request': Please use the 'new' operator, this DOM object constructor cannot be called as a function.

async blob() {

I'm not sure we should modify the behavior of arrayBuffer, blob or json. (at least not with the way you have suggested)

the importance is that fetch can make a request with the constructed Request you have given (without service worker intervening). Most of our user don't even use service worker. So i believe it's important to handle the body conversion (FormDataPolyfill to Blob) in the constructor instead.

So it would be good if you also tried some actual test with fetch(new Request(...)) without service worker also

Here is my attempt at some start of it:

globalThis.Request = globalThis.Request && (NativeRequest => {
  function Request(input, init) {
    if (init && init.body && init.body instanceof FormDataPolyfill) {
      init.body = init.body._blob()
    }

    if (arguments.length === 0) return new NativeRequest()
    if (arguments.length === 1) return new NativeRequest(input)
    if (arguments.length > 1) return new NativeRequest(input, init)
  }

  // deligate the instanceof check to check if
  // it compares to nativeRequest instead
  Object.defineProperty(Request, Symbol.hasInstance, {
    value: function (instance) {
      // no need to use [].includes since new Request() always return NativeRequest 
      return instance instanceof NativeRequest
    }
  })

  return Request
})(globalThis.Request)

This will make it so that fetch(new Request(...)) works and the kind of body can never be a instance of FormDataPolyfill which makes your async blob() { function obsolete since it will be converted to a Blob in the constructor

[tomsaleeba]

Edit: update link to test rig code Edit2: update links to fork and test rig code after giving it a good tidy up

For my use case, I still need an extra check in there. I should've linked to what I'm doing initially but better late than never. When I say I'm using a Service Worker, it's not just an out-of-the-box one like most people would use. I've got logic in the SW that I've coded up.

Here is my code where I'm expecting this polyfill to help me: https://github.com/ternandsparrow/wild-orchid-watch-pwa/blob/940e765a173225cff38a8dbc2047671107f2ecef/sw-src/sw.js#L1120

So the order of events is something like

1. page loads in Safari 13.1
2. formdata-polyfill checks if it should load. The FormData global is present so it does not load
3. ...user uses page and eventually performs even that triggers SW code
4. in the SW, create a Request with a FormData body
5. the Request is pushed onto the Workbox background-sync queue
6. Workbox serialises the Request using .arrayBuffer() so it can be stored in IndexedDB. This is where the error is thrown because the polyfill is not being used and Safari cannot do Request.arrayBuffer() with a FormData body.

Note that fetch has not been called yet. So I need to intercept either the Request constructor or the .arrayBuffer call.

I've updated my fork with the changes that I think are needed, you can find the commit here: https://github.com/ternandsparrow/FormData/commit/bca472678ebcc98a8e646ca913e268756fdf38fb.

I've put this code into my test rig that you can find at: https://gitlab.com/tomsaleeba/fiddles/-/tree/59fe13fd7aaab57dde70e3c8cf2964de98c60357/safari-capabilities.

Some notes about my fork so far:

I'm not super happy that the Request and Response constructor polyfills have part of their functionality loaded async but I can't think of a non-async way to do it. I've expose a Promise that can be awaited if it's a big problem. Plus, using the Request/Response constructor on page load hopefully isn't a common use case, although I do this in my test rig.

I can't move that async logic for checking if we need the Request polyfill into the Request constructor because the constructor can't be async. I've run this test rig in Chrome, Safari 13.1 (the troublemaker for me) and Safari 13.0 and it works as expected everywhere.

I haven't introduced any new linting errors into the code but with standardjs 14.3.4, there are a number of lint errors.

I don't have any unit tests yet. I've tried but it's harder than I expected to control when the Request/Response constructors are polyfilled.

[tomsaleeba]

As some brief feedback, I've been running my fork in production for a few weeks now and it's working great.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tomsaleeba]

I've put a PR together for this: https://github.com/jimmywarting/FormData/pull/113

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jimmywarting]

FYI: new Response(new FormData(formElm)) dose work in safari but not new Response(new FormData())

The solution is simple really you need at least one field in the formData - i reported this as a bug to wpt, chrome & safari that you can't decode empty FormData instances