larsgw / sync-fetch

Synchronous wrapper around the Fetch API
MIT License
37 stars 12 forks source link

Check out `await-sync` - replace child process with WebWorkers #36

Open jimmywarting opened 1 year ago

jimmywarting commented 1 year ago

I just created this async-to-sync package today that i named await-sync

it utilize web workers instead of spawning sync processes and data is transfered synchronous over SharedArrayBuffer so less code needs to be copied over.

Therefore it's also more compatible with other enviorments like Deno, Bun, and also Web Workers.

it's as simple as just doing:

import { createWorker } from 'to-sync'

const awaitSync = createWorker()

const fetchSync = awaitSync(async function (...args) {
  const res = await fetch(...args)
  const ab = await res.arrayBuffer()
  return new Uint8Array(ab)
})

const uint8 = fetchSync(url)
const text = new TextDecoder().decode(uint8)
const json = JSON.parse(text)
const blob = new Blob([uint8])

if you would use this instead then your package would have the possibility of being able to run in Deno, Bun.js and also Web Workers. but why web workers when you sync xhr...? that's a good question...

larsgw commented 1 year ago

I think someone else might have mentioned web workers before. I haven't used Deno or Bun but I'm not against supporting that. I think the main doubt I had was the same as what you mention:

but why web workers when you sync xhr...? that's a good question...

But there's actually a clear answer to that: sync xhr has a bunch of limitations, including not allowing binary responses and (at least in the past) some cryptic, undocumented CORS restrictions.

jimmywarting commented 1 year ago

including not allowing binary responses

Oh, that brings me back in the memory lane... Remembered trying out to fetch some pdf or some kind of binary file and had problem with that.

And now i remembered that i also wanted to try and read Blob's in a sync mannar also in the main thread in browsers too.

i even have a old gist for that.

const blob = new Blob(['123'])
const xhr = new XMLHttpRequest()
const url = URL.createObjectURL(blob)

// required if you need to read binary data:
xhr.overrideMimeType('text/plain; charset=x-user-defined') 
xhr.open('GET', url, false)
xhr.send()

const uint8 = Uint8Array.from(xhr.response, c => c.charCodeAt(0))
const text = new TextDecoder().decode(uint8)
const arrayBuffer = uint8.buffer
const json = JSON.parse(text)

the trick is to override mime type and use a charset that isn't recognised. then xhr won't try to be smart and change the response, so you will actually get back the raw bytes. but then you can't use responseType but you will be able create those later more manually

jimmywarting commented 1 year ago

ironically MDN says it's bad to use sync

This restriction is designed in part to help ensure that synchronous operations aren't used for large transactions

And yet they have a tutorial on how to get around it here: Receiving binary data in older browsers

overriding the MIME type, force the browser to treat it as plain text, using a user-defined character set. This tells the browser not to parse it, and to let the bytes pass through unprocessed.

larsgw commented 1 year ago

I've implemented the trick for binary data from MDN that you mentioned, it's released in v0.4.4.