Fix Blob Compatibility

[atrauzzi]

Introduction

I recently started using React Native for an application initiative at current job. During some web-based front end work, I created some internal libraries to standardize our data access. These libraries make use of axios to send requests and thus far have presented no issues when working with sending & slicing Blob instances from browsers. I know this logic is good because it's currently deployed and we receive files from web browsers running it daily.

Given that React Native - after a fashion - builds on top of the JS ecosystem, this seemed like a great opportunity to reuse our data access library as we begin work building our app! 🎉

The Core of It

Unfortunately, one of our internal libraries which makes direct use of axios to send binary data to signed S3 bucket URIs failed on React Native. Here's the salient bit from our internal library:

const partBytes = data.slice(start, start + size);

if (!partBytes.size) {

    throw new Error(`Upload of ${upload.id} failed, empty chunk encountered ${index}.`);
}

const request = axios.put(presignedUploadUri, partBytes, {
    onUploadProgress: (progressEvent: any) => {

        upload.parts[index].progress = Math.round((progressEvent.loaded * 100) / progressEvent.total);

        this.reportProgress(upload, callbacks);
    },
    headers: {
        "content-type": "binary/octet-stream",
    },
});

You can see in the example above that I'm using the standard Blob API to slice up the file and am then passing the result to axios to ship to S3. Unfortunately, when the blob being sent to the library originates from React Native, instead of sending bytes to the destination, I end up sending a string body that looks like this:

{
    "_data": {
        "blobId": "9fe44032-4518-4eba-9373-c759ce8daa06",
        "offset": 0,
        "size": 30548
    }
}

For people familiar with React Native internals, this is probably enough of a description, I have some suspicions about what's going on here. It mostly pertains to "blobs" being predicated on originating from fetch and only ever being sent back to fetch, resulting in a cloistered implementation.

That said, I have taken the time to put together a demonstration repository so as to avoid any concern over nuance.

This is an issue for me because I now have to duplicate my binary upload routine inside of my React Native project. I presume because I need a specialized fetch implementation to recognize these specialized pseudo-blob instances. This also concerns me as pure JS libraries (maybe also native libraries) won't be able to depend on a standard Blob implementation for byte-level manipulations.

Discussion Points

• Implementing blobs properly - such that they can be used between libraries
• Checks & balances during React Native development to ensure that any bias towards fast paced or high profile items doesn't drown out slow moving concerns like web standards support
• How did base64 strings get a pass for a platform that targets power and performance sensitive devices?

Bonus Meta Discussion Point

I've opened this discussion as per a kind response I received during a recent AUA. I first want to say, thank you so much for the invitation, it made me feel comfortable enough to raise this issue one more time. 😄

As a knock-on though, I would like to be serious for just a bit and point out that I was pretty much ready to throw in the towel on this issue.

Communication and triage around this topic has not been the greatest. Despite me attempting multiple times in multiple channels to raise the topic on top of the fact that I'm a newcomer to React Native. There's no documentation that clearly outlines the ability to obtain references to files on-device as blobs. I actually had to make two posts about it during the AUA because the first one got glossed over, very much in the same form as all my other attempts to raise awareness. I even tried on the reactiflux discord.

I've seen a lot of chatter about community involvement. But in this paragraph, I have mentioned github, twitter, reddit and discord as the platforms I've attempted to reach out on. Each time, to no response.

The tldr; of this next paragraph is that I'm quite certain a loose initial description from me way back when I first encountered the problem should have been enough for someone on the React Native team to go "oh yeap, I think I see the issue you're having".

I think the React Native team needs to graduate from the more performative and cuddly aspects of communication and transparency and now start looking at what it will take to be proactive and qualitative about the initiative. You'll need a cohesive strategy for both identifying and prioritizing the deeper issues that can sometimes arise in the realm of platform work. Most of all though, you can't possibly expect all of your users who encounter real issues to go to the numerous extents I have here. I've spent the whole day collating this information and preparing a demo repository - not because this issue isn't easily intuited by React Native maintainers - but to simply put it in such incontrovertible terms that it's more difficult to wave off. Someone already did the legwork to spot an issue and that wasn't enough to indicate internally that there might be some problematic design choices made in the blob implementation.

Quips about how hard it is to be a maintainer aside: It's just too much if this is what it takes for your users to get noticed!

I would finally like to add that I wholly recognize and appreciate the effort of the maintainers, Facebook and pretty much everything along the way to the software engineering marvel that is React Native.

Seriously, much love. ❤️

(I reserve the right to edit this or add entire missing paragraphs, I'm exhausted!)

[satya164]

Assigning myself so I remember to read it later

[TheSavior]

Thanks for writing this up. I've heard and recognized that blob support is something that needs to be worked on and made more consistent with web.

I'm not super familiar with how fetch in React Native works and all the problems with Blob, but one of the things you mentioned was that React Native serializes the chunks to JSON in order to pass them across the bridge. For blob to work, we can't do this anymore, right?

My understanding is that what we really need is the ability to pass complete objects from native to JavaScript without serializing. I believe Blob support is implemented natively in the browser, as shown by this simple example:

> Blob.toString()
"function Blob() { [native code] }"

In React Native land, Blob support would need to be implemented in native code to support everything that the web has. However, today there is no way to pass data between Native and JS without going across the bridge which requires serialization.

The solution to this serialization problem is JSI, which would allow us to create an object in native and directly hand JavaScript a reference to that object.

Does that sound like my understanding in what is required to sufficiently solve this problem with blobs is accurate?

If so, all the infrastructure changes we are making to take advantage of JSI and move away from a serialized bridge feels like higher priority work that needs to happen first to enable solutions for Blobs and other related issues. More specifically, the work on TurboModules seems like it will enable a solution for this.

Does this seem like it is consistent with your understanding of the problem space?

Overall, I guess I've been hearing these problems for a while, but it hasn't felt actionable to me because AFAIK the current architecture hinders our ability to do this successfully, so our focus on infrastructure has felt right. If my understanding is correct, then we probably could have at least had this conversation months ago to be more transparent. Although, honestly it felt like a trickle of interest and complaints here and there until this post which feels like a more reasonable home for such a discussion.

[satya164]

I am on phone so didn't read all of it, but I did the Android implementation of Blobs. It is implemented natively and there's no serialisation. The API is also consistent with web. There are missing stuff and one difference from web, but it doesn't deviate otherwise.

Though it seems the implementation broke by recent changes in React Native (not sure exactly what).

[raRaRa]

Any update on this? I've been trying to upload a file/blob to S3 and other services without luck in React native.

[alanrivetta]

Any notice?

[luisfuertes]

Any update about this? Need upload videos to S3 too

[shaatoot]

any updates? in dire need of the Blob implementation...

[evelant]

@satya164 did you ever get a chance to look into this? 5 years later and I think it's still an issue. If you don't have time to (or don't want to) look into it maybe unassign yourself so people aren't thinking that this is an active priority for someone.