Implemetation Of Resumable Upload Via Reactjs And Nodejs

[MOIN-AKHTAR]

Hello everyone i'm using gcp bucket for resumable upload i have created an endpoint on backend whihc will give me a signedUrl. BACKEND CODE (Nodejs with nestjs port 8080):-

async getSignedUrl(query, res) {
    try {
      res.set('Access-Control-Allow-Origin', 'http://localhost:3000/');
      const [url] = await storage
        .bucket('my-bucket')
        .file(query.fileName)
        .getSignedUrl({
          action: 'resumable',
          version: 'v4',
          expires: Date.now() + 12 * 60 * 60 * 1000,
          contentType: 'application/octet-stream',
        });
      return res.json({
        url,
      });
    } catch (error) {
      throw new HttpException(
        error.message || error.response || error,
        HttpStatus.BAD_REQUEST,
      );
    }
  }

Now i want to use signedUrl given by this endpoint on front end for resumable upload.Now inorder to used it as resumable according to documetation we have to hit a post request using this signedUrl that will give us SessionUrl this session url will be used for resumable upload but when i'm hitting post request using signedUrl which i got from backed refer to above code i'm getting this error. FRONT END CODE (React App running on port 3000):-

const uploadChunk = useCallback(
    (readerEvent) => {
      axios({
        method: "POST",
        url, ===> Reffering signedUrl which we got from backend code .
      })
        .then((res) => {
        //Expecting here sessionUrl for resumable upload....
          console.log(res.data);
        })
        .catch((err) => {
          console.log(err);
        });
    },
    [url]
  );

Access to XMLHttpRequest at 'https://storage.googleapis.com/blahblahbla...' from origin 'http://localhost:3000/' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. Than using gsutils my cors config is this:

[{"maxAgeSeconds": 3600, "method": ["GET", "POST", "PUT", "DELETE", "HEAD", "OPTIONS"], "origin": ["http://localhost:3000/"], "responseHeader": ["Content-Type", "Authorization", "x-goog-resumable", "Access-Control-Allow-Origin"]}]

I'm using :
"@google-cloud/storage": "^6.1.0"

I'm following this: https://cloud.google.com/storage/docs/access-control/signed-urls#signing-resumable Than why i'm getting this error any idea please

[ddelgrosso1]

Hi @MOIN-AKHTAR thank you for the question. From my initial look through the information you provided it appears the CORS config on the bucket is ok. When you are making the POST request to the signed URL are you adding the header X-Goog-Resumable: start?

[MOIN-AKHTAR]

Hi @ddelgrosso1 i'm getting 201 when i am setting these headers:

axios({
        method: "POST",
        url,
        headers: {
          "Access-Control-Allow-Origin": "http://localhost:3000",
          "Content-Type": "application/octet-stream",
          "X-Goog-Resumable": "start",
        },
        data,  ====> Chunk Of File Which is about 256 * 1024;
      })
        .then((res) => {
          console.log(res);
        })
        .catch((err) => {
          console.log(err);
        });

But i'm not getting sessionUrl unfortunatley. If i miss any header mention above than i'm getting CORS issue.Now why i'm getting 201 and not 200 according to docs if response is 200 we i'll get sessionUrl which will be used in next frequesnt request.

[MOIN-AKHTAR]

@ddelgrosso1 do you have complete front end and backend example please in official docs i can;t see anything like that?

[ddelgrosso1]

When you get a 201 response what does the response / response body look like?

[MOIN-AKHTAR]

[MOIN-AKHTAR]

@ddelgrosso1 This is what i'm getting data:"" but statusCode=201. Not a single sign of sessionUrl unfortunately.

[ddelgrosso1]

From your image I can't see all the returned headers. Is there a location header on the response you are getting?

[MOIN-AKHTAR]

[MOIN-AKHTAR]

@ddelgrosso1 check this out.

[ddelgrosso1]

Thank you @MOIN-AKHTAR. Per the documentation (4 in the list), the location header is the URI you should utilize to send data to the resumable session.

[MOIN-AKHTAR]

@ddelgrosso1 location key of header?

[ddelgrosso1]

Correct, the location key of headers is the resumable session URI you should utilize in subsequent PUT operations.

[MOIN-AKHTAR]

@ddelgrosso1 ok one more thing in the first post request i have sent 256 1024 chunk of data in first put request i will skip this 256 1024 which i uploaded in post request?And do i have to make put request again and again and what will happen when it will be the last chunk uploaded to gcp bucket what i'll get after back?

[ddelgrosso1]

I believe the reason you are getting a 201 is because you are sending data in the initial POST. I believe the more correct way to do this would be to initiate the resumable session via a POST call without data and use the resumable session URI to PUT the data on subsequent calls.

[MOIN-AKHTAR]

If i'm sending without data im getting CORS issue idk why.

[ddelgrosso1]

If you are sending data in chunks you can follow the documentation here. This should document how to send the appropriate headers / ranges for the chunks.

[MOIN-AKHTAR]

@ddelgrosso1 Refused to set unsafe header "Content-Length"

axios({
          url: url || sessionUrl,
          method: "PUT",
          headers: {
            "Content-Length": chunk.length,
            "Content-Range": `bytes ${firstChunk}-${lastChunk}/${file.size}`,
            "Content-Type": "application/octet-stream",
          },
          data: chunk, ===> Chunk of file
        })
          .then((data) => {
            console.log(data);
          })
          .catch((err) => {
            console.log(err);
          });

If i remove Content-Length header than i'm getting tis error:

TypeError: Failed to execute 'setRequestHeader' on 'XMLHttpRequest': String contains non ISO-8859-1 code point.

How i can resolve this issue try to find out this issue but not got any clear solution yet.Anone please

[ddelgrosso1]

Regarding the TypeError are you attempting to send data with non-iso characters? I found this issue over on the axios github.

[MOIN-AKHTAR]

@ddelgrosso1 Im also getting cors error in put request

[MOIN-AKHTAR]

Honestly documentation is too much confusing not able to resolve issue after 10 days and counting too much annoying.

[MOIN-AKHTAR]

Anyone please have a complete example for this frontend specially anyone i'm not able to get anything tried alot of stuffs and research but unfortunately not getting this work done.It's about 10 days and i'm not able to resolve this issue.Tried to take help from different places but nothing worked yet.

[ddelgrosso1]

I have attempted to recreate this in my local environment. While it is not an exact replica I am not seeing CORS issues. Does something like this work in your environment? Additionally, if you have feedback regarding the documentation I am happy to hear it so that we may improve.

Here are the relevant bits:

CORS Configuration JSON:

[ { "maxAgeSeconds": 3600, "method": ["POST", "PUT"], "origin": ["http://localhost:3000/"], "responseHeader": [ "Content-Type" ] } ]

Creating the signed URL / Session URI / PUT Data:

    const signedUrl = await f.getSignedUrl({
      action: 'resumable',
      version: 'v4',
      expires: Date.now() + 12 * 60 * 60 * 1000,
      contentType: 'image/jpeg',
    });
    const signedUrlToUse = signedUrl[0];
    const result = await axios({
      method: 'POST',
      headers: {
        "X-Goog-Resumable": "start",
        "Content-Type": "image/jpeg"
      },
      url: signedUrlToUse
    });
    const putResult = await axios({
      method: 'PUT',
      url: result.headers.location,
      headers: {
        "Content-Length": `${256 * 1024}`,
        "Content-Range": `bytes 0-${256*1024 - 1} / ${2*256*1024}`
      },
      data: chunk
    });`

[ddelgrosso1]

Hi @MOIN-AKHTAR unfortunately at this point I do not believe this is an issue directly related to the client library and I would be unable to investigate your application specific code. A few things I might suggest to further aid you, try temporarily removing the CORS configuration from the bucket and see if you can get your application to work under those circumstances. I might also suggest trying a single chunk upload first before moving onto the more complex multichunk scenario. Some other resources you might try are StackOverflow or filing a support ticket with GCS.

[MOIN-AKHTAR]

Thank God finally i was able to do it.Here is my solution.. https://github.com/MOIN-AKHTAR/Resumable-Upload-To-GCP/tree/main/client