search

Azure / azure-storage-js

Microsoft Azure Storage Library for JavaScript (This repo has been moved to https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/storage on Apr18, 2019. This Repo will be deprecated.)
https://docs.microsoft.com/en-us/javascript/api/overview/azure/storage/client?view=azure-node-preview
MIT License
74 stars 34 forks source link

Failing to upload using `uploadStreamToBlockBlob` #64

Open Connorelsea opened 5 years ago

Connorelsea commented 5 years ago

Which service(blob, file, queue, table) does this issue concern?

blob

Which version of the SDK was used?

@azure/storage-blob ^10.3.0

What's the Node.js/Browser version?

node v9.10.1

What problem was encountered?

Attempting to upload node file stream as blob using uploadStreamToBlockBlob and it fails, producing confusing error that I've been unable to pinpoint the cause of.

Steps to reproduce the issue?

I created a resolver function for apollo-server 2, where the resolver receives a file stream and then attempts to upload it as a new blob in a container named 'avatars'.

Resolver code

const uploadAvatar = isAuthenticatedResolver.createResolver(
  async (root, args, context, error) => {
    const { file } = args;
    const { stream, filename, mimetype, encoding } = await file;

    const account = "remix2";
    const accountKey =
      "redacted";

    // Use SharedKeyCredential with storage account and account key
    const sharedKeyCredential = new SharedKeyCredential(account, accountKey);

    // Use sharedKeyCredential, tokenCredential or anonymousCredential to create a pipeline
    const pipeline = StorageURL.newPipeline(sharedKeyCredential);

    const serviceURL = new ServiceURL(
      // When using AnonymousCredential, following url should include a valid SAS or support public access
      `https://${account}.blob.core.windows.net`,
      pipeline
    );

    const containerName = "avatars";
    const containerURL = ContainerURL.fromServiceURL(serviceURL, containerName);

    const blobName = "newblob" + new Date().getTime();
    const blobURL = BlobURL.fromContainerURL(containerURL, blobName);
    const blockBlobURL = BlockBlobURL.fromBlobURL(blobURL);
    const uploadBlobResponse = await uploadStreamToBlockBlob(
      Aborter.timeout(30 * 60 * 60 * 1000),
      stream,
      blockBlobURL,
      20
    );
    console.log(
      `Upload block blob ${blobName} successfully`,
      uploadBlobResponse
    );
  }
);

Error code

TypeError: Cannot read property 'on' of undefined
    at /Users/connorelsea/Projects/remix-server/node_modules/@azure/storage-blob/dist-esm/lib/utils/BufferScheduler.js:156:40
    at new Promise (<anonymous>)
    at e.<anonymous> (/Users/connorelsea/Projects/remix-server/node_modules/@azure/storage-blob/dist-esm/lib/utils/BufferScheduler.js:153:25)
    at /Users/connorelsea/Projects/remix-server/node_modules/@azure/storage-blob/node_modules/tslib/tslib.es6.js:97:23
    at Object.next (/Users/connorelsea/Projects/remix-server/node_modules/@azure/storage-blob/node_modules/tslib/tslib.es6.js:78:53)
    at /Users/connorelsea/Projects/remix-server/node_modules/@azure/storage-blob/node_modules/tslib/tslib.es6.js:71:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/connorelsea/Projects/remix-server/node_modules/@azure/storage-blob/node_modules/tslib/tslib.es6.js:67:12)
    at e.do (/Users/connorelsea/Projects/remix-server/node_modules/@azure/storage-blob/dist-esm/lib/utils/BufferScheduler.js:140:14)
    at /Users/connorelsea/Projects/remix-server/node_modules/@azure/storage-blob/dist-esm/lib/highlevel.node.js:304:58
    at /Users/connorelsea/Projects/remix-server/node_modules/@azure/storage-blob/node_modules/tslib/tslib.es6.js:97:23
    at Object.next (/Users/connorelsea/Projects/remix-server/node_modules/@azure/storage-blob/node_modules/tslib/tslib.es6.js:78:53)
    at /Users/connorelsea/Projects/remix-server/node_modules/@azure/storage-blob/node_modules/tslib/tslib.es6.js:71:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/connorelsea/Projects/remix-server/node_modules/@azure/storage-blob/node_modules/tslib/tslib.es6.js:67:12)
    at uploadStreamToBlockBlob (/Users/connorelsea/Projects/remix-server/node_modules/@azure/storage-blob/dist-esm/lib/highlevel.node.js:271:9)
    at _callee3$ (/Users/connorelsea/Projects/remix-server/resolvers/storage.js:179:38)
    at tryCatch (/Users/connorelsea/Projects/remix-server/node_modules/regenerator-runtime/runtime.js:65:40)
    at Generator.invoke [as _invoke] (/Users/connorelsea/Projects/remix-server/node_modules/regenerator-runtime/runtime.js:303:22)
    at Generator.prototype.(anonymous function) [as next] (/Users/connorelsea/Projects/remix-server/node_modules/regenerator-runtime/runtime.js:117:21)
    at step (/Users/connorelsea/Projects/remix-server/resolvers/storage.js:25:191)
    at /Users/connorelsea/Projects/remix-server/resolvers/storage.js:25:361

Have you found a mitigation/solution?

Not yet, reading through the code that it points to now to try to understand what is causing it, but hoping someone here might have an idea of what I'm doing wrong.

romitgirdhar commented 5 years ago

I am facing the same exact issue. Has anybody found a solution to this?

romitgirdhar commented 5 years ago

I think I might've figured it out. Make sure that your stream is an actual Readable stream. That was my problem at least. Once I created a stream using `var stream = require('stream'), and then passed that stream onto the function, it worked. I followed this SO post to get help with creating a proper stream: https://stackoverflow.com/questions/48534404/create-readstream-from-base64-encoded-string-by-file

XiaoningLiu commented 5 years ago

Yes, please make sure the stream is a valid Node.js Readable stream.

Connorelsea commented 5 years ago

Thanks for the tip, that helped me identify the issue and fix some other related issues.

For me const { stream, filename, mimetype, encoding } = await file; only returns { filename: 'profilepic.jpeg' }, so the stream is null. But I'm not sure why the stream is null but the filename isn't. I am trying to investigate now but may still need help. I will post more info here if I find something out or post what was wrong if I figure it out

Connorelsea commented 5 years ago

On the frontend, using apollo-client, I am attempting to use an upload mutation. I am passing a Blob object as the file variable. I was able to write this Blob object to my local filesystem using FileReader, and it looked visually correct, so I'm unsure what is causing the stream to be undefined on the backend.

Frontend Blob object:

image

Frontend mutation code:

    const src = gql`
      mutation uploadAvatar($file: Upload!) {
        uploadAvatar(file: $file) {
          filename
        }
      }
    `;

    const response = await mutate(src, { file: cropBlob }, apolloClient);

Backend mutation schema:

uploadAvatar(file: Upload!): File!
XiaoningLiu commented 5 years ago

Will close this issue as this is not a SDK issue, it sounds like a logical error needs your detailed investigation, sorry I cannot give more suggestions about this. Fell free to re-open if you have further questions about SDK.