xponential-asia / medusa-plugin-file-cloud-storage

Medusa.js plugin to store files in Google Cloud Storage
MIT License
5 stars 2 forks source link

Fix ENOENT error in file upload stream handling #41

Open YousefRabieKhalil opened 3 months ago

YousefRabieKhalil commented 3 months ago

Issue Description

When attempting to upload files using our current implementation with Google Cloud Storage, we're encountering ENOENT (No such file or directory) errors. This is likely due to an attempt to read from a non-existent file during the upload process.

Current Behavior

The current implementation in the getUploadStreamDescriptor method attempts to read from a local file that doesn't exist:

const pipe = fs
  .createReadStream(destination)
  .pipe(file.createWriteStream());

This results in an ENOENT error as the file hasn't been created yet.

Expected Behavior

The upload process should stream data directly to Google Cloud Storage without attempting to read from a local file.

Proposed Solution

  1. Remove the fs.createReadStream() call.
  2. Utilize the PassThrough stream (pass) by piping it to the Google Cloud Storage write stream.
  3. Update the promise to resolve when the write to Google Cloud Storage is finished.
  4. Modify URL setting to occur after the file has been successfully uploaded.

Here's a proposed change to the getUploadStreamDescriptor method:

async getUploadStreamDescriptor(
  fileData: UploadStreamDescriptorType,
): Promise<FileServiceGetUploadStreamResult> {
  try {
    // ... (existing code for key, parsedFile, etc.)

    const file = this.privateStorage_
      .bucket(isPrivate ? this.privateBucketName_ : this.publicBucketName_)
      .file(destination);

    // Create a write stream to Google Cloud Storage
    const writeStream = file.createWriteStream();

    // Pipe the PassThrough stream to the Google Cloud Storage write stream
    pass.pipe(writeStream);

    const promise = new Promise((resolve, reject) => {
      writeStream.on('finish', resolve);
      writeStream.on('error', reject);
    });

    // Get the URL after the file has been created
    promise.then(async () => {
      if (isPrivate) {
        url = file.cloudStorageURI.href;
      } else {
        url = await file.publicUrl();
        url = this.transformGoogleCloudURLtoCDN(url);
      }
    });

    return {
      writeStream: pass,
      promise,
      url,
      fileKey: destination,
    };
  } catch (error) {
    throw new MedusaError(
      MedusaError.Types.UNEXPECTED_STATE,
      error?.message || 'Upload file stream error.',
    );
  }
}

Additional Context

This issue is critical for the proper functioning of file uploads in our application. Resolving it will prevent ENOENT errors and ensure smooth file uploads to Google Cloud Storage.

Steps to Reproduce

  1. Attempt to export order, products from the mdusa admin panel while the plugin is installed.
  2. Observe the ENOENT error in the logs.
VasantSachdewa commented 3 months ago

@YousefRabieKhalil glad to hear you are using the library in your application, I will have the team look into it and update you!