Need help converting a video file to MP3 and uploading to AWS S3

[biigpongsatorn]

Hi,

I'm trying to write a Node.js code that stream downloads a video file from a URL and then stream converts it to an MP3 file, and then stream uploads the MP3 file to an AWS S3 bucket. I'm using the axios library to download the file and the aws-sdk library to upload the file to S3.

For the conversion part, I'm trying to use the node-fluent-ffmpeg library, but I'm running into some issues. Here's the code I have so far:

Version information

• fluent-ffmpeg version: 2.1.2
• ffmpeg version: 6.1.1
• OS: MacOS 13.6.1

Code to reproduce

s3StreamClient.ts

import {
  AbortMultipartUploadCommandOutput,
  CompleteMultipartUploadCommandOutput,
  S3Client,
} from '@aws-sdk/client-s3';
import { Upload } from '@aws-sdk/lib-storage';
import Axios, { AxiosResponse } from 'axios';
import { PassThrough } from 'stream';

// eslint-disable-next-line @typescript-eslint/no-var-requires
const ffmpeg = require('fluent-ffmpeg');

export class S3StreamClient {
  private readonly PART_SIZE = 1024 * 1024 * 5; // 5 MB
  private readonly CONCURRENCY = 4;

  private readonly client: S3Client;

  constructor(props: { sdkClient: S3Client }) {
    this.client = props.sdkClient;
  }

  async uploadVideo(props: {
    input: {
      url: string;
    };
    output: {
      bucketName: string;
      key: string;
    };
  }): Promise<string> {
    try {
      const inputStream = await this.getInputStream({
        url: props.input.url,
      });
      const outputFileRelativePath = props.output.key;
      await this.getOutputStream({
        inputStream,
        output: {
          ...props.output,
          key: outputFileRelativePath,
        },
      });
      return `s3://${props.output.bucketName}/${outputFileRelativePath}`;
    } catch (error) {
      console.error(
        { error },
        'Error occurred while uploading/downloading file.',
      );
      throw error;
    }
  }

  private async getInputStream(props: { url: string }): Promise<AxiosResponse> {
    const response = await Axios({
      method: 'get',
      url: props.url,
      responseType: 'stream',
    });

    return response;
  }

  private async getOutputStream(props: {
    inputStream: AxiosResponse;
    output: {
      bucketName: string;
      key: string;
    };
  }): Promise<
    CompleteMultipartUploadCommandOutput | AbortMultipartUploadCommandOutput
  > {
    const output = props.output;
    const passThrough = new PassThrough();
    const upload = new Upload({
      client: this.client,
      params: { Bucket: output.bucketName, Key: output.key, Body: passThrough },
      queueSize: this.CONCURRENCY,
      partSize: this.PART_SIZE,
      leavePartsOnError: false,
    });

    ffmpeg(props.inputStream).toFormat('mp3').pipe(passThrough, { end: true });

    return await upload.done();
  }
}

main.ts

const key = `directory1/key-1.mp3`;
const bucket = 'mybucket';
const URL = 'https://large-size-video.mp4'
await this.s3StreamClient.uploadVideo({
      input: { url },
      output: { bucketName: bucket, key },
});

(note: if the problem only happens with some inputs, include a link to such an input file)

Error

Error: ffmpeg exited with code 234: Error opening output file pipe:1.
Error opening output files: Invalid argument

Checklist

• [x] I have read the FAQ
• [x] I tried the same with command line ffmpeg and it works correctly (hint: if the problem also happens this way, this is an ffmpeg problem and you're not reporting it to the right place)
• [x] I have included full stderr/stdout output from ffmpeg

[biigpongsatorn]

I finally found out the solution! And what I miss is I have to add input and output format

.inputFormat('mp4')
.outputFormat('mp3')

example

private async getOutputStream(props: {
    inputStream: AxiosResponse;
    output: {
      bucketName: string;
      key: string;
    };
  }): Promise<
    CompleteMultipartUploadCommandOutput | AbortMultipartUploadCommandOutput
  > {
    const output = props.output;
    const passThrough = new PassThrough();
    const upload = new Upload({
      client: this.client,
      params: { Bucket: output.bucketName, Key: output.key, Body: passThrough },
      queueSize: this.CONCURRENCY,
      partSize: this.PART_SIZE,
      leavePartsOnError: false,
    });

    // Convert vdo to mp3 and upload to s3
    ffmpeg(props.inputStream.data)
      .inputFormat('mp4')
      .outputFormat('mp3')
      .on('start', () => {
        console.log('Processing....');
      })
      .on('error', (err) => {
        console.log('An error occurred: ' + err.message);
      })
      .on('end', () => {
        console.log('Processing finished !');
      })
      .pipe(passThrough, { end: true });
    return await upload.done();
  }