Add ffmpeg

[SaadBazaz]

Hey, Just wanted to say; this is exactly what I was looking for and thanks for building it, guys.

Is your feature request related to a problem? Please describe. Ffmpeg is an extremely common wasm usecase, and given that the official ffmpeg.wasm port does not support server-side runs, I think it's an awesome addition to Wick.

Propose a solution A simple ffmpeg.wasm port which can run on a NodeJS server.

Describe alternatives you've considered I've tried making this before, by combining ffmpeg.wasm and fluent-ffmpeg.

types.ts

export interface IFfmpeg {
  isLoaded: () => boolean
  load: () => void
  input: (blob: Blob, name?: string) => void
  run: (command: string[]) => void
  destroy: () => void
}

client.ts

import { FFmpeg } from '@ffmpeg/ffmpeg'
import { type IFfmpeg } from './types'
import { fetchFile, toBlobURL } from '@ffmpeg/util'

class FfmpegClient implements IFfmpeg {
  ffmpeg: FFmpeg
  baseURL: string
  // isLoaded = false;

  constructor() {
    this.ffmpeg = new FFmpeg()
    this.baseURL = 'https://unpkg.com/@ffmpeg/core@0.12.4/dist/umd'
    this.ffmpeg.on('log', ({ message }) => {
      console.log(message)
    })
    if (!this.isLoaded()) {
      this.load()
    }
  }

  isLoaded = () => {
    return this.ffmpeg.loaded
  }

  load = async () => {
    await this.ffmpeg.load({
      coreURL: await toBlobURL(
        `${this.baseURL}/ffmpeg-core.js`,
        'text/javascript'
      ),
      wasmURL: await toBlobURL(
        `${this.baseURL}/ffmpeg-core.wasm`,
        'application/wasm'
      )
    })
  }

  input = async (blob: Blob | File | string, name = 'input.webm') => {
    await this.ffmpeg.writeFile(name, await fetchFile(blob))
  }

  run = async (command: string[]) => {
    await this.ffmpeg.exec(command)
  }

  output = async (name = 'output.mp4') => {
    return await this.ffmpeg.readFile(name)
  }

  destroy: () => void
}

export default FfmpegClient

Something similar for server.ts, but with fluent-ffmpeg.

It's not exactly what Wick needs, but I hope it helps.

Additional context It's expected to have better performance than the browser, but who knows.

[jsoverson]

That sounds like a reasonable component.

What's your use case so I can experiment with that one first?

[SaadBazaz]

That sounds like a reasonable component.

What's your use case so I can experiment with that one first?

Creating a NextJS app where I can allow the user to render on the client, or the server. Based on their own preference of file.

[jsoverson]

ffmpeg to RSocket-wasm probably won't happen before a component model implementation pops up, so we'll have to defer this for now.

ffmpeg is too big to dig into as an outsider. Once standard WASM component model implementations start spreading, it'll be an easier task for project maintainers to adapt existing code from arbitrary languages to componentized WebAssembly.

[SaadBazaz]

@jsoverson ffmpeg has already been compiled to WASM: https://github.com/ffmpegwasm/ffmpeg.wasm

Is it possible to use that?

[jsoverson]

ffmpeg has already been compiled to WASM: https://github.com/ffmpegwasm/ffmpeg.wasm

Is it possible to use that?

It means it's possible. A lot of the hard work has been done. But it would need someone more familiar with ffmpeg to take it the last mile. I don't know enough to hack on it. I can certainly help with the wick side though.