search

discordjs / discord.js

A powerful JavaScript library for interacting with the Discord API
https://discord.js.org
Apache License 2.0
25.36k stars 3.97k forks source link

Corrupt pcm stream & opus stream never finishes. #4780

Closed GoogleSites closed 3 years ago

GoogleSites commented 4 years ago

Please describe the problem you are having in as much detail as possible: After either piping to a file or to ffmpeg using ffmpeg-fluent, both throw an error where the input data is corrupt/incorrect.

After converting the file to mp3 with the ffmpeg command-line tool, I get a 59KB file that is just static noise.

Using { mode: 'opus' } instead of { mode: 'pcm' } results in the stream never finishing. While the pcm stream is ~200KB once finished, the opus stream is ~8KB, which leads me to believe something isn't working there.

Include a reproducible code sample here, if possible:

import { Client } from 'discord.js';
import fs from 'fs';
import ffmpeg from 'ffmpeg-fluent';

const channelID = '0000000000000';

const client = new Client();

client.once('ready', async () => {
  await client.channels.cache.get(channelID).join();

  console.log('I have joined the channel.');
});

client.on('guildMemberSpeaking', async ({ guild, user, voice }, speaking) => {
  if (user.id === guild.me.id
    || voice?.channelID !== guild.me.voice?.channelID
    || speaking.bitfield === 0) return;

  const stream = guild.voice.connection.receiver.createStream(user.id, { mode: 'pcm' });

  /**
   * Throws: Error: ffmpeg exited with code 1: pipe:0: Invalid data found when processing input
   *
   * Also after piping to user_voice instead...
   * "ffmpeg -i user_voice -f mp3 audio.mp3"
   *
   * Throws: [adp @ 000001f3f668c200] Packet corrupt (stream = 0, dts = NOPTS), dropping it.
   */
  ffmpeg(stream)
    .inputFormat('pcm')
    .outputFormat('mp3')
    .pipe(fs.createWriteStream('user_audio.mp3'));

  stream.once('finish', () => console.log('The stream has finished.'));
});

client.login(process.env.DISCORD_TOKEN);

Further details: Let me know if I did anything wrong - I'm not sure what else to try at this point :)

Relevant client options:

Moebits commented 4 years ago

PCM just contains the audio samples, but ffmpeg needs to know metadata information such as bitrate, sample rate, and channels to convert it to mp3/wav. Here is a working function that I use:

import ffmpeg from "fluent-ffmpeg"

const convertPCM = async (input, dest) => {
    await new Promise((resolve, reject) => {
        ffmpeg(input).inputOptions(["-f", "s16le", "-ar", "48k", "-ac", "2"]).save(dest)
        .on("end", () => resolve())
        .on("error", () => reject())
    })
}

This sets the format to signed 16-bit PCM, sample rate to 48kHz, and channels to 2 (stereo).

GoogleSites commented 4 years ago

PCM just contains the audio samples, but ffmpeg needs to know metadata information such as bitrate, sample rate, and channels to convert it to mp3/wav. Here is a working function that I use:

import ffmpeg from "fluent-ffmpeg"

const convertPCM = async (input dest) => {
    await new Promise((resolve, reject) => {
        ffmpeg(input).inputOptions(["-f", "s16le", "-ar", "48k", "-ac", "2"]).save(dest)
        .on("end", () => resolve())
        .on("error", () => reject())
    })
}

This sets the format to signed 16-bit PCM, sample rate to 48kHz, and channels to 2 (stereo).

ffmpeg automatically retrieves most of that - I've changed to ffmpeg(stream).inputFormat('s16le'); for parsing and pcm streams now work. The issue of opus streams not ending is still an issue though, sadly.

Moebits commented 4 years ago

ffmpeg automatically retrieves most of that - I've changed to ffmpeg(stream).inputFormat('s16le'); for parsing and pcm streams now work. The issue of opus streams not ending is still an issue though, sadly.

My guess is that there is an error (that you aren't catching) so the stream never resolves. I never use opus, so I don't know the ffmpeg command for that. But there is this library prism media that has an opus decoder.

GoogleSites commented 4 years ago

ffmpeg automatically retrieves most of that - I've changed to ffmpeg(stream).inputFormat('s16le'); for parsing and pcm streams now work. The issue of opus streams not ending is still an issue though, sadly.

My guess is that there is an error (that you aren't catching) so the stream never resolves. I never use opus, so I don't know the ffmpeg command for that. But there is this library prism media that has an opus decoder.

The opus stream not finishing is unrelated to ffmpeg - it's from the ReadableStream returned by createStream. Using pcm fires the finish event, but opus does not (the only difference when I tried it was the mode option).

As the stream never fires a finish event, using ffmpeg to parse it never finishes (unless I use the timeout option, which would be a bit clunky).

amishshah commented 3 years ago

Hi there,

We're working on a new implementation of Discord's Voice API that has better playback quality and is more reliable than what we currently support in Discord.js v12 - check it out at https://github.com/discordjs/voice!

The new library solves many of the issues that users are facing, and as part of this, we're dropping built-in support for voice in our next major release. We have a PR (https://github.com/discordjs/discord.js/pull/5402) that adds native support for our new voice library - once this PR is merged, this issue will be closed.

You can still use our new voice library before that PR lands - just take a look at our music bot example to see how to get started upgrading your voice code. By using the boilerplate music player in the example, you can make it even easier to upgrade your code.

Note that the PR above only reduces some of the boilerplate code you'd otherwise have to write - you do not have to wait for the PR to be merged to start using the new voice library.

If you have any questions about this, feel free to: