search

discordjs / discord.js

A powerful JavaScript library for interacting with the Discord API
https://discord.js.org
Apache License 2.0
25.46k stars 3.97k forks source link

.opus files choke system as does { type: 'opus'} #3979

Closed KM0KA closed 3 years ago

KM0KA commented 4 years ago

Please describe the problem you are having in as much detail as possible:

I have my environment playing audio when I submit .ogg files and .mp3 files. It is my understanding that .ogg files will not use FFMEG, but the .mp3 files will to convert the stream to opus format.

I am having issues getting the audio to not lag. I am also having fidelity issues. The quality is lacking and is on the tinny side. I have used { highWaterMark: ?} at various settings from 12-100. Some marginal quality improvements. I have the DISCORD bitrate at the highest available (96 kbps).

I am using .ogg so as to not use FFMEG and get any performance improvements that course provides. How can I tell if I am bypassing FFMEG and the system is working as intended?

I had been pulling from AWS CloudFront which had horrible results, so I pull the file local before I initiate:

connection.play(file).

This resulted in dramatic improvements.

But I am still suffering from fidelity issues and dropping on occasion. So I have experimented.

I have converted mp3 to the highest quality of both .ogg and opus files. The end result is a file size larger than the original mp3 file.

When I attempt to play any .opus file, the system hangs. This is with and without setting the:

{ type: 'opus'} parameters.

If I play a .ogg file without setting this, it will play. If I attempt to set this flag { type: 'opus'} or { type: 'ogg/opus'} in any situation, the system hangs. So I think there is some bug that needs to be crushed.

How can I increase fidelity?

Is the bitrate setting supposed to be spelled out like in the following:

{bitrate :96000} instead of {bitrate :96} ?

I have tried {bitrate : 128000} and {bitrate :96000} with no discernible performance improvements.

Should I pull from a fs stream and pipe it in? Is there some tweak I need to do to get the fidelity to be acceptable?

Thanks in advance.

Include a reproducible code sample here, if possible:

// Place your code here

Further details:

KM0KA commented 4 years ago

As far as bypassing FFMPEG . . . even when using .ogg files, FFMPEG is being fired off and running a process, taking up .3 % of cpu cycles . . . so I have no idea if FFMPEG is being used with .ogg files or not . . . it is my understanding that .ogg feels are opus wrappers and FFMPEG is not used and the files are not converted . . .

KM0KA commented 4 years ago

I continue to explore settings to maximize performance . . .

I moved away from the Broadcast paradigm as I was not able to do any recursion to maintain a continuous looping . . . during any recursion operations (called on the "end event" . . . the dispatcher. . . even after destroying it. . . setting it to null...or other techniques to make sure it was cleared out. . . would not respond after the first loop . . . anytime I used the type: 'ogg/opus' option it wouldn't even play . . .

So I moved to a straight voice paradigm, where I kept fidgeting with the settings . . .

These are the configurations I have or am testing now . . . localFilePath is always a ".ogg" file containing high quality .mp3 to .opus converted audio . . . making the file size much bigger than the original mp3:

this.dispatcher = connection.play( localFilePath, { highWaterMark: 36, bitrate: 128000, volume: false, plp: 0, fec: true} );

I Boosted my Server to get the 128000 kbps . . . it does sound better . . . this configuration without using fs.createReadStream(localFilePath) still lags and drops a bit . . . running a 'top command' shows FFMPEG running . . .

Using fs.createReadStream:

this.dispatcher = connection.play(fs.createReadStream(localFilePath), { highWaterMark: 36, bitrate: 128000, volume: false, plp: 0, fec: true} );

I am still testing this to see if it lasts and drops . . . running a 'top command' shows FFMPEG running . . .

this.dispatcher = connection.play(fs.createReadStream(localFilePath), { type: 'ogg/opus', highWaterMark: 36, bitrate: 128000, volume: false, plp: 0, fec: true} );

The above configuration...using the type: 'ogg/opus' option . . . never plays . . . it always immediately fires off the 'on end' event and immediately loops to the start (the recursion talked about above)... running a 'top command' shows FFMPEG running . . .

this.dispatcher = connection.play(localFilePath, { type: 'ogg/opus', highWaterMark: 36, bitrate: 128000, volume: false, plp: 0, fec: true} );

The above locks up the system completely . . . and requires the process to be killed and restarted . . .

stripping every thing out:

this.dispatcher = connection.play(localFilePath );

it plays . . . it loops . . . but it lags and drops . . .

I opened up my UDP ports on the firewall . . . I am using Amazon AWS / Lightsail . . . I am running on a 2 CPU 4 gig box . . . it was/is playing before i opened up these ports . . . I don't know enough about what is under the hood in making this work . . . with DISCORD . . . I under stand that a Websocket is established for authentication and control, but that the audio is transmitted over UDP . . . how it was/is able to transmit prior to not having these ports open I do not know . . . but in any case . . . I don't see any performance improvement with the ports enabled or not . . .

In summary . . . using the " type: 'ogg/opus' " option in any configuration causes issues no matter what . . . and the FFMPEG is being fired off in every situation . . . using .3% of CPU . . . even when using opus encoded .ogg wrapped audio . . .

Hope these details help. Attempting to provide as much detail as possible . . . to make things better and better . . .

amishshah commented 3 years ago

Hi there,

We're working on a new implementation of Discord's Voice API that has better playback quality and is more reliable than what we currently support in Discord.js v12 - check it out at https://github.com/discordjs/voice!

The new library solves many of the issues that users are facing, and as part of this, we're dropping built-in support for voice in our next major release. We have a PR (https://github.com/discordjs/discord.js/pull/5402) that adds native support for our new voice library - once this PR is merged, this issue will be closed.

You can still use our new voice library before that PR lands - just take a look at our music bot example to see how to get started upgrading your voice code. By using the boilerplate music player in the example, you can make it even easier to upgrade your code.

Note that the PR above only reduces some of the boilerplate code you'd otherwise have to write - you do not have to wait for the PR to be merged to start using the new voice library.

If you have any questions about this, feel free to: