guildMemberSpeaking event no longer working?

[tacodan]

Please describe the problem you are having in as much detail as possible:

All of a sudden the "guildMemberSpeaking" stopped working. The bot was running did not crash, no errors, all of a sudden the event stop working. It will however emit once when a user starts talking but never will again for that user.

The following is an example of my entire test bot to show the issue:

const path = require('path');
SERVER_DIST = (path.basename(__filename) == "bot.js" ? "prod" : "beta");
CONFIG = require('./config.js').loadVar(SERVER_DIST);

const Discord = require('discord.js');
const client = new Discord.Client();

client.on('error', console.error);

client.on('ready', () => {
    console.log(`Logged in as ${client.user.tag}!`);
    let voiceChan = client.channels.get(CONFIG.ids.mainVoiceChanID);
    voiceChan.join().then(connection => {
        console.log('Connected to voice channel.');
    }).catch(err => console.log(err)); 
});

client.on('guildMemberSpeaking', (member, speaking) => {
    if(speaking) {
        console.log(member.displayName + ' started talking.');    
    }
    else {
        console.log(member.displayName + ' stopped talking');
    }
});

client.login(CONFIG.discordApi.token);

Further details:

• discord.js version: 11.5.1
• Node.js version: v10.15.3
• Operating system: CentOS release 6.10 (Final)
• Priority this issue should have – please be realistic and elaborate if possible: We use this event to track voice activity in a voice channel, this is very important to this bots usage.

• [ ] I have also tested the issue on latest master, commit hash:

[dragonbane0]

Can confirm this started happening for us yesterday as well and really killed us. In our case it happens with the connection.on('speaking') event in the exact same way as described. I assume a discord API change?

[TreZc0]

This is becoming a rather severe issue for us, is there any fix on the horizon?

[1-max-1]

The exact same issue happened for me as well. At first I thought it was a bug in my code, but now I am nearly certain it isn't. Unusual. A fix would be greatly appreciated. :-)

[njohns67]

Same issue here. The connection.on("speaking") event seems to fire twice immediately and then never again. I've been debugging for hours trying to figure out what I did wrong before finally finding this thread.

[monbrey]

Is everyone here encountering this issue only on stable? Has anyone had the same issue on the master branch of djs?

[njohns67]

@Monbrey I can't try it with the master branch atm but I can tomorrow. If you want to try it yourself I'm sure we'd all appreciate it. If not I'll let you know tomorrow

Edit: It doesn't seem that the master branch is even stable enough to test with which seems peculiar. I get a Syntax Error: Unexpected token * at src/structures/interfaces/Collector.js:203 Possibly could be some installation issue on my part.

[monbrey]

Syntax Error: Unexpected token * at src/structures/interfaces/Collector.js:203

This suggests your version of Node is outdated - I believe support for async iterators was added in Node 10 which the master branch requires.

[Moebits]

Same problem on master as well. Seems like there was an api change

[tacodan]

I know this feature isn't officially supported as it has been pointed out to me on discord but this feature is really important to my bot. I'd really appreciate it of someone could possibly look into it and determine if maybe there is a work around. I understand this might not be possible but either way I'd like to say in advance and that your efforts are greatly appreciated.

[DevYukine]

As you already pointed out this feature is not "official" supported by discord because recieving voice is completely undocumented and everything undocumented is subject to changes at any point without any notice.

Opinion Ahead!

This is my own opnion and does not reflect any position of the Library or maintainer/developer of it.

To me this is more like a legacy feature now, because it used to be (in-official) supported but discord decided against documenting it and i think with that d.js should not support it anymore. We should maybe deprecate this and remove it in the next major release because it might break at any point without a reliable way to fix that beside testing/reverse engineering against the api (which is forbidden btw :^) ).

@tacodan sorry that this might break the "really important" feature but i feel like this is just legacy & maintaining it is a pain at this point.

[dragonbane0]

That's unfortunate to hear. Our bot literally relies on being able to capture the isSpeaking event to properly show on our streaming layouts who is currently speaking or not. This is a major convenience feature for the viewers and currently is broken in the sense that everyone is showcased as speaking all the time. So this being supported was a major draw for discord.js.

However I understand if maintaining something that requires constant reverse engineering since discord wastes no effort to document and officially support it is too much to ask for.

If it could be looked into in case it is something simple it would be appreciated, else I try it with a fork I guess. If it gets removed entirely I hope at least getting the members who are inside a voice channel is still gonna work

[amishshah]

It seems that no speaking events are received after the first one from the voice gateway. I've asked for some more information on this :+1:

[tacodan]

It seems that no speaking events are received after the first one from the voice gateway. I've asked for some more information on this 👍

Thank you very much for your time looking into this amishshah. :)

[dragonbane0]

Co-signed. Thank you very much amishshah if you can figure something out. As simple as it is, it really allowed all sorts of cool stuff to know when someone speaks. Really strange discord's documentation includes some voice stuff, but not this particular field

EDIT: Looking into it a bit, it really does seem there is no isSpeaking event any longer after the first one. Yet the Discord client is still able to discern whether someone is speaking or not, so I guess maybe they determine the isSpeaking event solely based on whether new voice packages arrive for that user in a certain timespan or something?

[dragonbane0]

This is really weird. If you open discord in the browser where the voice connection seemingly uses the webrtc protocol instead of UDP, there are still isSpeaking events coming through for 0 and 1. Feels like a discord bug tbh, but the client can handle it, so it doesn't rely on it to determine whether someone is speaking I guess

[njohns67]

As a side issue, has anyone else had issues with connection.createPCMStream() after connecting to a voice channel? When on("speaking") broke I tried moving my voice receive code outside that block since it runs off the same connnection returned from joining the voice channel but it doesn't seem to be working.

[dragonbane0]

Capturing voice only works if you send some audio first since a while. The latest master fixed this by making the bot play some silence upon joining a voice channel. Also in the latest stable release, due the onSpeaking event never coming through with a false value, the streams you create with createPCMStream() might not get destroyed properly when the user stops speaking

[njohns67]

Thanks I forgot to include my code to play a sound. I'll keep trying to workaround for the on.("speaking") bug

Edit: It seems you were right; since the stream is supposed to open and close when the user is speaking it doesn't work properly, or I couldn't get it to.

[jhgg]

We (discord) now only send speaking event once per WS connection - as our clients dont need this event beyond doing initial ssrc => user_id association.

There is a work-around that involves synthesizing the speaking events from UDP packet flow (which is what the native client does.)

See this example psuedo-ish code for more details:

const speakingMembers = new Map();
const SPEAKING_DELAY = 250;

onRtpPacket(packet) {
    const userId = someSsrcToUserIdMapping.get(packet.ssrc);
    if (userId == null) return;
    const currentSpeakingTimeoutRef = speakingMembers.get(userId);
    if (currentSpeakingTimeoutRef == null) {
        this.emit('speaking', userId);
    } else {
        clearTimeout(currentSpeakingTimeoutRef);
    }
    const newSpeakingTimeoutRef = setTimeout(() => { this.emit('stopSpeaking', userId);  speakingMembers.delete(userId); }, SPEAKING_DELAY);
    speakingMembers.set(userId, newSpeakingTimeoutRef); 
}

[tacodan]

Randomly without restarting my bot, changing any code or updating to a different version of d.js it all of a sudden is now working again. Maybe discord reverted their breaking change?

1 of my 2 test servers it's working on..... so still odd.

Is this working for anyone else?

[bodenbao]

The VoiceConnection.on listen event is still broken. Just confirmed this.

[dragonbane0]

I couldn't wait for this to get fixed, since we actively use this feature in a relatively big production. So I investigated a bit and indeed inspecting the UDP packages seems to be the way to go. Here is my spin on a quick fix I made 2 days ago, which surprisingly resembles jhgg's version pretty much.

This is designed for the latest stable version of discord.js, but it can be easily translated I'm sure.

src/client/voice/receiver/VoiceReceiver.js after L30 add: this.speakingTimeouts = new Map();

Before L52 add:

if (this.speakingTimeouts.get(ssrc)) {
  clearTimeout(this.speakingTimeouts.get(ssrc));
  this.speakingTimeouts.delete(ssrc);
}
else {
  this.voiceConnection.onSpeaking({user_id: user.id, ssrc: ssrc, speaking: true});
}   
let speakingTimer = setTimeout(() => {
  try {
    this.voiceConnection.onSpeaking({ user_id: user.id, ssrc: ssrc, speaking: false });
    this.speakingTimeouts.delete(ssrc);
  }
  catch (ex) {
    console.log("Connection already closed");
  }
}, 50);

this.speakingTimeouts.set(ssrc, speakingTimer);

This gives you the connection.on('speaking') and client.on('guildMemberSpeaking') events back in a pretty much perfectly working state, as well as fix the streams not ending when someone stops speaking if you record them.

Remember you need to play some silence first in order for the bot to receive voice packages properly.

[Xilophinum]

The guildMemberSpeaking event is somehow working again, even though jhgg confirmed its only supposed to be firing only once. This makes it hard because it seems that I am making the event fire twice now if I implement any changes, or breaks it again all together, because I assume it is cancelling itself out. If this is only temporary, and is supposed to break again, what is the most ideal way to handle double events on master v12 until we find out? I am trying to handle the events inside the packetHandler push function, since it seems the most logical place to get the user and ssrc. Sorry to piggyback off the post, but this seems like the best place to secure an answer for myself and others trying to fix their bots.

[njohns67]

@dragonbane0 This does seem to properly emit the on("speaking") event but I'm running into some bugs that I didn't have previously. It seems the event fires twice in a row, causing a pcmStream error if you are trying to use the incoming voice data.

newMember.voiceChannel.join().catch(console.error).then(async function(connection) {
        connection.on("disconnect", () => console.log("Disconnecting dispatcher"))
        console.log("Connected")
        if(first){
           class Silence extends Readable {
               _read(){
                   this.push(Buffer.from([0xF8, 0xFF, 0xFE]))
               }
            }
           connection.playOpusStream(new Silence())
            first = false
        }
        const receiver = connection.createReceiver()
        connection.on('speaking', (user, speaking) => {
            console.log("Speaking")
            if (!speaking){
                return
            }
            const audioStream = receiver.createPCMStream(user)
           //More code to do stuff with the audioStream
        })
    })
})

This is my code minus any extraneous bits for readability's sake. Currently my console log looks like:

$ node bot.js
$ speaking
$ speaking
$  if (this.pcmStreams.get(user.id)) throw new Error('There is already an existing stream for that user.');
                                      ^

Error: There is already an existing stream for that user.

As you can see, it emits the event twice before crashing. Through testing I've found that it only emits twice the first time you speak. I'm guessing the first time is the event properly being emitted (as it was before everything broke) and the second time is your code emitting the event with subsequent emits being your code as well.

I was able to somewhat workaround this by modifing VoiceReceiver.js L161 to return the existing pcmStream instead of throwing an error, but this seems like a bandaid. It also cause the voice data to be corrupted.

[dragonbane0]

@njohns67 Yeah your assumption is correct. The first time would trigger twice. I guess the code can be altered to suppress the actual speaking event entirely now besides doing the user--->ssrc mapping

[njohns67]

Thank you for all your help. I ended up sticking with the modified VoiceReceiver.js code and simply returning the stream if there's an error. This may causes issues I'm unaware of but none have come up and until this is officially fixed I'm sticking with my bandaid for time's sake.

[tacodan]

I still haven't been able to get this working, I tried to apply the changes that dragonbane0 provided but still nothing :(

Can anyone help me out?

[bodenbao]

@tacodan I got it to work. Here's what I did. Hope it helps! This also keeps 'listen' from firing twice in the beginning...

VoiceReceiver.js - Add dragonbane0's code...

VoiceWebSocket.js - change the emit event from 'speaking' to 'startSpeaking'...

...
case Constants.VoiceOPCodes.SPEAKING:
        ...
        this.emit('startSpeaking', packet.d);
        break;
...

VoiceConnection.js - Add the 'startSpeaking' event and the ws.on and ws.remove listeners...

...
onStartSpeaking({ user_id, ssrc, speaking }) {
    const user = this.client.users.get(user_id);
    this.ssrcMap.set(+ssrc, user);
}
...
      ws.removeAllListeners('speaking');
      ws.removeAllListeners('startSpeaking');
    }
...
    ws.on('speaking', this.onSpeaking.bind(this));
    ws.on('startSpeaking', this.onStartSpeaking.bind(this));
  }
...

[tacodan]

@tacodan I got it to work. Here's what I did. Hope it helps! This also keeps 'listen' from firing twice in the beginning...

VoiceReceiver.js - Add nJohn67's code...

VoiceWebSocket.js - change the emit event from 'speaking' to 'startSpeaking'...

...
case Constants.VoiceOPCodes.SPEAKING:
        ...
        this.emit('startSpeaking', packet.d);
        break;
...

VoiceConnection.js - Add the 'startSpeaking' event and the ws.on and ws.remove listeners...

...
onStartSpeaking({ user_id, ssrc, speaking }) {
    const user = this.client.users.get(user_id);
    this.ssrcMap.set(+ssrc, user);
}
...
      ws.removeAllListeners('speaking');
      ws.removeAllListeners('startSpeaking');
    }
...
    ws.on('speaking', this.onSpeaking.bind(this));
    ws.on('startSpeaking', this.onStartSpeaking.bind(this));
  }
...

do you mean dragonbane0's code for VoiceReceiver.js or? Also would you be able to post your discord name so maybe we could chat via dm?

[bodenbao]

@tacodan yes, I mean dragonbane0's code... sorry... my account is bodenbao#0854.

[bsguedes]

Are these changes going to be integrated in an official release in discord.js, so the guildMemberSpeaking event returns to its normal behavior? Thanks!

[sillyfrog]

@Kunikita Do you have a fork with theses changes applied? Sounds like it maybe worth making a PR to get this into master as it would be great to have. I can also make a PR, but sounds like you have already done the work. Cheers.

[christopherfowers]

Hate to be "that guy" but has there been any progress on this? Or is this even going to be addressed since it's not technically supported by discord?

[Nameless9]

@sillyfrog hello, I was trying to use your fork version. I have some mixed result. Sometimes, the speaking event works, which is great. Other times, none of the events work, let alone, speaking event. Even disconnect event doesn't get triggered when I disconnect the voice connection.

These results occur randomly without any sign or warning. So surely something must be wrong in the backend?

[christopherfowers]

Yeah. Haven't had a chance to debug yet but noticed no events working. Was hoping to take a look some time today. I'll try tomorrow.

[Nameless9]

can I know if anyone else is experiencing none of the event-handlers on connection working during sometimes and then randomly everything working?

@christopherfowers: thanks

[christopherfowers]

@Nameless9 it looks like the bot may have something else going on. Noted that when the bot joins my voice channel, it is broadcasting (note the green circle around the avatar: . Yet there is no point that I have provided any statements for this action in code.

I assume you may find that you're experiencing something similar.

// jshint esversion: 8
const Discord = require("discord.js");
const config = require("./config.json");
const discord = new Discord.Client();

discord.login(config.discordApiToken);

discord.on("ready", () => {
  console.log(`Logged in as ${discord.user.tag}!`);
});

discord.on("message", async msg => {
  if (msg.content.startsWith("#join")) {
    let [command, ...channelName] = msg.content.split(" ");

    if (!msg.guild) {
      return msg.reply(
        "no private service is available in your area at the moment. Please contact a service representative for more details."
      );
    }

    const voiceChannel = msg.guild.channels.find(
      channel => channel.name === channelName.join(" ")
    );

    if (!voiceChannel || voiceChannel.type !== "voice") {
      return msg.reply(
        `I couldn't find the channel ${channelName}. Can you spell?`
      );
    }

    const connection = await voiceChannel.join();
    console.log("Ready!");

    connection.on("speaking", (user, speaking) => {
      console.log(speaking);
      if (speaking) {
        console.log(`${user.username} is speaking`);
      } else {
        console.log(`${user.username} stopped speaking`);
      }
    });
  }
});

[sillyfrog]

What exact version of node are you using? I have hacked something together so you can see it in action in Docker. This is based on what I used for all my dev and testing: https://github.com/sillyfrog/discord.js-listenexample

That should be a basis for you to get started, you can then work from there to hopefully debug it. This also shows all of the packages I'm using in both Linux and node (which if missing could be causing issues).

[ptree44]

@christopherfowers I'm working with @Nameless9 .. we have that very exact issue. We have tried node-opus and opusscript. We have tried joining channels, leaving, rejoining, creating new broadcasts, etc to no avail. We even did .disconnect() rigt after joining a channel, the disconnect event handler isn't logging that

We tried it on node 10.2 and node 10.16

Edit: And now 11.14

More edit: @sillyfrog we tried using your listenexample, it did not work );

[christopherfowers]

What exact version of node are you using? I have hacked something together so you can see it in action in Docker. This is based on what I used for all my dev and testing: https://github.com/sillyfrog/discord.js-listenexample

That should be a basis for you to get started, you can then work from there to hopefully debug it. This also shows all of the packages I'm using in both Linux and node (which if missing could be causing issues).

I am on windows 10 with node 12.13.0

I do tend to run linux docker containers and I suppose I could attempt this from my mac even.

I can try to rule out node versions first. Thankfully we have NVM to the rescue. I'll let you know what I find out.

[christopherfowers]

Can confirm the Demo you made works @sillyfrog. @ptree44 if you would like some help getting that working let me know. We can sidebar this conversation. Meanwhile going to press on with seeing if node versioning is my issue.

[christopherfowers]

@ptree44 can you tell me more about your environment? What OS are you on running on? What packages do you have installed (specifically node packages)?

[ptree44]

@ptree44 can you tell me more about your environment? What OS are you on running on? What packages do you have installed (specifically node packages)?

"dependencies": { "@discordjs/uws": "^11.149.1", "big-integer": "^1.6.48", "body-parser": "1.19.0", "brain.js": "^1.2.2", "bufferutil": "^4.0.1", "cloudscraper": "^4.1.2", "discord.js.master": "github:discordjs/discord.js#master", "discord.js.stable": "github:discordjs/discord.js#stable", "eris": "0.10.0", "erlpack": "github:discordapp/erlpack", "express": "^4.16.4", "firebase": "6.3.4", "get-pixels": "^3.3.2", "googleapis": "^39.2.0", "is-json": "^2.0.1", "libsodium-wrappers": "^0.7.6", "obs-websocket-js": "^3.1.0", "opusscript": "0.0.7", "pnpm": "2.25.5", "promise": "^8.0.3", "request": "2.88.0", "timer-node": "1.0.4", "tmi.js": "^1.2.1", "when": "3.7.8", "ytdl-core": "^1.0.3", "zlib-sync": "^0.1.6" }, "engines": { "node": "10.16.x" },

from our package.json

[Lovinity]

Quick question relating to this. In the distant past, I recall the speaking event could fire for changes in speaking state even if the bot was not in the voice channel. Analyzing the suggested code changes above, it appears that is no longer the case and the bot absolutely has to be in a voice channel in order to receive speaking states. Is this true? And if so, is there a possible workaround to this, or is it bottlenecked by Discord's API?

[ptree44]

@Lovinity The guildMemberSpeaking event supposedly should work for bots outside of the voice channel. I can't get the guildMemberSpeaking event nor speaking event for VoiceConnection to work though, but I do seem to be in the minority.

[Lovinity]

@ptree44 They do not work for me either. Earlier in the thread, people mentioned it firing once or twice and that's it. For me (using master branch), it doesn't ever fire. I debugged by checking the raw event, and raw is not coming up with any speaking nor guildMemberSpeaking events either.

This applies both when the bot is not in a voice channel, and when it is in a voice channel.

[Fyko]

This has since been fixed with #3578.

Update discord.js with

yarn upgrade discordjs/discord.js
or
npm update discordjs/dsicord.js

[ptree44]

@Lovinity I tried on multiple node versions and it didn't work for me. I've also tried fresh projects , still didn't work. It's a needed feature in my bot , very disappointed.

[ptree44]

@Fyko that did not fix it in my circumstance.

[Lovinity]

I too have the latest master commit and it does not work.