search

Olical / snowball

Voice activated Discord bot running on GCP
https://hub.docker.com/r/olical/snowball/
The Unlicense
41 stars 22 forks source link

Bot not responding to wake up command #1

Closed ampc closed 5 years ago

ampc commented 6 years ago

Hello again, I've already created a service account credential with cloud speech and text-to-speech api access and saved it in config folder as google.json. I get the bot to join my server and channels but it's not responding to voice commands. I don't see any activity in google dashboards either. 🤔 

PS D:\repos\bot> docker run -ti --rm -v ${PWD}/config:/usr/snowball/config olical/snowball:latest
18-10-03 19:29:52 5e280cd55d2f INFO [snowball.main:10] - Starting nREPL server on port 9045
18-10-03 19:29:52 5e280cd55d2f INFO [snowball.main:15] - Starting components...
18-10-03 19:29:53 5e280cd55d2f INFO [snowball.config:16] - Loading base config from config.base.edn and user config from config/config.edn
18-10-03 19:29:53 5e280cd55d2f INFO [snowball.discord:150] - Connecting to Discord
Discord4J: ERROR INITIALIZING LOGGER!
Discord4J: No SLF4J implementation found, reverting to the internal implementation (sx.blah.discord.Discord4J$Discord4JLogger)
Discord4J: It is *highly* recommended to use a fully featured implementation like logback!
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
19:29:53.924: [INFO][main][sx.blah.discord.Discord4J] - Discord4J v2.10.1 9b1c81a (https://github.com/austinv11/Discord4J)
19:29:53.926: [INFO][main][sx.blah.discord.Discord4J] - A Java binding for the official Discord API, forked from the inactive https://github.com/nerd/Discord4J. Copyright (c) 2017, Licensed under GNU LGPLv3
18-10-03 19:29:54 5e280cd55d2f INFO [snowball.discord:31] - Connected, waiting until ready
18-10-03 19:29:54 5e280cd55d2f INFO [snowball.discord:32] - Not ready, sleeping for 1000ms
19:29:55.295: [INFO][HttpClient@1894307438-19][sx.blah.discord.Discord4J] - Websocket Connected.
19:29:55.570: [INFO][Dispatch Handler][sx.blah.discord.Discord4J] - Connected to Discord Gateway v6. Receiving 1 guilds.
18-10-03 19:29:55 5e280cd55d2f INFO [snowball.discord:33] - Ready
18-10-03 19:29:55 5e280cd55d2f INFO [snowball.discord:172] - Starting audio channel from subscription
18-10-03 19:29:55 5e280cd55d2f INFO [snowball.comprehension:20] - Starting phrase channel
18-10-03 19:29:55 5e280cd55d2f INFO [snowball.speech:56] - Starting up speech client
18-10-03 19:29:56 5e280cd55d2f INFO [snowball.comprehension:108] - Starting speech to text systems
18-10-03 19:29:56 5e280cd55d2f INFO [snowball.comprehension:181] - Porcupine frame length is 512 samples and the sample rate is 16KHz, as expected
18-10-03 19:29:56 5e280cd55d2f INFO [snowball.command:139] - Starting command dispatcher loop
18-10-03 19:29:56 5e280cd55d2f INFO [snowball.presence:17] - Starting presence poller
18-10-03 19:29:56 5e280cd55d2f INFO [snowball.main:23] - Everything's up and running!
18-10-03 19:29:56 5e280cd55d2f INFO [snowball.discord:137] - Audio manager exists, subscribing to audio
Olical commented 6 years ago

Did you ever get it to pick you up? As I mentioned over twitter (for future people):

To be clear, this part doesn't require Google, the part that isn't being detected is the local detection with porcupine. Since it's not as powerful and generic as Google speech detection (but MUCH cheaper) it's pretty finicky with picking up the wake word. You have to learn the best way to pronounce the wake words I find.

ampc commented 6 years ago

I'll try to modify porupine call 👍

ampc commented 6 years ago

Is it possible to see anything in the logs related with sound capture (before sending to google) ?

ampc commented 6 years ago

I tried to change the wake up call to something like "hey google". I only modified the Makefile line

cd wake-word-engine/Porcupine && tools/optimizer/linux/x86_64/pv_porcupine_optimizer -r resources/ -w "hey snowball" -p linux -o ../ to cd wake-word-engine/Porcupine && tools/optimizer/linux/x86_64/pv_porcupine_optimizer -r resources/ -w "hey google" -p linux -o ../

I built the image and get the following error when running the container:

18-10-06 22:17:46 741ea1b327e2 INFO [snowball.main:15] - Starting components... Exception in thread "main" java.lang.ClassNotFoundException: snowball.porcupine.Porcupine, compiling:(snowball/comprehension.clj:1:1) at clojure.lang.Compiler.load(Compiler.java:7526)

Edit: Also needed to run make build

Olical commented 6 years ago

Ah, sorry, there's more things that needed to be built in the container that I had built locally, such as that class. snowball.porcupine.Porcupine is a JNI binding that lets the Clojure talk to the native library. I'll take a look.

Olical commented 6 years ago

So yeah, if your make build runs okay it should be fine. That builds everything in the wake-word-engine directory. I didn't want to do that as part of the Dockerfile because it might involve adding too many files that aren't required at runtime :thinking: maybe I can find a way.

Olical commented 6 years ago

Pushed a few changes. You can now change the path for your custom ppn file in the config under :comprehension :wake-phrase. For instance, the default hey snowball translates to the path wake-word-engine/hey_snowball_linux.ppn.

I've also added a way to tweak the sensitivity, you may want to make it a little higher, like 0.75? It might make a difference.

Added a fix for an issue where two speakers can trigger the wake word too, like if I said "hey" then my friend said "snowball" it could wake up. Just an issue around the porcupine usage, might require something more elaborate to fix that.

Last thing: Fixed the makefile so the Porcupine.class file is compiled if it's missing. So your make build will actually build all required files now. I hope this helps!

(no hard feelings if you've given up on this, it's awkward to get working and you're the first person to try it other than me :laughing:)

Olical commented 6 years ago

Mine isn't working now either by the way, so maybe I broke something but I don't remember changing anything 🤔

ampc commented 6 years ago

I'll try again this weekend :)

Olical commented 6 years ago

Okay well this is really weird. If I set my Discord server to Western Europe, Snowball doesn't receive any audio from Discord, at all. If I set it to Central Europe, it works fine...

This is so weird.

ampc commented 6 years ago

Wow :thinking: Mine is always in Western Europe

Olical commented 6 years ago

Up to you, but if you try moving it to Central Europe it may well work. You don't have to keep it there, but it's a good experiment. I would put money on it suddenly working... the question is why? I tweeted at Discord, maybe it's a bug on their side? That one server doesn't send audio to bots?

Either that or it's a bug in Discord4J.

ghost commented 6 years ago

I can't get the wake word working either. My Discord server is US East. Here is the docker container log: 18-10-13 18:14:07 6e2f0a580e43 INFO [snowball.main:10] - Starting nREPL server on port 9045 18-10-13 18:14:07 6e2f0a580e43 INFO [snowball.main:15] - Starting components... 18-10-13 18:14:09 6e2f0a580e43 INFO [snowball.config:16] - Loading base config from config.base.edn and user config from config/config.edn 18-10-13 18:14:09 6e2f0a580e43 INFO [snowball.discord:150] - Connecting to Discord Discord4J: ERROR INITIALIZING LOGGER! Discord4J: No SLF4J implementation found, reverting to the internal implementation (sx.blah.discord.Discord4J$Discord4JLogger) Discord4J: It is *highly* recommended to use a fully featured implementation like logback! SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. 18:14:09.226: [INFO][main][sx.blah.discord.Discord4J] - Discord4J v2.10.1 9b1c81a (https://github.com/austinv11/Discord4J) 18:14:09.228: [INFO][main][sx.blah.discord.Discord4J] - A Java binding for the official Discord API, forked from the inactive https://github.com/nerd/Discord4J. Copyright (c) 2017, Licensed under GNU LGPLv3 18-10-13 18:14:09 6e2f0a580e43 INFO [snowball.discord:31] - Connected, waiting until ready 18-10-13 18:14:09 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18:14:10.346: [INFO][HttpClient@1368679878-19][sx.blah.discord.Discord4J] - Websocket Connected. 18-10-13 18:14:10 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:11 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:12 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:13 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:14 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:15 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:16 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:17 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:18 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:19 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:20 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:21 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:22 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:23 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:24 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:25 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:26 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:27 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:28 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18:14:29.663: [INFO][Dispatch Handler][sx.blah.discord.Discord4J] - Connected to Discord Gateway v6. Receiving 1 guilds. 18-10-13 18:14:29 6e2f0a580e43 INFO [snowball.discord:33] - Ready 18-10-13 18:14:29 6e2f0a580e43 INFO [snowball.presence:17] - Starting presence poller 18-10-13 18:14:29 6e2f0a580e43 INFO [snowball.speech:56] - Starting up speech client 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.discord:172] - Starting audio channel from subscription 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.comprehension:20] - Starting phrase channel 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.comprehension:108] - Starting speech to text systems 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.comprehension:181] - Porcupine frame length is 512 samples and the sample rate is 16KHz, as expected 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.command:139] - Starting command dispatcher loop 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.main:23] - Everything's up and running! 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.discord:57] - Joining General 18:14:31.293: [INFO][HttpClient@513159551-46][sx.blah.discord.Discord4J] - Voice Websocket Connected. 18-10-13 18:14:31 6e2f0a580e43 INFO [snowball.discord:137] - Audio manager exists, subscribing to audio

Olical commented 6 years ago

Yep it seems to only work in central Europe as far as I can tell, I've opened an issue in the Discord4J library just in case it's in there. Not sure how it could have anything to do with my code.

On Sat, 13 Oct 2018, 19:25 krcm0209, notifications@github.com wrote:

I can't get the wake word working either. My Discord server is US East. Here is the docker container log: 18-10-13 18:14:07 6e2f0a580e43 INFO [snowball.main:10] - Starting nREPL server on port 9045 18-10-13 18:14:07 6e2f0a580e43 INFO [snowball.main:15]

  • Starting components... 18-10-13 18:14:09 6e2f0a580e43 INFO [snowball.config:16] - Loading base config from config.base.edn and user config from config/config.edn 18-10-13 18:14:09 6e2f0a580e43 INFO [snowball.discord:150] - Connecting to Discord Discord4J: ERROR INITIALIZING LOGGER! Discord4J: No SLF4J implementation found, reverting to the internal implementation (sx.blah.discord.Discord4J$Discord4JLogger) Discord4J: It is highly recommended to use a fully featured implementation like logback! SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. 18:14:09.226: [INFO][main][sx.blah.discord.Discord4J] - Discord4J v2.10.1 9b1c81a (https://github.com/austinv11/Discord4J) 18:14:09.228: [INFO][main][sx.blah.discord.Discord4J] - A Java binding for the official Discord API, forked from the inactive https://github.com/nerd/Discord4J. Copyright (c) 2017, Licensed under GNU LGPLv3 18-10-13 18:14:09 6e2f0a580e43 INFO [snowball.discord:31] - Connected, waiting until ready 18-10-13 18:14:09 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18:14:10.346: [INFO][HttpClient@1368679878-19][sx.blah.discord.Discord4J]
  • Websocket Connected. 18-10-13 18:14:10 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:11 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:12 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:13 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:14 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:15 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:16 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:17 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:18 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:19 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:20 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:21 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:22 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:23 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:24 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:25 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:26 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:27 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18-10-13 18:14:28 6e2f0a580e43 INFO [snowball.discord:32] - Not ready, sleeping for 1000ms 18:14:29.663: [INFO][Dispatch Handler][sx.blah.discord.Discord4J] - Connected to Discord Gateway v6. Receiving 1 guilds. 18-10-13 18:14:29 6e2f0a580e43 INFO [snowball.discord:33] - Ready 18-10-13 18:14:29 6e2f0a580e43 INFO [snowball.presence:17] - Starting presence poller 18-10-13 18:14:29 6e2f0a580e43 INFO [snowball.speech:56] - Starting up speech client 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.discord:172] - Starting audio channel from subscription 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.comprehension:20] - Starting phrase channel 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.comprehension:108] - Starting speech to text systems 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.comprehension:181] - Porcupine frame length is 512 samples and the sample rate is 16KHz, as expected 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.command:139] - Starting command dispatcher loop 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.main:23] - Everything's up and running! 18-10-13 18:14:30 6e2f0a580e43 INFO [snowball.discord:57] - Joining General 18:14:31.293: [INFO][HttpClient@513159551-46][sx.blah.discord.Discord4J] - Voice Websocket Connected. 18-10-13 18:14:31 6e2f0a580e43 INFO [snowball.discord:137] - Audio manager exists, subscribing to audio

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Olical/snowball/issues/1#issuecomment-429564360, or mute the thread https://github.com/notifications/unsubscribe-auth/AATPXY1Bdr_GR6PTxMbaiidJW0uwbG0rks5ukjAcgaJpZM4XGuMb .

Olical commented 6 years ago

For people of the future, here's the issue I raised with Discord4J https://github.com/Discord4J/Discord4J/issues/452 - it may end up not being anything to do with the library, but I thought I'd link that issue with this one for future reference.

Olical commented 5 years ago

It looks like if you send some audio first then it works fine. I'll make snowball say hello when it joins for now.