search

lavalink-devs / Lavalink

Standalone audio sending node based on Lavaplayer.
https://lavalink.dev/
MIT License
1.56k stars 674 forks source link

Failed to connect [Error] #229

Closed Autrix closed 4 years ago

Autrix commented 4 years ago

Description

it don't allow me to play music, it shows error: Failed to connect to [server.region] ( log included down in the post )

Version info

Client used:

Output of java -jar Lavalink.jar --version:


       .   _                  _ _       _    __ _ _
      /\\ | | __ ___   ____ _| (_)_ __ | | __\ \ \ \
     ( ( )| |/ _` \ \ / / _` | | | '_ \| |/ / \ \ \ \
      \\/ | | (_| |\ V / (_| | | | | | |   <   ) ) ) )
       '  |_|\__,_| \_/ \__,_|_|_|_| |_|_|\_\ / / / /
    =========================================/_/_/_/

        Version:        3.2.1.1
        Build:          828
        Build time:     31.07.2019 11:02:24 CEST
        Branch          master
        Commit:         d9c9aa0
        Commit time:    31.07.2019 10:59:59 CEST
        JVM:            11.0.4
        Lavaplayer      1.3.19

spring.log

owenselles commented 4 years ago

Hey we have this issue too. We fixed it by changing our voice region to Europe. A lot of servers our bot is in are affected as well.

Autrix commented 4 years ago

I did and still same issue ( i changed every region that on the serverlist ) still nothing works for me

aschenkuttel commented 4 years ago

same problem here (eu-central)

freyacodes commented 4 years ago

We believe this is a Cloudflare problem

ghost commented 4 years ago

Starting to think a bunch of companies held a secret conference and all plotted to ruin the week of bot developers.

owenselles commented 4 years ago

Is there something we can do to fix this atm?

freyacodes commented 4 years ago

Is there something we can do to fix this atm?

The only workaround seems to migrate to a server that isn't affected.

WaffleThief123 commented 4 years ago

Same thing here, it's mainly cloudflare. I'm reading into discord API changes, and it seems that they made some major changes. I'll link them here below as i find the related ones.

below is the exception from my spring.log, repeated across every public discord websocket

java.lang.IllegalStateException: Failed to connect to wss://sydney49.discord.media/?v=4
        at space.npstr.magma.impl.connections.hax.ClosingUndertowWebSocketClient$1.handleFailed(ClosingUndertowWebSocketClient.java:73) ~[impl-0.12.3.jar!/:na]
        at org.xnio.IoFuture$HandlingNotifier.notify(IoFuture.java:215) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.AbstractIoFuture$1.run(AbstractIoFuture.java:211) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.IoUtils$2.execute(IoUtils.java:70) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.AbstractIoFuture.runNotifier(AbstractIoFuture.java:354) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.AbstractIoFuture.runAllNotifiers(AbstractIoFuture.java:233) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.AbstractIoFuture.setException(AbstractIoFuture.java:251) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.FutureResult.setException(FutureResult.java:89) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at io.undertow.websockets.client.WebSocketClient$ConnectionBuilder$2.notify(WebSocketClient.java:342) ~[undertow-core-2.0.26.Final.jar!/:2.0.26.Final]
        at org.xnio.AbstractIoFuture$1.run(AbstractIoFuture.java:211) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.IoUtils$2.execute(IoUtils.java:70) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.AbstractIoFuture.runNotifier(AbstractIoFuture.java:354) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.AbstractIoFuture.runAllNotifiers(AbstractIoFuture.java:233) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.AbstractIoFuture.setException(AbstractIoFuture.java:251) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.FutureResult.setException(FutureResult.java:89) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:391) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:372) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.conduits.WriteReadyHandler$ChannelListenerHandler.writeReady(WriteReadyHandler.java:65) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at io.undertow.protocols.ssl.SslConduit$SslWriteReadyHandler.writeReady(SslConduit.java:1273) ~[undertow-core-2.0.26.Final.jar!/:2.0.26.Final]
        at io.undertow.protocols.ssl.SslConduit$3.run(SslConduit.java:275) ~[undertow-core-2.0.26.Final.jar!/:2.0.26.Final]
        at org.xnio.nio.WorkerThread.safeRun(WorkerThread.java:582) ~[xnio-nio-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.nio.WorkerThread.run(WorkerThread.java:466) ~[xnio-nio-3.3.8.Final.jar!/:3.3.8.Final]
Caused by: java.nio.channels.ClosedChannelException: null
        at io.undertow.protocols.ssl.SslConduit.write(SslConduit.java:369) ~[undertow-core-2.0.26.Final.jar!/:2.0.26.Final]
        at org.xnio.conduits.ConduitStreamSinkChannel.write(ConduitStreamSinkChannel.java:150) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:385) ~[xnio-api-3.3.8.Final.jar!/:3.3.8.Final]
        ... 7 common frames omitted
WaffleThief123 commented 4 years ago

It seems to be an issue with cloudflare directly, because when i do a simple curl against the v4 api base, I get a 404.

image

WaffleThief123 commented 4 years ago

Although according to discord API docs, we should be using v3 for transmission/receive of RTP Streams, but it seems that Lavalilnk is set up for v4 entirely based on the logging i'm getting.

image

WaffleThief123 commented 4 years ago

Here's a v4 curl from a proper linux box...

root@vcli-ytdl-test:~# curl sydney49.discord.media/?v=4/hello -v
* Expire in 0 ms for 6 (transfer 0x5623a93f2f50)
* Expire in 1 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 0 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 1 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 0 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 0 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 2 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 0 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 0 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 2 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 0 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 0 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 2 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 0 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 0 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 2 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 1 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 1 ms for 1 (transfer 0x5623a93f2f50)
* Expire in 1 ms for 1 (transfer 0x5623a93f2f50)
*   Trying 162.159.130.235...
* TCP_NODELAY set
* Expire in 149998 ms for 3 (transfer 0x5623a93f2f50)
* Expire in 200 ms for 4 (transfer 0x5623a93f2f50)
* Connected to sydney49.discord.media (162.159.130.235) port 80 (#0)
> GET /?v=4/hello HTTP/1.1
> Host: sydney49.discord.media
> User-Agent: curl/7.64.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Date: Thu, 17 Oct 2019 11:22:28 GMT
< Content-Length: 0
< Connection: keep-alive
< Set-Cookie: __cfduid=d53f2f49139c720dca5309785fcf37c3e1571311348; expires=Fri, 16-Oct-20 11:22:28 GMT; path=/; domain=.discord.media; HttpOnly
< CF-Cache-Status: DYNAMIC
< Server: cloudflare
< CF-RAY: 5271e5192fd546ea-EWR
<
* Connection #0 to host sydney49.discord.media left intact

and respectively a v3

root@vcli-ytdl-test:~# curl sydney49.discord.media/?v=3/hello -v
* Expire in 0 ms for 6 (transfer 0x564cd8d57f50)
* Expire in 1 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 0 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 1 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 0 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 0 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 2 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 0 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 0 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 2 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 0 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 0 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 2 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 0 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 0 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 2 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 1 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 1 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 4 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 1 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 1 ms for 1 (transfer 0x564cd8d57f50)
* Expire in 1 ms for 1 (transfer 0x564cd8d57f50)
*   Trying 162.159.137.234...
* TCP_NODELAY set
* Expire in 149997 ms for 3 (transfer 0x564cd8d57f50)
* Expire in 200 ms for 4 (transfer 0x564cd8d57f50)
* Connected to sydney49.discord.media (162.159.137.234) port 80 (#0)
> GET /?v=3/hello HTTP/1.1
> Host: sydney49.discord.media
> User-Agent: curl/7.64.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Date: Thu, 17 Oct 2019 11:23:41 GMT
< Content-Length: 0
< Connection: keep-alive
< Set-Cookie: __cfduid=d88b1def1545b11853fd6d9c2aec460a71571311420; expires=Fri, 16-Oct-20 11:23:40 GMT; path=/; domain=.discord.media; HttpOnly
< CF-Cache-Status: DYNAMIC
< Server: cloudflare
< CF-RAY: 5271e6dd2afce710-EWR
<
* Connection #0 to host sydney49.discord.media left intact

Judging by this, it seems their API is all kinds of screwy and nobody has noticed yet...

WaffleThief123 commented 4 years ago

Now that i've slept, and have woken up and had coffee in my system, I'm going to be doing some more investigation. Here goes nothing.

Autrix commented 4 years ago

Now that i've slept, and have woken up and had coffee in my system, I'm going to be doing some more investigation. Here goes nothing.

Mainly this happened after the ban wave for the bot developers who abused the API to broadcast messages from they're bots ( Sadly i was one of them who got suspended ). After that, bots all of sudden started to lose ( connection / not stable / refuse to connect to API )

i think there's a major change that happened but no one told us about it.

WaffleThief123 commented 4 years ago

@AutrixSalv Cool, thanks for the heads up. I opened an issue here on Github for Discord

https://github.com/discordapp/discord-api-docs/issues/1149

DoNotSpamPls commented 4 years ago

@wertyy102 v4 is the version that should be used, but it isn't documented.

WaffleThief123 commented 4 years ago

Looks like others are having the same issue, with whatever other voice API Calls they are making. I'm going to go ahead and suggest closing this issue as a non-issue to lavalink and just hope everyone sees the fact that it's discord's side.

If you see this please track the issue and make reports here as it IS NOT Lavalink's fault.

https://github.com/discordapp/discord-api-docs/issues/1149

NineBallo commented 4 years ago

Latest news on the issue for anyone scrolling, and possibly incorrect implementation on lavalink's end due to breaking api changes similar to discord.py.

Our investigation of this issue continues, and we've isolated 2 specific issues.

We recently changed our voice servers to proxy TCP traffic through cloudflare to stop some variants of naive DoS attacks.

Correct client implementations should have handled this change seamlessly. We announced that we would do this change back in July in the API server: https://canary.discordapp.com/channels/81384788765712384/381871767846780928/601151474738790512 and the proper way of connecting to our voice infra has been documented for a while now: c606741

The first issue is due to an incorrect implementation of our voice API in discord.py - I had committed a fix for this in July: Rapptz/discord.py@8fdcb4d however - no version has been released since June. So, discord.py doesn't work, because it's implementation is wrong. This is probably the issue affecting small bots using discord.py. This should be resolved in the next discord.py release (1.2.4) - and upgrading the dependency to that version when released should resole the issue.

The second is some DoS protection measure triggering on larger bots, causing them their connections to fail, regardless of library. This is the issue we are currently investigating.

EDIT -- Might be fixed EDIT 2 -- Confirmed with my bot working

We think the second issue is now resolved - based on our findings, the issue was not Cloudflare's fault. Specifically, as part of this change, we started supporting and allowing clients to negotiate a TLS v1.3 connection to our voice servers. This seems to break something deep down in Lavalink, and thus causes the connection handshake to hang and timeout.

In the interim, I've gone ahead and disabled TLS 1.3 - meaning connections will fall back to the old TLS 1.2 default.

The first issue is also resolved, following a release of v1.2.4 with the change for discord.py: https://pypi.org/project/discord.py/#history

odddellarobbia commented 4 years ago 
We think the second issue is now resolved - based on our findings, the issue was not Cloudflare's fault. Specifically, as part of this change, we started supporting and allowing clients to negotiate a TLS v1.3 connection to our voice servers. This seems to break something deep down in Lavalink, and thus causes the connection handshake to hang and timeout.

In the interim, I've gone ahead and disabled TLS 1.3 - meaning connections will fall back to the old TLS 1.2 default.
freyacodes commented 4 years ago

This was fixed a while ago. Closing.

infiniwave commented 4 years ago

Anyone still having this issue should revert from Java v14 to v13.

NightmareDiscord12 commented 4 years ago

Anyone still having this issue should revert from Java v14 to v13.

I'm using v13 and still getting the problem

Jirubizu commented 4 years ago

I am using java 14 and getting the issue. I thought it might be my dns servers but its not.

CapitaoFNCB commented 4 years ago

I'm using Java 14 and getting the issue too. Private bot for 2 guilds only, around 100 users...