Bungee Crashes - Packets bots?

[KarolCieslar]

My bungeecord version: git:BungeeCord-Bootstrap:1.12-SNAPSHOT:b5f17e7:190 by md_5 My server is online mode: true

Hi, all of last 3 days I am having problems with BungeeCord.. Someone just crash my Bungee but i dont know how..

Crashes are randomly, just when someone who do it just want to do it.. BungeeCord log do not show anythink. BungeeCord just stop responding

I disable all my plugins becouse i think it was problem but Bungee crashes still. Sometimes when i have plugins installed and when not too i i have a lot of "hs_Err_pid" files". Bungee just crash becouse OutOfMemory when "attacker" crash bungee. https://pastebin.com/aUXBhSSG

ALL of the time when crash in the last line of logs is " [INFO] [/IP HERE] <-> InitialHandler has connected" but IP is always proxy from countries which my players are not playing on my serever.

Question is.. How someone do this and how to protect my Bungee?

[pjorun]

I have this same problem, but i know much more about this issue. md_5 made not good secure code about 2 years ago and he do not check it. Bots are sending packets and this make huge CPU usage and crash. All other fork BC do not secure this.

[Black-Hole]

Java is crashing with an internal exception, you should update your Java version, Java 8 Update 74 is more than two years old.

@pjorun If you believe that's a fixable issue that could be "fixed" with a simple fix, please create an issue for it. I believe there is no simple fix for DDoS attacks. Those should be handled by the infrastructure provider. There are providers with a decent DDoS protection.

[pjorun]

@Black-Hole yes, this is fixable. I have latest Java version but this is now fixable in this site. First your BC have huge CPU usage 500/700% later ram usage and crash. There is a problem with receiving packets.

[Janmm14]

Please try out removing/renaming the file native-compress.so inside the jar, run that. It will then use java (de-)compression instead of native. Please report new exceptions appearing.

This looks to me like a new error inside the native compression code either in zlib or in bungee's wrapper.

[md-5]

Your java is eons outdated

This looks to me like a new error inside the native compression code either in zlib or in bungee's wrapper.

I don't think there is any error aside from invalid data being sent. The crash is likely a java bug and probably already fixed.

[Miracle407]

I have exactly the same problem as you up from few days. I did not change nothing up from year. Someone is crashing my bungee. I see nothing more in logs than "InitialHandler has connected" at one of last lines where one of IPs is VPN. It is really java problem and update will help? Please help me. I have a lot of players and they are leaving my server due to this kicks :(

[MorkaZ]

Same problem. Someone is crashing big networks (including mine) using this method. @md-5 I have java 8 172 and my bungee is having same issues.

[md-5]

Then post your crash report.

[MorkaZ]

@md-5 (NULL = NULL SYMBOL): NULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULL This is last line in my proxy logs. Everything stopped responding on my server because bungee took all RAM and a lot of CPU. Is there any way to enable debug or something that will show what is happening in background?

[KarolCieslar]

I have the same log sometimes!

[pjorun]

I use latest java8, latest bc and latest spigot. This is not Java problem.

[md-5]

Bungee does not print nul in logs

[KarolCieslar]

@md-5 I make packet receiver. When "attacker" crash my bungee i have ~700% bungee CPU usage and in console i have that packet: https://pastebin.com/4bG9cByD

[Janmm14]

maybe a zip bomb?

[KarolCieslar]

@Janmm14 What do You mean?

[MorkaZ]

@md-5 here is what is causing it. Good job @GlobooX By the way: why it is closed when bug is not fixed?

[md-5]

I have disabled forge support by default if the bug lies there, otherwise I see nothing that's an exploit aside from DoS style spam.

[KarolCieslar]

@ghacproductions Can You update HexagonMC?

[MorkaZ]

@md-5 Is there any way to disable it in previous version? I am using HexagonMC fork (I have some 1.7 "classic mc" servers) but it is 1 version behind :(

[md-5]

So you're not even using BungeeCord.

This tracker is for BungeeCord only. For all I know you've wasted my time with a bug that doesn't actually even exist in Bungee.

[MorkaZ]

We have reported it there because it is BungeeCord problem. HexagonMC's Bungeecord is just a fork that adds 1.7 protocol feature. Note that people above are using BungeeCord and they have this problem too. I am using normal bungeecord on my second network and it got nuked too but fortunetly just 1 time.

[KarolCieslar]

@md-5 Yes, it was BungeeCord bug, becouse i try use normal BC to test and attacker using the same protocol to crash my server.

[Xayanix]

@md-5 you are such a egoistic person, what if i tell you that this bug exist in BungeeCord since 2016? I suggest you look at packetQueue array at ForgeClientHandler.

For all I know you've wasted my time with a bug that doesn't actually even exist in Bungee. kek

[md-5]

@Xayanix if you know the source of the bug then why not open a PR fixing it?

I spend hours and hours of my time to write this software for free (I spent more than six hours today working on 1.13), it is hardly much for me to ask that you are actually using my software and not a fork before opening a bug report.

[Xayanix]

I'm not hating you. I just don't like how you treat people here. Good luck with 1.13 and have a nice day ;)

[Fejm]

I've recreated bot client based on your logs and found other issues inside bungee.

Should be fixed now.

https://github.com/SpigotMC/BungeeCord/commit/968916c0b83550961e96da858f4be29a81c785b8

[MorkaZ]

@Fejm Byte Array had maximum value while attack, so it will not help. The problem is when someone will sent 10000+ packets in one moment. Memory will be filled in instantly and a lot of CPU will be taken due to java's garbage collector.

This channel between forge and server is broken and should be better programmed. Do not say it is not.

1. You do not have to be on the server to sent this packet (server is accepting this packet without connected player).
2. You can spam this packet. It should be sent when server will request it, else packet should be rejected.
3. Adding tons of checks will not help. The problem is in development stage. Listener works now like ping listener and this is sad..

[Fejm]

@MiMic10110 after these fixes i'm unable to crash bungee with 250 bots and packets spam while without patch it's possible to crash bungee with one bot.

And you are wrong. PluginMessage packet is on PLAY state so you need to full handshake to server. You can't send it without connected player.

Sorry, my english is not so good.

[MorkaZ]

@Fejm Ok, if it is true, then good job, but really this packet should be sent after server request because it is just single message data.

[Black-Hole]

Next time the attacker might use 10000 bots. You can't mitigate that at the application layer. So you have to use a provider that is providing DDoS protection, escapecelly if you're running an offline server.

[MorkaZ]

@Black-Hole This is why I am informing that this messaging channel is badly coded. Packets should be only accepted if server will request for them if they are single like this one.

[Leymooo]

@Black-Hole DDoS protections usually work only with layer 3/4. (Like OVH, Hetzner, Ddos-gurad), layer7 protection cost a lot of money, something like a 200-300$ per ip with 50-100Mbit. I think we can mitigate L7 attack, but need a good network speed and a good CPU.

You can check a my fork of bungeecord with built in antibot (captcha and GeoIp).

[MorkaZ]

Just buy server with best anti-ddos protection. Like OVH's GAME servers. This antiddos is activating instantly on attack.

[Leymooo]

@MiMic10110 OVH game antiddos does not protect from L7 attacks

[Black-Hole]

I can asure you that OVH is protecting against L7 attacks. There are special filters for Minecraft in place.

[Leymooo]

Filters for minecraft only for UDP(Minecraft Bedrock, Query). For TCP there is no filters.

[Black-Hole]

You might have misread their firewall description. There is special firewall hardware for UDP attacks. Mitigating TCP attacks is much easier using filter rules. Look, I'm only a customer of OVH services, so I won't look up the pages where OVH describes their firewall in much detail. There are DDoS attacks every now and then to the server I'm working for. Sometimes the server CPU will be stressed for up to 2 minutes. But after the DDoS protection kicks in, players can play without any lag or connection loss.