search

MichaIng / DietPi

Lightweight justice for your single-board computer!
https://dietpi.com/
GNU General Public License v2.0
4.87k stars 496 forks source link

dietpi.com seems to be down #3341

Closed Joulinar closed 4 years ago

Joulinar commented 4 years ago

@MichaIng I'm getting again the Cloudflare's Always Online message

Error 522 - Connection timed out 😠

bbsixzz commented 4 years ago

Works great from Sweden, must be local.

Atticuss commented 4 years ago

Not local -- confirmed issue for myself as well. Philadelphia, PA. Is there a mirror for the images anywhere? Everything I found pointed back to dietpi.com.

MichaIng commented 4 years ago

Hey guys, I updated phpBB3 which should usually go quick and smooth, but me dump forget to disable extensions and clear cache first and two extensions were incompatible leading to phpBB error. However the extensions were easy to fix luckily and issue is solved.

But hmm, 522 @Joulinar @Atticuss? It should have been 500 internal server error, when accessing the forum only 🤔. However can you confirm its working again?

Joulinar commented 4 years ago

Nope its already on the main web side dietpi.com, not the forum. Seems similar as last time. There as well it was depending on the location.

MichaIng commented 4 years ago

@Joulinar You're right, same symptoms on server as well, cannot connect/ping certain resources... what a coincidence, I was just happy to have phpBB update finished and now this. Good that we already have a ticket open on VPS provider, I'll reopen it, hopefully we get some investigation this time.

MichaIng commented 4 years ago

It seems like we were attached, brute-force attempt and DDoS.

Starting with 23:11:42 UTC I see a bunch of SSH connection attempts. First ~5 times around 10 each within the same second, before blocked by fail2ban, then 20 to 30 each from different IP, hence fail2ban did not trigger, all ending with "SYN flooding on port 443". Now I am wondering, why port 443, since this is not the SSH port. Checking Cloudflare statistics:

So I guess some mechanism on the VPS network blocked the related Cloudflare IPs where those requests have gone through. Through their GUI I see some DDoS protection firewall but the "disable" button does not work 🤔. However have to ask Cloudflare why this was not blocked in the first place, perhaps we have to place/change some firewall/filter rules.

Ah, meanwhile APT works again, I guess @Joulinar @Atticuss access for you works again as well?

Joulinar commented 4 years ago

@MichaIng yep we are back online. Thx for investigating it that late during night. I hope it did not damage anything on the servers.

Atticuss commented 4 years ago

Confirmed back up for myself as well. Also worth noting that Cloudflare won't protect you if the scanner was hitting your IP directly -- CF only be able to see the attack if the scanner is hitting you by domain name, as then all traffic gets routed through CF itself for them to see. Probably just an internet-wide scanner that saw 22 open and threw it's usual giant list of brute-force attempts at it. That's on you to protect against.

https://support.cloudflare.com/hc/en-us/articles/200170166-Best-Practices-DDoS-preventative-measures

MichaIng commented 4 years ago

@Atticuss As said, the SYN flooding was on port 443, also since I found it in the Cloudflare logs, it indeed was proxied through Cloudflare, from all I know. Port 22 requests on the Cloudflare proxied domain should be blocked, but we have bypassing domain as well for SSH of course. I'll ask Cloudflare about this, probably they have some more information and suggestions how to protect against such attacks.

tanderson1992 commented 4 years ago

I hope this is the appropriate place to report. I'm a new user and tried to sign up for the DietPi forum so that I could ask a question. After filling in the registration information, nothing happens when I press the submit button. After several tries, I noticed a box in the lower right corner. Hovering over the box expands the message, "This site key is not enabled for the invisible captcha." I'm guessing the registration doesn't submit because the invisible captcha isn't working properly.

MichaIng commented 4 years ago

@tanderson1992 Many thanks for your report. Looks like the reCAPTCHA keys faded out or something, I just created and applied new ones and for me the reCAPTCHA icon appears again. Can you please try again to register?

tanderson1992 commented 4 years ago

Registration works now, thank you!

Edit: The initial registration worked, but the confirmation email didn't send.

MichaIng commented 4 years ago

@tanderson1992 Can you please try it again. While investigating the issue, I enabled IPv6, but mail provider blocked that new address by default... I just unlocked it.

tanderson1992 commented 4 years ago

Thank you. I can't register because my name is already in use. Is there a way to ask it to re-send the activation email, or can you delete user tanderson92 and let me try again?

MichaIng commented 4 years ago

@tanderson1992 I just triggered a reminder mail for all newly registered accounts, which include the activation link.

tanderson1992 commented 4 years ago

That worked, thank you!

MichaIng commented 4 years ago

Lol, we are working for the devil:

- Uptime : up 6 weeks, 6 days, 6 hours, 46 minutes

Just the 4 doesn't fit, dammit 😄.

Joulinar commented 4 years ago

Software from hell 😆