discord / discord-api-docs

Official Discord API Documentation
https://discord.com/developers/docs/intro
Other
5.93k stars 1.26k forks source link

Discord crawlers won't crawl Website #6406

Closed TubaApollo closed 1 year ago

TubaApollo commented 1 year ago

Description

I had a blocklist in place that included Discord crawler IPs, so crawling obviously wasn't working, now after disabling it, it still wont crawl it at all. I can't even find any requests made from discord (at least not with any discord user agent), embeds on other platforms (e.g. Facebook) work fine. It's definitely something host related as this isn't the only domain on that IP I'm having issues with. I am pretty sure that there shouldn't be any captcha/block, preventing it from crawling so I am kinda clueless.

Domain: https://test.apollo.moe

Steps to Reproduce

Paste link anywhere in discord See embeds not working (Other hosts with same content do show embeds)

Expected Behavior

Embed with Text + Image show up

Current Behavior

It doesn't show up at all

Screenshots/Videos

No response

Client and System Information

Web-Client/Desktop-Client/Android-App

Server is using nginx 1.12.4 In this case these are static html/CSS files but same problems occur with PHP sites.

RedDaedalus commented 1 year ago

Using Discord's embed debugger, it appears that your website is timing out when Discord is trying to reach it.

TubaApollo commented 1 year ago

Great tool, no idea why it times out though, I guess i'll have to figure that out myself, thanks.