search

discord / discord-api-docs

Official Discord API Documentation
https://discord.com/developers/docs/intro
Other
5.95k stars 1.26k forks source link

Discord Embeds seem to not crawl our website #6208

Closed SgtSlade closed 1 year ago

SgtSlade commented 1 year ago

Description

Discord,

We've switched our site hosting service to Hostinger's VPS and since then, our site, https://arclightbase.com/, has lost it's message embeds. We've ran down the list of issues that might cause this issue, including...

  1. OpenGraph tags being broken. We've ruled that out by using OG checkers and a temporary hosting service to make sure embeds display
  2. SSL chains now working properly. Using SSLLabs we've ruled out our chains not behaving as they should be
  3. Unsupported cipher suites. Again, using SSLLabs we've made sure that all our TLsv 1.2 and 1.3 ciphers are supported by the discord crawler.

After examining NGINX access logs we've noticed the complete lack of request attempts from the discord crawler. Our current theory is that the request is somehow being blocked, but without any information on the crawler request response i doubt we'll be able to fix the issue by ourselves.

Steps to Reproduce

Attempting to paste any build link should result in an embed, but it doesn't. Our control group link was... https://arclightbase.com/

As it makes for a good benchmark

Expected Behavior

An embed containing the build preview, name and description should pop up.

The screenshot below is using a temporary hosting service to demonstrate the end result:

Discord_SShg2gfmgw

Current Behavior

Only the bare link remains and no OG data is displayed.

Screenshots/Videos

No response

Client and System Information

Server information: System - Ubuntu Static file server - Nginx Webapp server - Django with Gunicorn

jhgg commented 1 year ago

I looked into this briefly, it looks like your server (or hosting provider perhaps) may be blocking traffic from us.

I cannot even curl your website from one of our unfurler pods. Since we have no specific blocking of websites here, I think the issue must be upstream of us.

Good luck with figuring this out!

SgtSlade commented 1 year ago

@jhgg Is there any way we could get a verbose output of the curl you tried? We've contacted the Hostinger support but they deny having any form of traffic blocking or anything like that. We also don't have a pivot point to test anything, since other crawlers we tried (Facebook and Twitter) seem to get through just fine.

jhgg commented 1 year ago

There wasn't much output. It hangs trying to connect to the hostinger's IP that your domain resolves to, until I kill the process.

scinscinscin commented 1 year ago

As of October 18, 2023 it looks like hostinger has unblocked requests from Discordbot, I have two sites previously that would not embed and it looks like everything is fine now.