Open dimaqq opened 2 months ago
Hi. I'm not sure to understand the issue. Can you expand on what you are trying to do and what's the error you are getting?
I imagine that the fact that rtd is proxies by cloudflare is configured in rtd, something that I don’t have access to.
cloudflare, in turn shows a “I’m not a robot” page when Facebook tries to load a “social image”. Because that’s server-to-server communication, human check can never pass.
instead, rtd’s cloudflare account should be configured to allow machine/direct/non-browser access from a white list of ip addresses.
Thanks for the explanation. I'm pinging @ericholscher here because I think he will have more context about this.
Seems like the site is working to me?
-> curl -IL https://ops.readthedocs.io/en/latest/
HTTP/2 200
The site is working fine, it's facebook loading the "social image" or preview that doesn't work.
You can use this tool from facebook to debug:
https://developers.facebook.com/tools/debug/?q=https%3A%2F%2Fops.readthedocs.io%2F
Btw, this seems to affect all projects on .io, heres pypi:aapns
https://developers.facebook.com/tools/debug/?q=https%3A%2F%2Faapns.readthedocs.io%2Fen%2Flatest%2F
CloudFlare advisees to "check your security events in your Cloudflare dashboard"
ref: https://community.cloudflare.com/t/link-preview-is-not-working/643722/2
Oh... I'm guessing this is an issue from the locked down configuration we have for Facebook's AI crawler which is spamming our site: https://about.readthedocs.com/blog/2024/07/ai-crawlers-abuse/ -- is there a specific user agent that they are using for these requests, we can. perhaps unblock it, but their abusive behavior is pretty awful, so might not be possible.
Details
Expected Result
Either being able to use social previews, or clean error if previews are broken
Actual Result
Example solution: https://community.cloudflare.com/t/attention-required-message-when-sharing-link/88999/3