blocked image when sharing on twitter - cloudflare bot protection

[raramuridesign]

Hi Support I have been troubleshooting for sometime and see that the images which are used in Opengraph or Twitter Cards can be blocked.

Our format is something like this using wsrv. This does not get pulled into the preview on twitter.

<meta name="twitter:image" content="//wsrv.nl/?w=1200&h=630&fit=cover&q=90&url=capeleopard.org.za/media/k2/items/cache/11ff87b754b564cf2c60e2628d5ec100_M.jpg" />

If we revert back to the original code, like below, then the twitter preview works.

<meta name="twitter:image" content="https://capeleopard.org.za/media/k2/items/cache/11ff87b754b564cf2c60e2628d5ec100_M.jpg" />

After further investigation, it seems that you will need to change some settings in cloudflare to allow twitter or facebook bots to access these image references. This can be done by

1. disabling the bot fight mode
2. disabling hotlinking
3. or adding some rules which target specific file extentions to bypass the above.

Cloudflare should allow you to bypass the check on specific extentions like: jpg, png etc.

The main issue, is that using the image like this prevents the preview from loading. Would it be possible if you could test this and let us know if a solution can be looked at? Thanks Matthew

ps for now I have amended this in the code to bypass this so our images load, but if a fix is added, then that would be perfect. ;-)

[raramuridesign]

@andrieslouw Here is a rule which works on our side for allowing Twitter Bot Any bot which needs to access the images could be added to this WAF ruleset. Meaning you can protect the content from other malicious usage. M

[kleisauke]

I think the issue is that Twitter Cards require absolute URLs, it doesn't support protocol relative URLs. See for example: https://www.drupal.org/project/metatag/issues/2925974

-<meta name="twitter:image" content="//wsrv.nl/?w=1200&h=630&fit=cover&q=90&url=capeleopard.org.za/media/k2/items/cache/11ff87b754b564cf2c60e2628d5ec100_M.jpg" /> 
+<meta name="twitter:image" content="https://wsrv.nl/?w=1200&h=630&fit=cover&q=90&url=capeleopard.org.za/media/k2/items/cache/11ff87b754b564cf2c60e2628d5ec100_M.jpg" /> 

[raramuridesign]

@kleisauke I have just tested this, and it does not work.

[kleisauke]

According to https://cards-dev.twitter.com/validator this is due to our robots.txt, in which we defined that all proxied images should not be indexed.

INFO:  Page fetched successfully
INFO:  8 metatags were found
INFO:  twitter:card = summary tag found
INFO:  Card loaded successfully
WARN:  The image URL https://wsrv.nl/?w=1200&h=630&fit=cover&q=90&url=capeleopard.org.za/media/k2/items/cache/11ff87b754b564cf2c60e2628d5ec100_M.jpg specified by the 'twitter:image' metatag may be restricted by the site's robots.txt file, which will prevent Twitter from fetching it.

See also: https://github.com/weserv/images/issues/165#issuecomment-438777292.

[raramuridesign]

@kleisauke Ok, I see the issue. So I guess this will not be changed and or updated ? M

[jesperordrup]

I just ran into the same problem with twitter. wsrv.nl robots.txt does not allow twitter to fetch og:image

User-agent: *
Disallow: /*?*
Allow: /$

I guess we cant use wsrv for og images?