search

BoldGrid / w3-total-cache

GNU General Public License v2.0
152 stars 85 forks source link

Googlebot can't crawl CSS #419

Closed MartijnOud closed 3 years ago

MartijnOud commented 3 years ago

I noticed in 2.1.7. there is a change "Disallow crawling of cache directory" from #147. Is there a way this could be made into a setting so I can turn it back on?

Googlebot needs to crawl the CSS of my website to be able to see the responsive website. This changes make it so Google cant and starts throwing Search Console warnings about the website failing the mobile friendly test.

https://developers.google.com/search/mobile-sites/mobile-seo/responsive-design

Important: Be sure not to block the crawling of any page assets (CSS, JavaScript, and images) for any Googlebot using robots.txt or other methods. Being able to fully access these external files helps our algorithms detect your site's responsive web design configuration and treat it appropriately.

mavas84 commented 3 years ago

Hello @MartijnOud

Thank you for training the time to report this and open this ticket. The problem with the recent update has been reported numerous times and we are working on a fix! Thank you for your patience!