Why it is a problem doesn't make much sense to me, since I wasn't
blocking it in our robots.txt, but Google seems to think it is
blocked. However, when users see this content, they are actually looking
at fossrit.github.io/calendar/ and not the above URL, so we actually
don't need Google or any search engine crawling those pages.
So, this adds Front Matter that explicitly excludes calendar events in
the sitemap and also adds a Disallow rule to tell search engines not
to crawl that directory.
This should fix the error reported in the Google Search Console.
jwflory
commented
4 years ago
Google Search Console threw an error when it tried to crawl this URL:
https://fossrit.github.io/meetings-meetups/2020/03/26/virtual-foss-hours/
Why it is a problem doesn't make much sense to me, since I wasn't blocking it in our
robots.txt, but Google seems to think it is blocked. However, when users see this content, they are actually looking at fossrit.github.io/calendar/ and not the above URL, so we actually don't need Google or any search engine crawling those pages.So, this adds Front Matter that explicitly excludes calendar events in the sitemap and also adds a
Disallowrule to tell search engines not to crawl that directory.This should fix the error reported in the Google Search Console.