search

Metro-Records / la-metro-councilmatic

:metro: An instance of councilmatic for LA Metro
MIT License
6 stars 2 forks source link

Google Indexing Blocked by robots.txt #901

Open neilarellano opened 2 years ago

neilarellano commented 2 years ago

Hi Team!

We received this alert from Google regarding indexing blocked by robots.txt. We found #175 and was wondering if we still need to suppress robots. Please advise.

image

hancush commented 2 years ago

Hey, @neilarellano – No, I don't think we need to block robots from the entire site since we implemented Captcha. Our pattern on other properties has been to restrict crawling on the search page, and add a site map so events and board reports can be discovered and indexed by crawlers, which is good for SEO. Sound ok to you all?

shrayshray commented 2 years ago

@hancush sounds good to me!

shrayshray commented 2 years ago

@hancush actually, lets hold off on this for now and discuss in our next meeting.

shrayshray commented 1 year ago

@camcachoo do you remember why we wanted to wait on this?

shrayshray commented 1 year ago

@hancush we're ready to move forward with this and stop blocking robots.