algolia / docsearch-configs

DocSearch - Configurations
https://docsearch.algolia.com/
MIT License
456 stars 1.12k forks source link

A 404 page link shows in the search result #944

Closed YiniXu9506 closed 5 years ago

YiniXu9506 commented 5 years ago

Do you want to request a feature or report a bug?

report a bug

If it is a DocSearch index issue, what is the related index_name ?

index_name= pingcap.com

What is the current behaviour?

If the current behaviour is a bug, please provide all the steps to reproduce and screenshots with context.

Step 1. Go to https://pingcap.com/en

Step 2. Search Haproxy in the search at the top navigation

Step 3. Scroll down to Deploy TiDB in Kubernetes on Your Laptop section, the links show there direct users to a 404 page (https://pingcap.com/docs/tidb-in-kubernetes/local-dind-tutorial/#access-the-database-and-monitor-dashboards). Like the screenshots:

image1 image2

What is the expected behaviour?

The links of 404 page should not be showed in the search result.

What have you tried to solve it?

I searched the 404 link (https://pingcap.com/docs/tidb-in-kubernetes/local-dind-tutorial/) in the submitted sitemap, and didn't find the link in the sitemap.

Any quick clues?

The 404 link was in sitemap, but we update the docs structure, so the URL also has been changed. Algolia might crawl the urls from the cache.

Any other feedback / questions ?

How often algolia bot crawls the URLs from a updated sitemap?

@s-pace PTAL, thanks

s-pace commented 5 years ago

Will be fixed at the next crawl. Closing in the meantime. Feel free to reopen of needes