Closed meshi6 closed 2 months ago
We need to update the robots.txt
file for tangle.tools. cc @monaiuu.
User-agent: *
# Ensure important content accessibility
Allow: /
# Specify sitemap location
Sitemap: https://www.tangle.tools/sitemap.xml
The current file appears incorrect.
Double checking on the removal of all disallow links
cc @meshi6
Updated:
Is there an existing issue for this?
Current Behavior
Problem: Our dApp and website share the same DNS and web address: tangle.tools. As a result, dApp errors and server timeouts negatively impact our website’s ranking on Google Search because these issues affect the web vitals that Google considers for ranking.
Expected Behavior
Solution: Instruct Google to disregard the dApp URLs and their web vitals. TASK 1 Add a robots.txt file with the appropriate instructions to the root level of the CNAME app.tangle.tools. Specifically, place it at https://app.tangle.tools/robots.txt. Here’s the content for the file: makefile Copy code
User-agent: * Disallow: / https://www.app.tools/ https://www.app.tools/ https://app.tools/ https://app.tools/ https://app.tangle.tools/nomination?tab=Nominations http://app.tangle.tools/nomination http://app.tangle.tools/nomination/ http://app.tangle.tools/claim http://app.tangle.tools/claim/
Steps To Reproduce
No response
Screenshot
Anything else?
No response