search

omz13 / kirby3-wellknown

Kirby3 plugin to provide wellknown files (/robots.txt, .well-known/, etc)
12 stars 0 forks source link

Specific content for robots.txt #4

Closed Uelii closed 5 years ago

Uelii commented 5 years ago

Hi David,

First, thank you for creating all those useful plugins for Kirby CMS :)

I want to insert specific content to my robots.txt file. As described in your README, I added the following entry into the config.php:

image

Purpose of this entry is to exclude a page from being crawled by the robots. However, it seems that this content is not put into the robots.txt file. When I call the URL to this file I only get the "standard" content for the sitemap (example domain):

Any use of this file - robots.txt - or failure

to obey the robots exclusion standards set

forth at http://www.robotstxt.org/ is strictly prohibited

Sitemap

SL sitemap

Sitemap: https://domain.com/sitemap.xml

Can you tell me what's wrong?

Uelii

omz13 commented 5 years ago
  1. Its a bug. Fix coming soon.
  2. The value is a string not a string in an array. (Upcoming fix will allow for this.)
  3. '\n' expansion is not done. Upcoming fix will allow for this.
Uelii commented 5 years ago

Works, many thanks!