[CORE] WRONG URLs in common.php::regenerateRobots()

[emanwebdev]

common.php::regenerateRobots()

Generated URLs are wrong when OC is installed in a /sub-directory/

All generated URLs are absolute

should take /sub-director(y|ies)/ into account

Can URLs in robots.txt be relative?

Anyway, robots.txt should be placed at the ROOT of the website!!!

we have

oc::fwrite('../robots.txt', $robots_content)

should be

oc::fwrite(SITE_ROOT.'/robots.txt', $robots_content)

[neo22s]

Seems more correct ;)

[emanwebdev]

Not so simple alas....

From https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt :

"The robots.txt file must be in the top-level directory of the host" "Crawlers will not check for robots.txt files in subdirectories"

OK, corrected as above

but....

exemple of an OC website found from the Testimonials on your website

http://www.negoce-land.com/robots.txt http://www.negoce-land.com/petites-annonces/robots.txt

have 2 robots.txt alas.

but now the new issue

we must MERGE these 2 robots.txt files!!!!

and the merging must be clever enough not to duplicate entries related to OC...

i.e. subsequent calls to regenerateRobots() must look for previous entries, remove them, and add the new different ones

Solutions?:

• at install, rename & save initial robots.txt for subsequent merges (easier but not so clean solution...?)
• str_replace but if FRIENDLY_URL changed, won't work coz new content will not match previous, again coz of the Disallow: /$con

[neo22s]

people should not install in a directory, only problems....htaccess form other apps may interrupt etc...

Do not worry much.

;)