search

jakartaee / specifications

Documentation base for Specification pages be published at jakarta.ee via Hugo and git submodules
Eclipse Public License 2.0
119 stars 79 forks source link

Indexing Issues with apidocs pages #583

Open excentrickristy opened 1 year ago

excentrickristy commented 1 year ago

There is a duplicate URL issue present in the apidocs as well because of the frames/no frames option - not sure if this is the right place to get that fixed?

Example: https://jakarta.ee/specifications/platform/8/apidocs/javax/servlet/servletcontextlistener https://jakarta.ee/specifications/platform/8/apidocs/index.html?javax/servlet/ServletContextListener.html

Possible solution: Set the “no frames” version of the page as the canonical URL, but is it also possible to disallow crawling in robots.txt for URLs containing /apidocs/index.html plus parameters? This would help free up crawl budget if it won’t create issues in accessing unique content.

excentrickristy commented 1 year ago

Is this something that we can address at a server level?