Googling FIWARE topics brings up outdated documentation

[Maghnie]

Bug description Not sure if this is the best place for this "bug report", because this is more of a "bug" in the user experience with googling FIWARE documentation. For example, when I googled a FIWARE concept, the top result led to a seemingly outdated documentation page. (screenshot below)

How to reproduce it Since Google search results are personalized, not sure how this can be reproduced in a stable way.

Expected behavior When googling for a FIWARE concept, only the updated docs should show up in the results, or the outdated docs could redirect to the newest docs. So in the example above, instead of: https://fiware-orion.readthedocs.io/en/1.3.0/user/federation/index.html , this page should show up in Google: https://fiware-orion.readthedocs.io/en/master/user/federation.html

Additional information

[fgalan]

As far as I know the Google algorithm show first the pages that have many other pages linking to them. Thus, for that particular example, I guess that the document https://fiware-orion.readthedocs.io/en/1.3.0/user/federation/index.html is more linked that the https://fiware-orion.readthedocs.io/en/master/user/federation.html

To change this behaviour we should be able to:

• Change the links in external web pages that we don't even know (only the Google spiders know :) and that in any case are pages that don't belong to us so we cannot change them
• Change the Google indexing algorithm. Only Google itself can do this :)

As you can see, this is completely out of the scope of our rearch.

[fgalan]

I would like to take this opportunity to tell you that documentation about the Orion NGSIv2 API has been recently restructured and improved, available at this link:

https://github.com/telefonicaid/fiware-orion/blob/master/doc/manuals/orion-api.md

Please, update your bookmarks with it.

[Maghnie]

I'm not sure either how to block Google from showing outdated pages, but there might be a way with the "noindex" meta tag: https://developers.google.com/search/docs/crawling-indexing/block-indexing#:~:text=You%20can%20prevent%20a%20page,other%20sites%20link%20to%20it.

Another solution, possibly simpler, would be to display a sort of warning at the top of old documentation pages, while linking to the latest docs, as scipy does here: https://docs.scipy.org/doc/scipy-1.3.1/reference/

[fgalan]

Another solution, possibly simpler, would be to display a sort of warning at the top of old documentation pages, while linking to the latest docs, as scipy does here: https://docs.scipy.org/doc/scipy-1.3.1/reference/

I understand that this is already happening. For instance if you go to https://fiware-orion.readthedocs.io/en/3.1.0/ you'll see:

[Maghnie]

Yes, that looks great. Something like that would be perfect to have on all outdated doc pages.

It seems that docs with a version less than 2 don't show this warning, e.g.: https://fiware-orion.readthedocs.io/en/1.15.0/user/multitenancy/index.html

[fgalan]

It seems that docs with a version less than 2 don't show this warning, e.g.: https://fiware-orion.readthedocs.io/en/1.15.0/user/multitenancy/index.html

Not sure why it works that way... we have no control on how readthedocs renders their pages. Maybe you could open an issue at readthedocs about that, please? In that case, please include the link to that issue as comment here.

Thanks in advance!

[fgalan]

After the discussion on https://github.com/readthedocs/readthedocs.org/issues/9633 if you tell me which versions are not correct I can relaunch the build for all them. Thanks!

[Maghnie]

It seems that all versions before 2.0 were affected by this bug, so I'd say only these below need a relaunch 😅

• [ ] 1.15.0
• [ ] 1.14.0
• [ ] 1.13.0
• [ ] 1.12.0
• [ ] 1.11.0
• [ ] 1.10.0
• [ ] 1.8.0
• [ ] 1.7.0
• [ ] 1.6.0
• [ ] 1.5.0
• [ ] 1.4.1
• [ ] 1.4.0
• [ ] 1.3.0
• [ ] 1.2.3
• [ ] 1.2.2
• [ ] 1.2.1
• [ ] 1.2.0
• [ ] 1.1.0
• [ ] 1.0.0
• [ ] 0.28.0
• [ ] 0.27.0
• [ ] 0.26.1
• [ ] 0.26.0
• [ ] 0.25.0
• [ ] 0.24.0
• [ ] 0.23.0

[fgalan]

I have relaunched compilation in all ones in the above list. However, given that sometimes it has been concurrently and I'm sure what RTD does in that case, could you have a look to all them (checking the ones that are already fixed), please? Thanks!

[LZimmermannLZI]

I checked the above versions to include the warning message and it is present in all entries:

• [x] 1.15.0
• [x] 1.14.0
• [x] 1.13.0
• [x] 1.12.0
• [x] 1.11.0
• [x] 1.10.0
• [x] 1.8.0
• [x] 1.7.0
• [x] 1.6.0
• [x] 1.5.0
• [x] 1.4.1
• [x] 1.4.0
• [x] 1.3.0
• [x] 1.2.3
• [x] 1.2.2
• [x] 1.2.1
• [x] 1.2.0
• [x] 1.1.0
• [x] 1.0.0
• [x] 0.28.0
• [x] 0.27.0
• [x] 0.26.1
• [x] 0.26.0
• [x] 0.25.0
• [x] 0.24.0
• [x] 0.23.0

Unfortunately the link in the warning note is not working properly. Clicking the link in a subfolder of the documentation provokes a 404 error. The reason seems to be that the link only changes the version number in the URL. The URL structure of the docs seem to have changed in the newest version. Example: The actual URL for the pagination subtopic of v 3.7.0 looks like this: https://fiware-orion.readthedocs.io/en/3.7.0/user/pagination.html

The redirected link of the warning note of v 1.15.0 looks like this: https://fiware-orion.readthedocs.io/en/3.7.0/user/pagination/index.html

The Problem doesnt occur at the starting page of each version doc.

[fgalan]

Thanks for your review!

Unfortunately the link in the warning note is not working properly. Clicking the link in a subfolder of the documentation provokes a 404 error. The reason seems to be that the link only changes the version number in the URL. The URL structure of the docs seem to have changed in the newest version. Example: The actual URL for the pagination subtopic of v 3.7.0 looks like this: https://fiware-orion.readthedocs.io/en/3.7.0/user/pagination.html

The redirected link of the warning note of v 1.15.0 looks like this: https://fiware-orion.readthedocs.io/en/3.7.0/user/pagination/index.html

The Problem doesnt occur at the starting page of each version doc.

Not sure how to fix this. It seems to be a RTD thing, isn't it?

Maybe you could open an issue on RTD and include the link as comment here (for traceability), as you did before, please? Thanks!

[Maghnie]

As suggested in readthedocs/readthedocs.org#9644, the page redirects approach could be used to correctly reroute older subpages of the documentation.

Maybe something like this can be added to the readthedocs configuration:

Type: Page Redirect
From URL: /multitenancy/index.html
To URL: /multitenancy.html

With some kind of regex in place of multitenancy.

[fgalan]

Maybe you could do a pull request on Orion repository to show how it looks like, please? Thanks!

[Maghnie]

Maybe we can, but this issue is gonna need the "backlog" label 😅

[fgalan]

Maybe we can, but this issue is gonna need the "backlog" label 😅

No problem. "backlog" label assigned ;)