Docs gone since 7.9

[CalebCurry]

I noticed the docs went away going from 7.8.1 to 7.9. Now, the readthedocs page is empty. If this is intentional as the 7.9 docs are not complete perhaps it'd be best to default to 7.8.1 docs in the meantime so users know something exists.

[jtv]

No it's not intentional. But something changed in how buildthedocs builds worked, and so the existing configuration suddenly no longer worked. It was quite annoying.

I thought I'd fixed that. Wasn't trivial. But now it turns out that I still need to do more...

[jtv]

By the way the docs didn't fully go away - you can still search, and find class/function documentation!

It looks as if the indexes and the Markdown docs are gone.

[alexolog]

That's a problem, we are using the docs heavily. Please fix them. Thank you!

[jtv]

I'll try, but I'm fairly ill right now. I think the documentation build logs are publicly accessible on readthedocs... If so, perhaps you can have a look and see if you can find the problem. They made some incompatible changes recently so the builds just suddenly started breaking, without any help from me.

Until then, all the documentation in there is also in the libpqxx source tree.

[jtv]

Also, you should be able to build the documentation on your own machine.

[alexolog]

Also, you should be able to build the documentation on your own machine.

How do I do it? It did not create a doc subdirectory when I built it.

[jtv]

I haven't done it in a while because Ubuntu's packaging of Firefox no longer lets me just view a file on the local filesystem. But it worked something like:

You may need some additional packages installed on your system - you'll have to dig around to find out which.

Configure a libpqxx build using the configure script. Pass the --with-documentation option on the command line.

Do a regular build. It should create the docs tree somewhere in the build tree. If it does not, that's typically a sign that one of those packages was not installed.

[jtv]

I've been trying various things by the way, but haven't succeeded yet at getting the readthedocs documentation to build properly again. :-(

[CalebCurry]

Perhaps we should just go from the raw .md files and move it over to something like docusaurus? Or even a plan readme folder? @jtv

[jtv]

I was thinking I might go with mkdocs, but still on ReadTheDocs. But frankly I feel pretty demotivated at this point. Something made an incompatible change and I'm supposed to guess what it is.

[egorpugin]

Is it possible to bisect somehow?

[jtv]

Bisect what though? I don't know that I get to choose versions for all the components involved. Nor am I sure which components might be relevant, or which versions have been running.

It could be something simple, like how CircleCI seem to have just moved their generically named config file from a custom directory into the project root. Or it could be something that I've been doing wrong all along but that never caused quite enough trouble. At any rate it feels like I pretty much have to start from scratch.

One problem IIRC is that I couldn't find a reliable way to navigate the filesystem on the build slave.

[jtv]

I think I've made some progress. I can get ReadTheDocs / Sphinx / Doxygen / Breathe to generate documentation. But it's all a single page, with the introduction all the way at the bottom. I'll still need to find a way to control that.

Unfortunately the documentation for Breathe is driving me up the walls. It skips the basics and goes straight into the details. It also seems to contradict itself. I'm sure it's alright for people who already know and don't need the documentation, but...

[tt4g]

@jtv FYI. Since Read the Docs basically uses the latest version library, it seems to be recommended that the version be fixed so that the document generation is not broken by the release of a new version.

It is recommended to use pip-tools to list the libraries and versions to be installed in requirements.txt. See: https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html#pin-your-transitive-dependencies

[jtv]

Will certainly do that when I get it working again @tt4g .

[jtv]

I've sort of got documentation, but not in a nice way... It's all in one page, with the introduction all the way at the bottom.

[jtv]

Thanks for the tip about master_doc @tt4g — but weirdly, neither master_doc nor root_doc seems to do anything!

[tt4g]

@jtv I have investigated conf.py and plugins since then, but have not found the cause. It seems difficult to pinpoint the cause since several libraries are used for document generation.

[jtv]

Well in a disturbing way I'm glad it's not just me, thanks. If only we had better documentation... I may have to search for examples instead of the official docs.

[tt4g]

I used the ReadTheDocs Docker image to investigate what was going on.

As it turns out, prior to version 7.9, Doxygen generated documents were hosted on ReadTheDocs. Doxygen generates XML and HTML documents in doc/xml and doc/html. When I accessed https://libpqxx.readthedocs.io/en/7.8.1/, I saw doc/html/index.html generated by Doxygen.

It seems that doc/html/index.html was being displayed because the Doxygen generated HTML file was copied as a ReadTheBox document by html_extra_path.

https://github.com/jtv/libpqxx/blob/21afb0d6ec3d31424df6b7aac04c6b20c55c0a65/doc/conf.py#L135-L139

However, since version 7.9, documents generated by Sphinx from doc/index.rst seem to appear instead of doc/html/index.html. I think the internal behavior of Sphinx has changed so that index.html generated from index.rst overwrites doc/html/index.html copied by html_extra_path.

Unfortunately, since Breathe does not seem to be able to generate the same index.html as Doxygen, it seems better to consider hosting Doxygen-generated documents in ReadTheDocs without using Sphinx.