[docu] qhelpgenerator on doxygen

ferdymercury commented 3 years ago

Explain what you would like to see improved

I daily use QtCreator in my ROOT-based projects, and it would be great if there would be a config-flag to build ROOT with the GENERATE_QHP enabled in the doxygen file. https://github.com/root-project/root/blob/master/documentation/doxygen/Doxyfile#L1514

This way, one can consult the Doxygen documentation directly within QtCreator, no need to use a web browser, which is also great if you don't have internet connection. See e.g. https://bugreports.qt.io/secure/attachment/48119/Added_doxygen_doc_2.PNG

Optional: share how it could be improved

cmake config flag, or automatically detect if qhelpgenerator is found as done in llvm https://github.com/root-project/root/search?q=qhelpgenerator

Setup

   ------------------------------------------------------------------
  | Welcome to ROOT 6.25/01                        https://root.cern |
  | (c) 1995-2021, The ROOT Team; conception: R. Brun, F. Rademakers |
  | Built for linuxx8664gcc on May 17 2021, 10:49:42                 |
  | From heads/master@v6-25-01-861-g5ea261143f                       |
  | With                                                             |
  | Try '.help', '.demo', '.license', '.credits', '.quit'/'.q'       |
   ------------------------------------------------------------------

couet commented 3 years ago

I am not completely sure how this "additional index file" can be used. Should it be made available for download from this page ? https://root.cern/reference/ . Or it is enough to make it available next ro the normal index.html ?

ferdymercury commented 3 years ago

In my opinion, it does not matter too much where it is stored, as you have to go to QtCreator and add it to the "help database" by hand using a file dialog. Maybe including it in the binaries inside a documentation folder would be enough, so that one does not need to reupdate the webpage too often?

In any case, I think it would be great if it is automatically generated when building from source, always in the same position, so that one does not need to update the qtcreator database path pointer.

couet commented 3 years ago

So what you need is GENERATE_QHP enabled in Doxyfile while generating the reference guide , right ? Have you tried it ? does the result doxygen produces is what you need ?

ferdymercury commented 3 years ago

Thanks. Yes, that's what would be needed. I didn't try it with ROOT but it works well with all my other Cmake projects. It generates a .qch file that one can then import as Help Guide within QtCreator See https://www.creatis.insa-lyon.fr/~grenier/?p=273

couet commented 3 years ago

Sorry for my ignorance but this .qch file contains what ? I guess it cannot be the whole reference guide ? that would be a huge file ... Is it references to the guide on the web ? in that case I do not understand your statement in your first post when you said: which is also great if you don't have internet connection

ferdymercury commented 3 years ago

See also https://reviews.llvm.org/D2967

ferdymercury commented 3 years ago

It is a compressed qt help file, stored locally. Yes, it is potentially a very large file. (Could be enabled via cmake flag). It contains all the html info of the guide.

couet commented 3 years ago

We do not use cmake to build the reference guide. It is a separated Makefile. Yes that file might be huge. How are managed the pictures ? they are somehow included in it ? I will try what is suggested in your insa-lyon link

couet commented 3 years ago

I see the command qhelpgenerator needs to be installed. I do not have Qt (and do not want it) on my machine. Can qhelpgenerator be installed without the full Qt (which requires a licence if I remember well) ? here https://www.qt.io/download a free trial is available ... but we cannot rely on free trial.

ferdymercury commented 3 years ago

You don't need a license for Qt if you click on OpenSource. The option is a bit hidden, but you ll get full Qt, not just a free trial. Also, you can install just QtCreator without installing full Qt. Yes, i think qhelpgenerator can be installed independently, check the LLVM link to see related apt packages.

ferdymercury commented 3 years ago

https://www.qt.io/download-open-source and when installing unselect all except for qtcreator. There are also offline installers that don't require login.

ferdymercury commented 3 years ago

Here the file format description https://doc.qt.io/qt-5/qthelp-framework.html I think images are linked, not embedded https://forum.qt.io/topic/73965/linked-png-in-qthelp-file

couet commented 3 years ago

https://www.qt.io/download-open-source and when installing unselect all except for qtcreator. There are also offline installers that don't require login.

I have done the procedure mentioned here. It create a Qt folder in my home directory with QtCreator inside. But qhelpgenerator does not show... seems not really straightforward

couet commented 3 years ago

Ok, the installed went wrong. I re-did it and i see qhelpgenerator now. I will test on a subset of the doc.

couet commented 3 years ago

I have modified the following entries in Doxyfile:

GENERATE_QHP = YES       
QCH_FILE  = ROOT.qch    
QHP_NAMESPACE = ROOT.Project
QHG_LOCATION = /Users/couet/Qt/6.1.2/macos/bin/qhelpgenerator

I regenerated the hist part of the reference guide on my local machine. I was expecting the file ROOT.qch will appear somewhere . But it does not.

ferdymercury commented 3 years ago

Running doxygen might give you a warning message about qhelpgenerator. It could be the path, see https://stackoverflow.com/a/49751322

ferdymercury commented 3 years ago

By the way, once you open qtcreator, you can click on File, open file or project, and double click on the main Cmakelists.txt, or on the one from the Hist subdirectory.

couet commented 3 years ago

It seems doxygen tried to run this qhelpgenerator but failed:

Running qhelpgenerator...
Error in line 15: Opening and ending tag mismatch.
error: failed to run qhelpgenerator on index.qhp
lookup cache used 13266/1048576 hits=365874 misses=14050
finished...

That's weird because very likely this index.php file has been generated by doxygen, and it seems to have a wrong syntax.

ferdymercury commented 3 years ago

Thanks for checking. What's the doxygen version?

Could be https://github.com/doxygen/doxygen/issues/5333 Or https://github.com/doxygen/doxygen/issues/6254

couet commented 3 years ago

I install doxygen from sources . I am using the master version 1.9.2

ferdymercury commented 3 years ago

Thanks. Then it's probably due to https://github.com/doxygen/doxygen/issues/5333 Can you open index.qhp with a text editor and post here the content around line 15? And maybe also in that open doxygen issue.

ferdymercury commented 3 years ago

How can I build the documentation to try myself? I build ROOT from source, but I do not see any doxygen output.

couet commented 3 years ago

we have a dedicated web page for developers explaining exactly that.

ferdymercury commented 3 years ago

Thanks! Is there a page explaining the dependencies? I tried and I get this error: ImportError: No module named nbformat

couet commented 3 years ago

Ah yes nbformat . That's used by converttonotebook.py to convert the tutorials to notebooks. May be I should add that. In principle that should not stop the build of the documentation . You are installing the tutorials ? you are trying to install the full doc ? (it is 5 hours long)

ferdymercury commented 3 years ago

Yes, I guess I was doing that hehe. If I want to build only the 'hist', how can I select it?

ferdymercury commented 3 years ago

I installed in pip the nbconvert, but I still get: Exception: Jupyter command jupyter-nbconvert not found. [EDIT]: ok, doing apt install jupyter-nbconvert should solve it

couet commented 3 years ago

Yes you better do only a part of the manual. The way to do it is explained here. https://root.cern/for_developers/doxygen/#how-to-generate-the-doxygen-documentation

ferdymercury commented 3 years ago

Thanks. I get now this error: jupyter_client.kernelspec.NoSuchKernel: No such kernel named python2

ferdymercury commented 3 years ago

Ok, I am now using just this as input:

./mainpage.md ../../graf2d/gpad/

It builds well, however, I do not see much when opening the html:

ferdymercury commented 3 years ago

Ok, I found the issue why qhelpgenerator is failing, it is due to:

/opt/root_src/documentation/doxygen/mainpage.md:4: warning: found subsubsection command outside of subsection context!
/opt/root_src/documentation/doxygen/mainpage.md:10: warning: found subsubsection command outside of subsection context!
/opt/root_src/documentation/doxygen/mainpage.md:18: warning: found subsubsection command outside of subsection context!

If I change in mainpage.md to:

\mainpage %ROOT Reference Documentation

# Introduction
Welcome to %ROOT!

This is the reference manual of the ROOT software tooklit.
You can find in the [reference documentation page](https://root.cern/reference/) pointers to reference manuals for all %ROOT versions.

## Other types of documentation:

- [ROOT Primer](https://root.cern/primer/).
- [ROOT Introductory Course](https://github.com/root-project/training/tree/master/BasicCourse).
- A rich set of %ROOT [tutorials and code examples](https://root.cern/doc/master/group__Tutorials.html) are offered to developers to exercise specific functionality.
- A general [Manual](https://root.cern/manual/) is provided for a more in depth explanation of concepts and functionality available in the %ROOT system.
- A number of topical [User Guides and Manuals](https://root.cern/topical/) for various components of the system.

## Provide your feedback
If you have suggestions about how to improve this documentation, you can let us know:

- With a [PR](https://github.com/root-project/root)
- On the [ROOT Forum](https://root-forum.cern.ch)
- On [our tracker](https://github.com/root-project/root/issues)

then ROOT.qch is correctly generated.

couet commented 3 years ago

What did you change in mainpage.md ?

ferdymercury commented 3 years ago

What did you change in mainpage.md ?

The ### to # or ##. It complains that there is a subsubsection with no parent 'section'.

couet commented 3 years ago

Ah ok ... I do a PR for that ? BTW I improved the Doxygen doc: https://github.com/root-project/web/pull/639

couet commented 3 years ago

The ### to # or ##. It complains that there is a subsubsection with no parent 'section'.

Ah ok ... but that looks ugly (I tried) the headers are far too big. I guess we never use "#" only in the root ref guide. So fixing the main page might be not enough . The same problem might show with other pages. For instance here: https://github.com/root-project/root/blob/8a63f78a3f910b3bb8b7758a5af06a80d09e567f/hist/hist/src/TH1.cxx#L139

ferdymercury commented 3 years ago

The bug is gonna be solved soon in doxygen: https://github.com/doxygen/doxygen/pull/8754

In the meanwhile, we could compile doxygen using this fork/branch https://github.com/albert-github/doxygen/tree/feature/issue_8753

ferdymercury commented 3 years ago

I tried with building the docu of only the "core" libraries, and the qch file is correctly generated. When added to qtcreator, pressing F1 shows this beautifully:

ferdymercury commented 3 years ago

In the meanwhile, we could compile doxygen using this fork/branch https://github.com/albert-github/doxygen/tree/feature/issue_8753

It's has now been merged into 'master' https://github.com/doxygen/doxygen/commit/e84b043d4a6a11eb70c1baf3aca89644f156b0d8, so no need to use a specific fork or branch, just git update.

ferdymercury commented 2 years ago

@couet the bug in doxygen is now fixed in official release doxygen version 1.9.3. https://github.com/doxygen/doxygen/issues/8753#issuecomment-1003347799

couet commented 2 years ago

@ferdymercury thanks to let me know. See the bottom of https://root.cern/doc/master/ . 1.9.3 is already the doxygen version used to build the ROOT reference guide. So we can close this issue ?

ferdymercury commented 2 years ago

In my opinion, it does not matter too much where it is stored, as you have to go to QtCreator and add it to the "help database" by hand using a file dialog. Maybe including it in the binaries inside a documentation folder would be enough, so that one does not need to reupdate the webpage too often?

In any case, I think it would be great if it is automatically generated when building from source, always in the same position, so that one does not need to update the qtcreator database path pointer.

I would say it would be handier if the QCH is generated directly and is downloadable via the CERN website. (At least for stable releases 6.24, 6.26, ...).

For this to work, GENERATE_QHP should be set to True in the Doxyfiles of the ROOT repo, and then QHG_LOCATION set to a path where qhelpgenerator is installed in the ROOT build-servers (if not in the default path).

couet commented 2 years ago

For this to work, GENERATE_QHP should be set to True in the Doxyfiles of the ROOT repo, and then QHG_LOCATION set to a path where qhelpgenerator is installed in the ROOT build-servers (if not in the default path).

Can you propose a PR for that ?

ferdymercury commented 2 years ago

Done, but I do not know if something else must be done to make the generated QCH file visible in https://root.cern/releases/release-62406/ or https://root.cern/reference/

couet commented 2 years ago

Thanks ! I would say the table on the page https://root.cern/reference/ should have one more column pointing to the qch. So I guess un extra PR is needed to modify https://github.com/root-project/web/blob/main/reference/index.md (may be an explanation saying what these files are and how to use them might be useful).

ferdymercury commented 2 years ago

I created it now. I just added the link for 6.24 as I do not think it worth to retrospectively add all the previous versions I guess... https://github.com/root-project/web/pull/716

couet commented 2 years ago

Thanks . I merged. I guess we can close this issue now ?

ferdymercury commented 2 years ago

Thanks!

Mmm but the Qch file has to be uploaded first manually to https://root.cern/doc/v624/ROOT.qch (right now the link is broken).

Or is there a CI script that we have to modify so that this step happens automatically on every release ?

couet commented 2 years ago

I guess it should be copied automatically as ROOT.tag. The Scrip dubbing the doc are here: https://github.com/root-project/rootspi/tree/master/rdoc The script copying the doc at the right pace is https://github.com/root-project/rootspi/blob/master/rdoc/synchtml.sh . It copies the folder where the doc was generated in the right place to be accessed fro the web site. ROOT.qch should be like ROOT.tag

ferdymercury commented 2 years ago

Ok, I understand. Then we are just missing the merge on https://github.com/root-project/root/pull/9525/files

Also: should we exclude ROOT.tag and ROOT.qch from the tarball in synchtml.sh ?

couet commented 2 years ago

Also: should we exclude ROOT.tag and ROOT.qch from the tarball in synchtml.sh ? Why ? it is something users might want in the tar-ball ?

root-project / root