Add infrastructure to create API documentation

[lmoureaux]

Description

Based on the reg_utils setup with some modifications:

• Added the napoleon plugin to parse Google style and Numpy style comments (they are more readable in-source)
• make html instead of make doc (using the target of mfSphinx.mk directly)
• Updated author list (from the contributors list on Github)

Generated documentation is sparse and all formatting is lost. It will be necessary to format it in a format sphinx can understand, but this is out of the scope of this PR.

The generated docs can be found here.

Types of changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)

Motivation and Context

94

How Has This Been Tested?

Documentation was generated on lxplus inside a virtualenv. Waiting for the first Travis build to finish.

Checklist:

• [x] My code follows the code style of this project.
• [ ] My change requires a change to the documentation.
• [ ] I have updated the documentation accordingly.
• [x] I have added tests to cover my changes.
• [ ] All new and existing tests passed.

[lmoureaux]

Despite Travis reporting success, building the documentation on Travis failed because napoleon wasn't available. Under investigation.

[lmoureaux]

• Travis build still not fixed; will need to do it in Docker image
• Documentation structure improved
• I'm not sure where buildMapFiles should appear. Currently it's not documented anywhere

[mexanick]

@lmoureaux you don't need to build the docs in docker container unless you're importing some other GEM modules (like some CDLLs). Even in that case you can wrap in mock import the autodoc importing. From the warnings which travis provided it seems you only miss numpy, so installing it under your venv should help

[jsturdy]

also, i suspect a typo in one of the requirements files, as the package root-numpy doesn't exist (it's root_numpy)

[lmoureaux]

you don't need to build the docs in docker container unless you're importing some other GEM modules

If any module has import ROOT or import root_numpy, I'll need to install it on the machine. Since it's already done in the Docker image, it's the easiest way (and additionally Debian packages for ROOT are terribly outdated)

[mexanick]

Then just run smth similar to the command below in .travis.yml instead of your make html. You'll need to populate the command with relevant python packages though

    - docker exec -ti $DOCKER_CONTAINER_ID /bin/bash -ec "cd /home/daqbuild; virtualenv building -p python --no-site-packages; . building/bin/activate; cd ${REPO_NAME}; pip install pip==9.0.3; pip install --upgrade setuptools; pip install sphinx; pip install python/reg_interface/rpm/*.tar.gz; export REG_XML_PATH=$PWD; export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rwreg/lib:/opt/wiscrpcsvc/lib; make doc"

[jsturdy]

For "complicated" commands running inside the docker container, as mentioned previously to @mexanick, it will probably be best to write a script that lives in .ci, as this has the added benefit that it would work in a migration to gitlab-ci Not a prereq for this PR, but a consideration moving forward

[lmoureaux]

Still failing :disappointed:

Can't install root_numpy because it doesn't find root-config in $PATH and $ROOTSYS is not set. Where is ROOT in the Docker image? It's called devtoolsroot so it must be there, right?

[jsturdy]

yes, there are various versions of root installed in /opt/root/<root version>

• for cc7 images see: https://gitlab.cern.ch/cms-gem-daq-project/gemdaq_ci_worker/-/jobs/1344875
• for slc6 images see: https://gitlab.cern.ch/cms-gem-daq-project/gemdaq_ci_worker/-/jobs/1344871

[lmoureaux]

Building the documentation on Docker requires installing the dependencies (gempython) there. I can wget the package similar to setup_gemdaq.sh, but I'd like a more robust solution.

Is it acceptable to leave the Docker build fail (ie as it is now) until proper pip or rpm repos are set up?

[jsturdy]

Is it acceptable to leave the Docker build fail (ie as it is now) until proper pip or rpm repos are set up?

Ideally, the docs will build without requiring any dependencies be installed, but can be mentioned in the docs somewhere that "version X.Y.Z of <package> (for which docs are being built) depends on API version A.B.C of <required package>" (@mexanick had set things up like this such that the doc building performs a basic test of the package, by trying all the imports, which is a useful minimal check of the package). However, at some point even the unit testing will be done in travis/gitlab-ci and this will require their installation, so I don't know whether it will be beneficial to do things the "right" way... requires further thought.

All that to say, your travis isn't "failing" per se in any event (because the error is only in the after_success, so it is an acceptable solution for now (until we get docs publishing done through the CI job at least, because then we will need the docs to build properly :-) )

[lmoureaux]

Ideally, the docs will build without requiring any dependencies be installed

The problem is that Sphinx imports the packages to be documented, and therefore any failing import statement prevents the docs from being built. Since it's perfectly legitimate to have import statements at module scope, I don't think there's much we can do except installing the deps.

it is an acceptable solution for now

Removing the WIP tag to not prevent merge.