Unable to build epydoc docs

[mikepurvis]

It seems that unlike with Sphinx and Doxygen, successfully generating epydoc depends on the package being built and on the PYTHONPATH. And the epydoc docs clearly build successfully in ros_buildfarm (eg), so I'm struggling to understand where this goes wrong:

rosinstall_generator rosdoc_lite rospy --rosdistro indigo --deps --tar > test.rosinstall
rosinstall src test.rosinstall -j8
catkin build rospy rosdoc_lite
source devel/setup.bash
rosdoc_lite src/ros_comm/rospy

Result:

Documenting a catkin package
Documenting rospy located here: /home/mpurvis/rosdoc_ws/src/ros_comm/rospy
{'epydoc': {'builder': 'epydoc', 'config': 'epydoc.config'}}
Catkin package, no need to generate python paths
generate_epydoc() paths:
generate_epydoc() additional_packages:
generate_epydoc() PYTHONPATH: /home/mpurvis/rosdoc_ws/devel/lib/python2.7/dist-packages
epydoc-building rospy [epydoc --html rospy -o doc/html/. --config /home/mpurvis/rosdoc_ws/src/ros_comm/rospy/epydoc.config]

[ ... snip, output has identical warnings and errors to console log from ros_buildfarm ... ]

UNEXPECTED ERROR:
[Errno 2] No such file or directory: 'doc/html/.'

Use --debug to see trace information.

Unable to generate epydoc for [rospy]. The return code is 3
Traceback (most recent call last):
  File "<string>", line 187, in generate_docs
  File "/home/mpurvis/rosdoc_ws/src/rosdoc_lite/src/rosdoc_lite/epyenator.py", line 80, in generate_epydoc
    output = subprocess.check_output(command, stderr=subprocess.STDOUT, env=env, cwd=path)
  File "/usr/lib/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['epydoc', '--html', 'rospy', '-o', 'doc/html/.', '--config', '/home/mpurvis/rosdoc_ws/src/ros_comm/rospy/epydoc.config']' returned non-zero exit status 3
plugin [epydoc] failed
[]
[]
[]
copying /home/mpurvis/rosdoc_ws/src/rosdoc_lite/src/rosdoc_lite/templates/msg-styles.css to doc/html/msg-styles.css
Done documenting rospy you can find your documentation here: /home/mpurvis/rosdoc_ws/doc

The weirdest thing of all, though, is that if I manually execute the epydoc command outside of rosdoc_lite's mangling of the PYTHONPATH, it totally succeeds:

epydoc --html rospy -o doc/html/. --config /home/mpurvis/rosdoc_ws/src/ros_comm/rospy/epydoc.config
ls doc/html/  # yay, docs

I tried not building rospy, and also building it in a separate workspace with catkin_make rather than catkin_tools, and using the install rather than devel space— no dice. What am I doing wrong here?

[jack-oquin]

Mike,

I was out of town Monday and Tuesday, sorry for the delayed response.

I don't have much personal experience with epydoc, mostly being a sphinx and doxygen user. I probably have less idea what's going on than you do. So, I'll ask some simple questions in hopes they might stimulate us to come up with answers or at least further experiments:

• It looks like you are building rosdoc_lite and rospy from source on Indigo with catkin build.
• Then, you are using the built version to document rospy.
• And, that's failing, right?
• You tried building them other ways and in separate workspaces, which also fails.

I recently noticed that rosdoc_lite is unable to document itself (#65), due to a missing doc dependency on python-catkin-sphinx (ros-infrastructure/catkin-sphinx#6). That's not relevant to epydoc, but maybe there is a similar undeclared dependency for that path?

Basically, the build farm uses rosdoc_lite in strange and specially-crafted ways. Building it locally is unlikely to yield exactly the same results. :frowning:

[mikepurvis]

Okay, have poked and prodded a bit more, looks like a cwd issue. The following works as expected:

rosinstall_generator rosdoc_lite rospy --rosdistro indigo --deps --tar > test.rosinstall
rosinstall src test.rosinstall -j8
catkin build rospy rosdoc_lite
source devel/setup.bash

pushd src/ros_comm/rospy
rosdoc_lite . -o ../../../doc
popd

So, this can be worked around, but IMO it is a bug in rosdoc_lite.

[jack-oquin]

Yeah, rosdoc_lite expects the current directory to be at the top of the package. That's annoying, and not clearly documented.

It does not seem appropriate to change that behavior now, but I suppose we could add something like a --chdir option, so you could do this:

rosdoc_lite --chdir=src/ros_comm/rospy

[mikepurvis]

$ rosdoc_lite -h
Usage: rosdoc_lite [options] [package_path]

Options:
  -h, --help            show this help message and exit
  -q, --quiet           Suppress doxygen errors.
  -o OUTPUT_DIRECTORY   The directory to write documentation to.
  -t TAGFILE, --tagfile=TAGFILE
                        Path to tag configuration file for Doxygen cross
                        referencing support. Ex: /home/user/tagfiles_list.yaml
  -g GENERATE_TAGFILE, --generate_tagfile=GENERATE_TAGFILE
                        If specified, will generate a doxygen tagfile in this
                        location. Ex: /home/user/tags/package.tag

The package_path arg seems to strongly imply that you can specify the path to the package you wish to generate documentation for, but it turns out that the only valid value for this field is .? That seems absurd, particularly when generating in another folder works fine for Sphinx and Doxygen— only epydoc has this issue.

[jack-oquin]

I agree that the documentation implies different behavior. But, I hesitate to change how it works after all these years.

What do you suggest? Just chdir to the optional package_path?

[mikepurvis]

That seems to be how the Sphinx plugin handles things:

https://github.com/ros-infrastructure/rosdoc_lite/blob/d6f63f94c62b3b73f5d3350a7773ffb7478b353a/src/rosdoc_lite/sphinxenator.py#L55-56

[jack-oquin]

Ah! That's why it only fails with epydoc.

In that case, a similar patch to the epydoc plugin seems appropriate and more-consistent.