Document full repos

[rkent]

This is an optional but helpful PR to allow rosdoc2 to be used over a an entire repository with packages (or any other parent directory for that matter) and it will document all of the packages it finds. It also eliminates the need to specify '--package-path', defaulting to the current PWD.

Many significant ROS projects are coherent at the repository level, and are likely developed as a larger unit than just the individual package. If a developer want to view their project documentation, they would likely want to see all of the packages together. This allows that easily.

From a developer point of view, I want to use it to test cross-package linking, so I would need to run rosdoc2 over multiple packages to do that. This makes it doable in a single command.

I've also used this to document the entire set of packages for a rosdistro, starting from the checkout done by rosindex. That's the reason for the error checks and continuing, so a single package failure hours in does not bring the whole run down.

So while this is optional, it makes my life easier and could be useful to others.

[rkent]

Making this a colcon extension adds another level of abstraction, and I am not enthusiastic about this if the point is just to save a little globbing code. Right now I just type "rosdoc2 build" at a directory, and it just works. But then we always fear the unknown :)

The three requests starting with 'Change it to be a function call ...' are all very reasonable.

[cottsay]

Hi @rkent, thanks for the contribution.

I talked with @tfoote about this today and I believe that using colcon's package discovery would be better than catkin's as well, but I believe that inverting the model is the better choice. The top-level colcon executable is intended for building and testing packages, so calling into rosdoc2 is out of scope.

That said, it has always been a goal for colcon as a project to provide each subsystem and extension point for other programs to use, and I think this would be a great place to do that. I've created a demo script which imports the relevant parts of colcon and discovers all packages in the working directory.

https://gist.github.com/cottsay/47a33dde4520e6fc7178af5f4e628d63

There are some rough edges here, and I think we could make some improvements in colcon-core to make this a little easier and support this use case better.

I look forward to your thoughts.

[tfoote]

Yeah, I know it was asking to increase the scope. After talking with @cottsay I think that we've found an actual better middle ground and get many of the selection features and searching from colcon while keeping the complexity for the code here approximately the same. So to that end I think we can change it to a function and track the return codes and use the demo of the discovery that @cottsay prototyped. And users can then use the same package selection options that they're used to using for builds here. And when/if package selection logic changes it only needs to happen in one place.

[space192]

is there any way it could be use to create a single documentation for all the "scaned" packages so that I can get a coherent browsing for the users ?

[rkent]

@space192 I am not sure what you mean by the "scaned" packages. I'll assume though that you mean all packages that are subdirectories of a parent (where the parent is typically a repository containing all of the packages).

This PR is really meant more as a convenience to the developer of a group of related packages in a single repository. You could simply call rosdoc2 build -p . from the repo directory, and it would run rosdoc2 over all of the packages in that repository. You could then view all of the package's documentation by serving the doc_build folder locally (with for example python -m http.server)

But it should not affect what is actually seen on the 'official' packager documentation pages http://docs.ros.org/en/rolling/p/.

There remains a significant issue as I see it of how to present documentation for a number of related packages. This is a very common problem faced by most large groups of related packages. It often makes sense for the bulk of the documentation to exist at the repository level and not in the individual packages. But rosdoc2 as currently designed is defined as a program to provides per-package documentation, not documentation for a group of packages.

The only support ros2 infrastructure currently provides for a group of packages, as as I understand it, is to show the readme of a repository, either if you click on the repository, or if you try to show there a package without its own readme. See for example https://index.ros.org/r/etsi_its_messages/

I've thought some personally of ways to try to extend rosdoc2 to be more useful with a group of packages, but have not proposed anything yet.