Enable sphinxcontrib-programoutput to run project code

[douglatornell]

The sphinxcontrib-programoutput extension provides a DRY way to include project's CLI help text in docs (among other things). With the extension installed, including:

.. program-output:: foo --help

should result in the help text for foo being inserted in the docs. That works locally for me, but fails on RTD because the bin/ directory of the project's build venv is not included in PATH. I verified that with:

.. program-output:: echo $PATH
   :shell:

The log from that test is at https://readthedocs.org/builds/salishsea-meopar-tools/1352996/

[ericholscher]

Not sure if this is something that we can fix. It sounds like a custom setup for your own project.

[zaro0508]

I have the same issue. I believe there is a compatibility bug with sphinx 1.2.2 and sphinxcontrib-programoutput. When i run with sphinx 1.2.2 i get the following: "ERROR: Command u'jenkins-jobs --help' failed: [Errno 2] No such file or directory". When I run with sphinx 1.1.3 it works just fine. Both test runs are on my local machine. It seems like rtfd.com always runs with sphinx ver 1.2.2 even when you tell it to use a requirements file that specifies an older version of sphinx.

My Project: https://readthedocs.org/builds/jenkins-job-builder/ The overall html reports passes, but the sphinxcontrib-programoutput portion reports: /var/build/user_builds/jenkins-job-builder/checkouts/latest/doc/source/installation.rst:197: ERROR:

Is there a way to make rtfd builds use an older version of sphinx? I would expect the build to use the sphinx version that is specified in the requirements file, no?

google groups discussion on the same issue: https://groups.google.com/d/msg/read-the-docs/qVCx-lEwqME/CaWFv_YgSnMJ

[jayvdb]

I was able to force sphinx to be any version, by changing it requirements.txt. I enable the 'use virtual environment' admin setting.

https://github.com/jayvdb/pywikibot-core/commit/0c35e0dd052bd54a417227ed5cd3ea8514534cf8

I also needed to remove the minimum version in conf.py , otherwise it would fail - I've not investigated that problem yet.

[zaro0508]

My project is setup the same, virtual environment is set and the min version in conf.py disabled but it keeps on building with sphinx 1.2.2 on rtfd. I do specify two requirements files in my project (i.e. '-rrequirements.txt -rtest-requirements.txt') which seems to work because it builds the docs, it just has errors with sphinxcontrib-programoutput.

[hashar]

The Debian package for Jenkins Job Builder has a patch which change the invoked command:

-.. program-output:: jenkins-jobs --help
+.. program-output:: cd ../.. && ./tools/jenkins-jobs.py --help
+   :shell:

http://bazaar.launchpad.net/~ubuntu-branches/debian/sid/jenkins-job-builder/sid/revision/6#debian/patches/workaround-programoutput.patch

That solves the issue of generating doc out of tox or when jenkins-job hasn't been installed with pip install -e.

[gregmuellegger]

@douglatornell @zaro0508 We are using Sphinx 1.3.1 now on the server. Is this resolved now for you?

[hashar]

I triggered a build https://readthedocs.org/builds/jenkins-job-builder/3131982/

It fails because sphinx is apparently not run in the venv that has all the requirements.

File "conf.py", line 54, in <module>   # That is Sphinx config file
    from jenkins_jobs.version import version_info as jenkins_jobs_version
   from pbr.version import VersionInfo
   jenkins_jobs/version.py", line 18, in <module>
   ImportError: No module named pbr.version

[gregmuellegger]

I think the reason for this is that you specified -rrequirements.txt -rtest-requirements.txt as your requirements file. You can find the following line in the build logs: Could not open requirements file: [Errno 2] No such file or directory: '-rrequirements.txt'

Please only provide the filename here. No pip options are allowed there. So if you need multiple files with requirements installed, please make one doc_requirements.txt or similiar that references the two other files.

[zaro0508]

I've stated my use case for passing in multiple requirements files in issue https://github.com/rtfd/readthedocs.org/issues/1112 so I'm not sure why @ericholscher shot down the idea. Anyways I guess the root of the problem is that passing multiple requirements files will not be supported

[gregmuellegger]

@zaro0508 The pattern I see in projects evolving is that you have one requirements file for every usecase and then use composition to bring in the exact packages you need.

requirements/_base.txt

pip==7.0.11
other_base_dep==1.0.0

requirements/development.txt

-r _base.txt
ipython==1.0.0

requirements/deploy.txt

-r _base.txt
gunicorn==18.0.0

requirements/docs.txt

-r _base.txt
-r tests.txt

requirements/tests.txt

-r _base.txt
mock

Ofcourse it's a matter of taste but we decided to only support one requirements file since that reduces the complexity for readthedocs quite a bit. With multiple files we need to make sure they are applied in the right order etc.

[hashar]

We went with docs-requirements.txt . I have updated the readthedocs.org project to use it and sphinx-programoutput got installed properly.

[ericholscher]

Great :fireworks: