Support conda for builds

[mrocklin]

Read the Docs Conda Support

This will add the ability to generate documentation with conda environments on Read the Docs. This is mainly useful for libraries with large C dependencies, including many packages in the Scentific Python ecosystem.

Task List

• [x] Add Project option to use conda
• [x] Abstract the "environment path" in the builders to respect conda & virtualenv
• [x] Ensure cleanup of conda environment in wiping & other env delete scenarios
• [x] Ensure Python 2 & 3 support
• [x] Write tests for proper conda environment setup
• [x] Document conda support
• [ ] Write a topical guide for Choosing a Build Environment
• [ ] Optional: Investigate pruning

  Abilities

You will be able to specify a conda environment.yml file, and Read the Docs will install these dependencies in your build environment.

Considerations

Read the Docs will keep seperate virtualenv & conda directories:

• /user_builds//env for pip
• /user_builds//conda for conda

Users will be able to define a way to install packages for a project:

• None
• Pip
• Conda

Read the Docs will need to change it's build code so that we don't hard-code virtualenv paths. We'll need to vary our environment creation, as well as bin path's for executables, based on the backend environment.

The other main thing is that we'll also need to install Sphinx & other build dependencies into the conda environment. We will continue to use pip for this, and it should be transparent, other than using the pip executable in the conda environment instead of the virtualenv.

It should also be noted that miniconda has a different install process from Python 2 and 3 -- also they recommend installing it from their bash scripts instead of pip. I hope that we will be able to use pip, as it will simplify our installation, and won't require an update to a bash script on version upgrades. We will have to see if we hit issues in testing.

Cleanup

Read the Docs will manage conda environment deletion on the removal of a project or version.

Documentation

We will need to add information about conda support to our documentation. We might want to add a topic guide around installing requirements, along with adding a specific reference for how to use & enable conda support.

Sponsorship

This work is being funded by Clinical Graphics -- many thanks for their support of Open Source.

[jakirkham]

So, I tried my hand at a simpler PR ( https://github.com/rtfd/readthedocs.org/pull/1919 ). From what I have gleaned from my brief look at the code here (not an expert), this looks like a workable solution.

It probably could use a test or two, but I am a bit unfamiliar with the test suite here and will need guidance for that. Though, I imagine doing something like asserting the variables are there and checking the output of conda info would be a good start.

[jakirkham]

Turns out this PR ( https://github.com/rtfd/readthedocs.org/pull/1924 ) ends up being a much better way to go about it.

[jakirkham]

So, I think the active environment issue is fixed thanks to @ericholscher and @agjohnson, if people want to try.

[mrocklin]

Verified that this works well for me. Thanks all!

[ericholscher]

Great :)

If there are other issues, please create new tickets for them, as this one is quite old and the main issues seem to be resolved.

Woo!

[astrojuanlu]

I just built the docs for poliastro in rtd and it works like a charm :smile: Thanks to @mrocklin and Clinical Graphics for this!

[astrojuanlu]

Just a quick notice: right now RTD uses docutils 0.11, which is not yet available for Python 3.5 for linux.

http://readthedocs.org/projects/poliastro/builds/3655474/

Hint: the following packages conflict with each other:
  - docutils ==0.11
  - python 3.5*

Packages: https://anaconda.org/anaconda/docutils/files?version=0.11

[astrojuanlu]

(Sorry, my above comment was already reported as #1901)

[jakirkham]

@Juanlu001, see ( https://github.com/rtfd/readthedocs.org/pull/1933 ).

[ericholscher]

1947 is the fix for that issue. I will get it merged & deployed today.

[tritemio]

I tried the using conda environment with fretbursts to see if I can get rid of the mocking layer.

However, building the docs fails on RTD:

http://readthedocs.org/projects/fretbursts/builds/3813468/

unless I reintroduce the mocking for scipy:

http://readthedocs.org/projects/fretbursts/builds/3813474/

[agjohnson]

@tritemio it would be best if you opened your issue as a bug, we won't be able to track it here. This was a feature addition that has been in production for a few months now.

Closing this issue. Please address bugs with additional tickets.

[tritemio]

@agjohnson, it makes sense. I opened a new issue https://github.com/rtfd/readthedocs.org/issues/2074