shell-literate Sphinx round2

[mr-c]

I've been thinking about the approach I started in #30 a lot the last few days and several things bothered me.

The first thing that bothered me was the frequent use of .. shell:: start and .. shell:: stop tags. Not only are they noisy and easily forgotten about several screens down but they do create a warning message when you run Sphinx.

The other thing that bothered me was that we were going to have to eventually reimplement all of Sphinx's ReST parsing abilities & quirks. Clearly extending Sphinx is the way to go but as their documentation and code reveal their system isn't as easily extensible as they would have you think. Also: if we had to make a custom directive then our ReST files would still generate noisy warnings when others used our docs without our special setup.

To summarize the new requirements: 1) Be backwards compatible without generating warning messages 2) Don't extend Sphinx itself 3) Don't reimplement ReST parsing

As a bonus I implemented my totally new solution in Python.

How did I do it? I'll let the tool's docstring do the talking:

Usage:
    1) Notate shell code that is to be both displayed and extracted with the
    "code-block" directive which has the highlight language specified as
    "bash".
    Example:
.. code-block:: bash

   cd mydirectory

   2) Notate shell code that is to be extracted only and not displayed with a
   ReST comment that begins with "code-block-hidden"
   Example:
.. code-block-hidden

   run mySecretSetupProgram.sh

   3) Use the Sphinx XMLBuilder to serialize your documentation to disk.
   Example: sphinx-build -b xml . build

   4) Use extract-shell.py to parse the XML files generated by the Sphinx
   XMLBuilder. Redirect the output as desired.
   Example: (ls build/mrnaseq/*.xml | xargs -n 1 ./extract_shell.py) > out.sh

I've updated mrnaseq/1-quality.txt as an example. You can see a rendered version of it at http://athyra.ged.msu.edu/~mcrusoe/khmer-protocols/mrnaseq/1-quality.html and the resulting extraction at http://athyra.ged.msu.edu/~mcrusoe/khmer-protocols/mrnaseq/1-quality.sh

I think the new method is easy to write and remember.

@LSheneman et al., what do you think?

[camillescott]

Yay for python! Are those hideous globals really the only way to do it though?

[LSheneman]

Closing this in favor of a complete and tested implementation, as well documented in #30