search

cmake-basis / legacy

Legacy CMake BASIS project for versions 3.2 and older. For newer versions, go to
https://github.com/cmake-basis/BASIS
Other
13 stars 11 forks source link

Improve usability of BASIS Manual and API Docs #254

Closed ahundt closed 10 years ago

ahundt commented 10 years ago

There should be a clear flow from the front page explaining what basis does to getting started pages and instructions to create your first project.

There should also be clear indication of where you are and where you should go that makes it easy to find where you are going. The manual should continue to be front and center, because many users won't need the API documentation and can just use the quick start guide. Nonetheless, the ability to maintain a mental model of your location on a single BASIS website and browse around within it would dramatically improve the experience of a new user.

I was thinking integration of Sphinx Documentation and API documentation would be a good way to reach this goal and make the site much easier to navigate.

It looks like the breathe library (documentation) could help support this goal by integrating the Sphinx Manual directly into doxygen.

Related discussion and issues can be found at:

Open sub-issues:

ahundt commented 10 years ago

Copied from @schuhschuh in Issue #234: A software often consists of two separate but linked documentations. One is the higher-level user manual and the other a developer manual or API reference. The latter is what we get from Doxygen, the first is what is written in reStructuredText and processed by Sphinx. The BASIS documentation can be improved in many ways and the outline be more intuitive, but I prefer not to have a low-level API reference mingled with a higher level description of how to use the software. In case of a library or package like BASIS, the manual will of course also need some references to the API. While Doxygen can now also process Markdown and thus be used to generate manual pages without code, it still contains the typical document structure of "Main page", "Related pages", "Modules", "Namespaces", "Classes", and "Files". This top-level document structure makes no sense for a user manual. Initially, before I added support for Sphinx, all documentation was indeed written with Doxygen markup, but I think we can achieve a much better overall documentation using tools like Sphinx which are not so specific to the generation of an API reference.

The main documentation of BASIS should be the one that is generated by Sphinx, this should be your main portal. It can be extended to contain some overview of the most often used CMake BASIS commands and explain these in more detail. This overview can then for further details which are more of interest for advanced users and BASIS developers link to the respective API reference pages.

Note that "BASIS" is also a standard. Hence, I tried to have a manual that consists in one part of the description of the "standard" which is just a collection of documents and policies but independent of any particular software, and a manual of how to use the BASIS modules (CMake, utilities,tools,...) to help implementing this standard in a software project. I see a lot of possibility to extend and improve upon the current documentation. It is in many parts yet incomplete.

The generation of the BASIS documentation is described at http://www.rad.upenn.edu/sbia/software/basis/howto/install.html#build

I completely agree that the clean and straightforward parts of the manual shouldn't dump you into the API Documentation. I know that it is possible to reorganize the pages in doxygen as they describe on the doxygen customization page, where they also mention breathe. I think we could use those tools to solve the real and legitimate problems you bring up regarding the doxygen "Main page", "Related pages", "Modules", "Namespaces", "Classes", and "Files" tabs. Doxygen obviously recognizes the problem because their own website doesn't even use the default layout!

schuhschuh commented 10 years ago

Right, I had a look at Doxygen's pages afterwards again. However, I also have big difficulties navigating in their manual. In my opinion this is yet the result of Doxygen being in first place designed for API documentation generation.

I agree that breathe will be very helpful. Also doxylink can be used in cases where you want to simply refer the reader to the separate API reference. Therefore, both Sphinx extensions are included and supported by BASIS already.

We don't need the customization of the Doxygen pages because the main documentation generator will be Sphinx, not Doxygen. Doxygen is only used for the API reference and for parsing the source code. The breathe extension actually helps you to extract this information from Doxygen's XML output so you can embed it into the reStructuredText documentation. reStructuredText also is a more powerful markup language than Markdown (Extra). While I like the latter for its simplicity, reStructuredText just gives you more markup possibilities and can further be extended.

schuhschuh commented 10 years ago

BTW Once I got to know that pandoc can be used to generate HTML-based slide shows using one of several such CSS/JavaScript libraries, I always wanted to convert the PowerPoint presentations such that the quick start guides and tutorial slides can be embedded much better into the HTML documentation.

ahundt commented 10 years ago

Ah I see Breathe is Doxygen -> Sphinx, great! Either way are you ok with the idea of integrating everything into a single portal that is properly laid out to maximize usability, rather than the two disconnected websites?

It may be worth considering the use of markdown front and center for all of this stuff by default if it is possible. I know that means there will be limitations, but one thing BASIS can help people achieve is easily making a fully integrated, documented, and distributed package without needing a fully integrated understanding of a million APIs and tools. Markdown is very simple, straightforward, easy to understand, and easy to write. That means the barrier of entry for writing decent documentation will be lower, which I think is more important than well integrated documentation. Pandoc may be able to help make Sphinx + Markdown possible.

BTW Once I got to know that pandoc can be used to generate HTML-based slide shows using one of several such CSS/JavaScript libraries, I always wanted to convert the PowerPoint presentations such that the quick start guides and tutorial slides can be embedded much better into the HTML documentation.

For the slide show, we could alternatively put the contents into markdown (or reStructured Text) and use utilities to turn it back into a presentation, that way the Quick Start and other materials which are good starter guides could be directly available in the manual while still being easy to use for a presentation. It looks like the Sphinx extension Heiroglyph may provide a way to do that.

schuhschuh commented 10 years ago

Of course I agree with a single portal, but this has always been the goal. It just isn't complete yet. There are not really two web sites. You notice that the API documentation URL is part of the main project site hosted at SBIA and that the layout of both was made as similar as possible. What is yet just missing are non-API reference style documentation for users of the BASIS CMake commands and utility functions. Plus of course a better structure of the manual.

With Markdown you will not be able to use breathe to embed the API documentation. This is one of the reasons why ReST is more suited. You can easily include highlighted code fragments in the documentation. For Markdown there are probably some extensions as used by GitHub, but the question is where do you get the non-standard Markdown processor from ? This is one of the issues with Markdown. That there exist different variations of it and everyone creates there own variant because the basic Markdown constructs are just not sufficient. Learning ReST is not a big deal. The basic markup syntax is similar to Markdown. It only becomes a bit more complicated when you use the various extra directives which are mainly needed to do what Markdown is not able to do.

With neither Markdown extension you could create a page like http://sphinx-doc.org/tutorial.html without embedding raw HTML code. And then you could also not create a PDF with the same look... check out the corresponding ReST source code of this page.

schuhschuh commented 10 years ago

Here some good examples on how Sphinx + ReST can be used and documentation be separated into different parts depending on audience, namely, a user guide, API reference, and a developer guide.

http://docs.python.org/3/ http://docs.scipy.org/doc/numpy/index.html

ahundt commented 10 years ago

That looks pretty nice! There is some disconnect such as with the main product page, but generally most pages have breadcrumbs so you can tell where you came from, where you are, and where you may want to go next. I'd be pretty happy if that is how things ended up working.

schuhschuh commented 10 years ago

These breadcrumbs are automatically added by Sphinx. We just disabled them for the SBIA projects such that the look of the web pages matches the one of their existing homepage. With BASIS 3, we can now get rid of this design and come up with a much more user friendly layout.

ahundt commented 10 years ago

BASIS site uses default Sphinx theme

I've restructured the documentation to use the default Sphinx theme 409ca3d, and have a reasonably clean organization 8bd3568. It could still use some improvement but I like the direction it is going.

Updated breathe

I've also updated breathe, since it turns out the library was already integrated. 693bebf

API Integration

It still needs some work but I got the API to also show up in the sphinx site. In particular, it needs better organization by module like in doxygen, and I'm not sure if there is an automatic way to do that.

ahundt commented 10 years ago

This new version 99bb16f looks a tiny bit sloppy and the embedded html sizing could be done better, but it allows the user to browse the API docs inline, which makes life easier. If breathe eventually gets the functionality needed to automatically organize the documentation better then we should use that instead of embedded html.

I could also use some assistance with improving how the embedded html works, I haven't been able to figure out how to automatically set the object size based on the content size, and how to remove the side margins. The changes for the site may have also had some unintended side effects for the PDF documentation, and that will need to be resolved as well.

schuhschuh commented 10 years ago

Nice work. I had a look at the new documentation. First of all, I would prefer a different Sphinx theme, however. Not so happy about the default. Not a big deal, though. The text is definitely better to read with this layout then with the previous SBIA colors and fonts.

Under the rubric How-to Guides, I noticed the "BASIS Installation Options". I think this section should not exist. These options are supposed to be under "CMake Options".

The embedding of the API documentation needs indeed still quite some work. At the moment it doesn't really blend in naturally at all. It will be important to use common colors, fonts, and layouts for both Doxygen and Sphinx. Instead of using an iframe, the Doxygen header and footer template HTML files should be adjusted.

There are some Sphinx errors. E.g., the directive :apidoc: is no longer recognized and would have to be replaced.

The documentation of the CMake options appears twice in the PDF (make manual). Not sure if that was always the case.

ahundt commented 10 years ago

On Thu, Dec 12, 2013 at 11:30 PM, Andreas Schuh notifications@github.comwrote:

Nice work. I had a look at the new documentation. First of all, I would prefer a different Sphinx theme, however. Not so happy about the default. Not a big deal, though. The text is definitely better to read with this layout then with the previous SBIA colors and fonts.

Yeah the colors could be better at the very least. If you want to pick something that you think is better feel free to make the change.

Under the rubric How-to Guides, I noticed the "BASIS Installation Options". I think this section should not exist. These options are supposed to be under "CMake Options".

This should be an easy fix that I can make.

The embedding of the API documentation needs indeed still quite some work. At the moment it doesn't really blend in naturally at all. It will be important to use common colors, fonts, and layouts for both Doxygen and Sphinx.

That would be a nice improvement, do you want to set that up with some layout and theme you would prefer?

Instead of using an iframe, the Doxygen header and footer template HTML files should be adjusted.

What do you propose instead of an iframe? Breathe isn't ready and ended up being harder to use than the Doxygen docs, and what is there now is at least more functional than the original two separate pages.

There are some Sphinx errors. E.g., the directive :apidoc: is no longer recognized and would have to be replaced.

Is there a way to make the :apidoc: directive work again? I wasn't sure what happened and this is something I meant to ask about.

The documentation of the CMake options appears twice in the PDF (make manual). Not sure if that was always the case.

Hopefully this should be fairly easy to fix.

ahundt commented 10 years ago

This theme, "readable" is much more readable than the default one as the name implies.

ahundt commented 10 years ago

I checked your comment about BASIS Installation options, and I don't think those options apply to general installation of other packages, that is why I labeled them as specific to BASIS.

schuhschuh commented 10 years ago

The new Doxygen layout/font looks pretty nice! Also the manual pages are going in the right direction, I would say. Well done.

On another note, I noticed that Doxygen includes the path to the scripts from which the documentation of the "BASIS Tools" (see Related Pages, e.g., basisproject.py) is extracted from. These are after the build in "/CMakeFiles/.dir/build" and this path should actually not be exposed/show up in the documentation. Acceptable would be "bin/" because that's where these tools/scripts are located after installation.

schuhschuh commented 10 years ago

Comments on current status, most minor which I may just change myself next:

schuhschuh commented 10 years ago

Replaced logo by commit ae460b69a6999275c6cb0bee0fe0674984c8b41e (forgot issue reference in commit message).

schuhschuh commented 10 years ago

Regarding :apidoc: directive. The Sphinx configuration file generated by BASIS (basis_add_sphinx_doc, <build>/doc/conf.py) contains the following:

doxylink = {}
breathe_projects = {'apidoc': '/Users/as12312/Software/SBIA/BASIS/build-clang-5.0-release/doc/apidoc/xml'}

but the directive should actually be one used by the doxylink Sphinx extension, the configuration of which is empty. This might be the reason for the error, that the doxylink extension is not configured correctly.

schuhschuh commented 10 years ago

It appears that the DOXYLINK and DOXYLINK_URL options of the basis_add_doc command in doc/CMakeLists.txt were missing, causing the :apidoc: directive error.

The URL will have to be set to something like http://opensource.andreasschuh.com/cmake-basis/apidoc/latest, given that we will host the API reference at this URL. How will this work with your iframe's ?

schuhschuh commented 10 years ago

The links to the API reference still need to be fixed.

schuhschuh commented 10 years ago

I have uploaded the updated documentation to http://opensource.andreasschuh.com/cmake-basis/.

schuhschuh commented 10 years ago

Should we wrap up this issue and have it merged into the develop branch soon ? I think any other particular additions/improvements of the documentation itself can be now tracked separately again instead of by one massive issue.

The smaller open sub-issues/-tasks yet to do for this one are given as task list of the issue description above.

ahundt commented 10 years ago

Yes. It seems the background of the website is white now? I liked the previous off-white color better, it was not as hard on the eyes.

schuhschuh commented 10 years ago

Nothing I should have changed.

ahundt commented 10 years ago

strange, perhaps some css files were missed or something? Are you seeing a white color too? If not perhaps it is a browser specific issue.

schuhschuh commented 10 years ago

If you refer to the overall page background, the readable-wide Sphinx theme does specify it to be all white in https://github.com/schuhschuh/cmake-basis/blob/feature/docUsability/src/sphinx/themes/readable-wide/static/readable.css_t#L39:

div.body {
    background-color: #ffffff;
    color: #3e4349;
    padding: 0 30px 30px 30px;
}
ahundt commented 10 years ago

I did some testing and ran into a few things we should look at improving before calling this done:

The documentation feels a bit deep, like it takes at least two clicks to get anywhere. Perhaps we can make it more shallow, particularly for the standards, and instead of putting the table of contents alone in each top level tab actually include the content there after the TOC?

Sidebar

Quick Start

Standards

What about the following organization

I'm sort of thinking that the way we had it before with a single link to all standards may have been easier to navigate, I think it is important to see it all at once so you don't get lost searching for the correct section. Perhaps with a little reorganization the new way you set things up would end up better.

ahundt commented 10 years ago

Yes, the more I think about it the more I'm coming to the conclusion that we should do another small amount of reorganization. What if we simplify the "Build System Standard" and "Implementation Standard" dichotomy into a simple set of "standards" so that everything could be seen at once and navigated easily? It may make everything much easier to find.

With the current layout I get a feeling of confusion and that I am in the wrong place, or that I'm jumping to the wrong menu. Some other things are also not in a place that you would think, such as version control standards being under the build system standard, they are separate parts. What do you think?

schuhschuh commented 10 years ago

"Programming Guidelines" is very commonly used and known. The previous "Style Guides" was probably incorrect as a "guide" is not the same as a "guideline", is it? Other terms which are well known are "Coding conventions" or "Programming Style Guidelines". The latter is, however, just unnecessarily long for the side bar.

See also: http://www.haskell.org/haskellwiki/Programming_guidelines http://www.codecogs.com/pages/standards/programming.htm http://caml.inria.fr/resources/doc/guides/guidelines.en.html https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/CodingGuidelines/CodingGuidelines.html http://geosoft.no/development/javastyle.html

Both "Build system Standard" and "(Software) Implementation Standard" are terms of the project name "Build system And Software Implementation Standard". If it is not clear what these refer to, then maybe a brief explanation on the overview page would be advisable instead of renaming the sections. These terms are already somewhat the core of the project.

schuhschuh commented 10 years ago

Looking at the documents under "Build system Standard", you will realize that all of it is related to general project set up and organization which affects the, resp., results in a particular, configuration of the build system. The "Implementation Standard" clearly only covers certain implementation guidelines for your project.

schuhschuh commented 10 years ago

The version control section of the FHS document can of course be moved to a How-to regarding source control instead, which should be close to the Branch and Release How-to. The latter might anyway need to be reworked at some point.

Other than that, the structure of the source, build, and installation tree are all related to how the build is set up.

ahundt commented 10 years ago

"Style Guides"

Google calls it a style guide: https://code.google.com/p/google-styleguide/

I wouldn't worry about the difference, guide can be short for guideline. I don't mind either way. I only think the word style is important and useful to convey the purpose accurately.

Both "Build system Standard" and "(Software) Implementation Standard" are terms of the project name "Build system And Software Implementation Standard". If it is not clear what these refer to, then maybe a brief explanation on the overview page would be advisable instead of renaming the sections. These terms are already somewhat the core of the project.

I agree that is the title of the whole thing and the project contains both. However, I don't think that defines or requires a boundary between the two. So how strongly on a scale from 1 to 10 do you feel about this split?

What about the following layout?

What problems do you see with a single "standards" link listing all standards and guidelines under it?

Perhaps eliminating the "Projects" group and moving all the sub items up a level be even better.

The version control section of the FHS document can of course be moved to the related How-to instead.

I think this makes more sense as part of the standard.

ahundt commented 10 years ago

One other item we missed. The Index is not very useful at all since it consists almost entirely of cmake options. Do you know how it works or how to improve it?

schuhschuh commented 10 years ago

Do you agree with the latest changes regarding documentation of the standards and other style guidelines?

I noticed the index page earlier, but have no clue how to fix this and considered it as not important enough for the moment. Do you want to have a look? Create a separate issue for later after merging this one with develop after it is completed?

ahundt commented 10 years ago

Yeah, index can be a 3.1 issue. It isn't important.

ahundt commented 10 years ago

Are the changes on the website or just in the files? I won't be able to generate files today to verify. The changes in code look good but I want to have a look at the final output too.

schuhschuh commented 10 years ago

Regarding general index page generated by Sphinx, it seems we will have to add appropriate markup directives to the documentation. See the index directive and object description directives. The latter do not apply to the BASIS documentation as it is at the moment, though. See also the documentation regarding the genindex document.

schuhschuh commented 10 years ago

Are the changes on the website or just in the files?

The BASIS homepage now reflects the status of commit 3eee6d7 3286934.

ahundt commented 10 years ago

I looked at your changes, and particularly the quick start changes are excellent! Very easy to follow. I'm still not able to see the new TOC because it is blank for me. I created Issue #271 to address the problem.

ahundt commented 10 years ago

I really like the new layout, it looks great! For a slight tweak in the spacing, what do you think of the following TOC layout?

.. title:: Overview

.. include:: intro.rst
.. include:: features.rst
.. include:: history.rst
.. include:: people.rst

.. toctree::
    :includehidden:
    :maxdepth: 3

    Overview <self>
    quickstart
    howto

.. toctree::
    :includehidden:
    :maxdepth: 3

    standard
    guideline
    apidoc

.. toctree::
    :includehidden:
    :maxdepth: 3

    changelog
    download
    install
    help

.. toctree::
    :includehidden:
    :maxdepth: 1

    Full Table of Contents <contents>
schuhschuh commented 10 years ago

Ok, I think the Getting Help menu fits well with the "for newbies" section. Also remember to update the order in contents.rst such that the "previous/next" button s work as expected. On 6 Jan 2014 17:51, "Andrew Hundt" notifications@github.com wrote:

I really like the new layout, it looks great! For a slight tweak in the spacing, what do you think of the following TOC layout?

.. title:: Overview

.. include:: intro.rst .. include:: features.rst .. include:: history.rst .. include:: people.rst

.. toctree:: :includehidden: :maxdepth: 3

Overview <self>
quickstart
howto

.. toctree:: :includehidden: :maxdepth: 3

standard
guideline
apidoc

.. toctree:: :includehidden: :maxdepth: 3

changelog
download
install
help

.. toctree:: :includehidden: :maxdepth: 1

Full Table of Contents <contents>

— Reply to this email directly or view it on GitHubhttps://github.com/schuhschuh/cmake-basis/issues/254#issuecomment-31669556 .

ahundt commented 10 years ago

Ok, should I make the contents section also use the separate toctrees?

schuhschuh commented 10 years ago

Depends on how Sphinx copes with it. Separate headers in the contents.rst might then be nice. On 6 Jan 2014 18:06, "Andrew Hundt" notifications@github.com wrote:

Ok, should I make the contents section also use the separate toctrees?

— Reply to this email directly or view it on GitHubhttps://github.com/schuhschuh/cmake-basis/issues/254#issuecomment-31670835 .

schuhschuh commented 10 years ago

I just noticed that the TOC is now visible on the front page even with the :includehidden: option. After reading the documentation of the toctree options, it seems the purpose of this option is different from what we want. It just instructs the toctree which has this option to also include links of those sub-toctree's which are :hidden:. But it will still render all links at the location where the toctree with the :includehidden: option is used. :(

schuhschuh commented 10 years ago

Though not my preference, I just added a proper section header to the visible Table of Contents on the home page. At least there is some reasoning of having a "Table of Contents" there with a link to the "Full Table of Contents" at the very end of it. As you have to scroll down to see it, those who use the sidebar links may not even notice it... or just ignore it.

ahundt commented 10 years ago

Hmm, that is extremely annoying.

ahundt commented 10 years ago

It seems they removed some very useful functionality there...

schuhschuh commented 10 years ago

I just figure that maybe the right way to do it is to use the contents.rst as the master_doc in the Sphinx configuration (apparently the default is actually contents... I just changed it to index :S ). The browser will still use index.html as the home page, but for Sphinx, the contents.rst file with the complete document hierarchy is the root. Then we can use toctree's in all other documents with the :hidden: option and use the :includehidden: option for the toctree's in contents.rst.

schuhschuh commented 10 years ago

The problem is resolved by using a different master document. I decided to move the toctree directives from index.rst to sidebar.rst. The contents.rst cannot be used because of the different (desired) settings for the maxdepth. As a side effect of these changes, the "Overview" sidebar menu now also shows the sub-sections when displayed at the moment. But this is actually not bad as it gives an overview of what's on the "Overview" page and you can jump directly to "Get Started" or "People" if you like.

ahundt commented 10 years ago

The problem is resolved by using a different master document.

I've discovered that this creates a new problem. Now the logo links to sidebar.html with the table of contents rather than the overview index.html page.