hannorein / rebound

💫 An open-source multi-purpose N-body code.
https://rebound.readthedocs.io/
GNU General Public License v3.0
852 stars 219 forks source link

Documentation #529

Closed hannorein closed 3 years ago

hannorein commented 3 years ago

I've been thinking a lot about how to best structure the documentation for this project. At least for the C part, I think the best way is to write the documentation manually rather than relying on some package do extract the documentation from the source files automatically. Using doxygen or a similar package is just either too constraining or too much fiddling around is required to make a simple change. So many times I wished I could just write a simple markdown file.

I'm experimenting with this now on the mkdocs branch. From a practical point of view, this means that the rebound.h file will only contain the actual prototype definitions and some sparse comments which are not needed in the public-facing documentation itself. All of the actual documentation is in several markdown files. This means that when changing the API, one has to edit not only rebound.h but also the markdown files to keep them in sync. On the positive side, the rebound.h file should end up a few thousand lines of code shorter!

I'm using mkdocs for the rendering of the markdown files with the materials plugin/theme. It's static, looks nice and clean, and can render equations. That's about all we need.

I'm still thinking about the python side of things. I'm probably going to keep the doc strings in the python code itself, but then write a simple script to pull them into various markdown files. For a bigger project sphinx would clearly be the better option, but it again requires too much fiddling around to get a nice looking documentation for a relatively small but somewhat non-standard package such as REBOUND.

Findus23 commented 3 years ago

Honestly, I find Sphinx pretty straightforward, so I am not sure what's the improvement when switching to mkdocs. If the improvement is just using Markdown (which I understand is more straightforward than rst) then one could simply use https://www.sphinx-doc.org/en/master/usage/markdown.html.

I have no experience with doxygen, so I can't really say if using it or manually edited files are better, but at least in python I'd prefer doc strings to a separate documentation as they allow to read the documentation with just one "jump to source" instead of having to look up the right part of the website.

Also I think it is okay for the function reference/documentation to be quite separate from the rest of the documentation as they serve a different goal. https://diataxis.fr/ expands nicely on this topic in my opinion and also splits up tutorials (how do I get started/ set up rebound), how-to-guides (the ipython notebooks) and explanation (explaining how rebound works and why it does things the way it does) from each other.

I don't really have a great solution for the c-python split, but I think they are different enough that one can't just write every example in both, so maybe keeping the split as it is now with most examples being in python and the few where it makes sense in C might be good enough.

hannorein commented 3 years ago

Thanks for the feedback. Exactly what I was hoping for!

I agree that there's nothing wrong with sphinx. But it also doesn't have any killer features we'd absolutely need for a project like this. When I was exploring various options, I liked mkdocs. It looks simple and clean. And the live-preview is very nice to work with.

REBOUND has a few quirks which might make an argument for doing things more manually:

Findus23 commented 3 years ago

One way to solve the c-python issue would be to write the tex parts of the how-to pages general and then have code-boxes with selectors allowing to switch between C and Python. I have seen this pattern on a few larger libraries, but it might work better when both languages are just different syntax for the same thing and not like in rebound where C and Python probably have different use-cases.

I guess the user-facing API are all classes. But they are already roughtly documented https://github.com/hannorein/rebound/blob/1efd409af957af95b4d7b972953ecf1854dec390/rebound/simulation.py#L381-L382

And the rest I think fits better into the why-guides that expand more about the physics than the code.

And for really in-deep things comments in the code might be enough (or pointers in the documentation towards comments in the code)

hannorein commented 3 years ago

Current experimentation status: https://rebound.readthedocs.io/en/mkdocs/

hannorein commented 3 years ago

I think I'm slowly converging on something that I'm happy with. I really like @Findus23's suggestion of code boxes with selectors. Apple is doing the same in their documentation with Objective-C/Swift. That is by far the easiest way to maintain a tutorial-style documentation without having to do everything twice. Still a lot of work to do, but this page already provides a good example of how this could work nicely: https://rebound.readthedocs.io/en/mkdocs/integrators/

dtamayo commented 3 years ago

Wow! You are the hero we need but don't deserve @hannorein

Findus23 commented 3 years ago

I have to say that the new docs are pretty great. I'm not yet used to the new page structure, but it is probably better than the old one.

If there is any way I can help, just tell me.

hannorein commented 3 years ago

Thanks! I'm still working on the basics, including figuring out how to structure it best. Give me a bit more time, but then I'd love your help!

hannorein commented 3 years ago

Ok. So I think I more or less have a complete draft. I'll continue to add more content in the future. For example, I'd love to have a more in-depth discussion on orbital elements (very general, i.e. not just their implementation in REBOUND, maybe we can attract some users to REBOUND just for the orbital element calculations). But for now, have a look at the overall structure. Does that make sense to you, is it confusing, etc? Don't hold back any negative feedback ;-)

https://rebound.readthedocs.io/en/mkdocs/

dtamayo commented 3 years ago

This is absolutely fantastic. I really like the tabs for C and Python, as well as the info and danger boxes to organize information.

I think one of the challenges is that we're all a bit far removed from what it's like to be new to REBOUND. Maybe it would be useful to compile a list on twitter or elsewhere of things people wish they'd known when they started using REBOUND? And what would have been most useful to see first? For example, my naive feeling is that it might be most useful for the very top of the Home page to link to the quickstart guide, and if the 'where to go from here' section at the bottom had a link to say the top first 5 things to learn in REBOUND, e.g.

  1. Setting units
  2. Orbital elements
  3. Saving and reloading simulations
  4. How to catch close encounters ...

I like how you have done a version of this in 'just getting started' under the planetary systems tab in the examples section, but I think many users would find it useful for this to be one of the first things they see on the documentation page. I might be wrong and that might also be a good thing to get feedback on on twitter! It might also be that the most useful would be a simplified notebook that showed the basics of all of the above in one. That could be something the community would be very useful in putting together.

I also like how you made separate sections for the simulation and particle classes, as well as the various modules and other concepts. To a newcomer though the logic of that organization in the tabs at the top might not be obvious though. Another possibility would be to replace them all with a single 'API' (or something else) tab, which gives a top level overview 'The most important classes you will use over and over are the simulation and particle classes... and then links to the simulation, particle, modules and other concepts sections'. Would it be useful for the quick start guide to be one of the top tabs?

You have a better sense of what is easiest to set up and maintain...what about setting up the examples on the explorer on the left so that it's a single name, and then it pulls up either the C example, the python example, or both when you click on it? That would somewhat streamline the menu options and make it easier to find what you want, and encourage people to look at both the C and python especially in cases where one is available and the other is not.

Clearly the ease of maintaining the documentation is crucial and some of these might not be feasible!

hannorein commented 3 years ago

Thanks Dan! That's super helpful. I think a bit more about it but I'll probably follow all your advice!

hannorein commented 3 years ago

I've changed the structure a bit following Dan's suggestions. I'll continue to fiddle around with it. But I think the new documentation is already much better than the old one. So I'll merge it into the main branch.

I made some minor changes, removed what I believed were deprecated functions, etc. @dtamayo, can you do a quick check that I didn't break REBOUNDx in the process?

dtamayo commented 3 years ago

I can't seem to get travis to work right now, but when I try things out locally it seems to work fine! Will keep trying to run the automatic tests over the weekend

hannorein commented 3 years ago

No rush!

hannorein commented 3 years ago

I'll keep working on this, but I think the current version is a good start, so I'll close the issue for now. Thanks for all the help! And please continue to suggest/make changes if you see an opportunity to improve things...

dtamayo commented 3 years ago

Hi Hanno,

Just got around to this and fixed the REBOUNDx examples to work with the new REBOUND. Also upgraded the versions of python being tested after an hour of not being able to resolve an easy_install issue specifically with Python 3.4!

hannorein commented 3 years ago

Yay! Sorry of I caused any trouble. CI is getting more and more annoying. I spend more time fixing those than the actual bugs...

dtamayo commented 3 years ago

Indeed! And no worries, the new add particle in C is extremely convenient!! I decided though to update them to use the old method just in case someone needs compatibility with an older version of REBOUND