Automatic HTML Documentation Mola

[modellstadt]

i tried this (for single files it works, but not for the package) dillenburger_b@arch-dept-vpn-76 ~ % python -m pydoc -w /Users/dillenburger_b/github/mola
no Python documentation found for 'blender' no Python documentation found for 'colab2D' no Python documentation found for 'colab3D' no Python documentation found for 'color' no Python documentation found for 'core' no Python documentation found for 'faceUtils' no Python documentation found for 'factory' no Python documentation found for 'graph' no Python documentation found for 'grid' wrote io.html no Python documentation found for 'marchingCubes' no Python documentation found for 'mathUtils' no Python documentation found for 'polyUtils' no Python documentation found for 'processing' no Python documentation found for 'rhino' no Python documentation found for 'slicer' no Python documentation found for 'subdivision' no Python documentation found for 'testing' no Python documentation found for 'vec'

[modellstadt]

I think pdoc3 is maybe a good and simple choice. https://pdoc3.github.io/pdoc/

[modellstadt]

I still get errors when i try to generate the html on my computer. any ideas? it should be straightforward.

[tetov]

Changed the file structure a bit to have the repo contain the package, instead of "being" the package, see below. I also had to excluded modules_*, to include those there needs to be some conditions on the imports. Like not trying to import stuff from Rhino when not run from Rhino. I can set that up if that's of interest.

README.md

./mola:
colab2D.py
colab3D.py
core_box.py
core_edge.py
core_face.py
core_grid.py
core_mesh.py
core_vertex.py
graph.py
grid_factory.py
__init__.py
io.py
mesh_factory.py
mesh_marching_cubes.py
mesh_subdivision.py
module_blender.py
module_processing.py
module_rhino.py
slicer.py
utils_color.py
utils_face.py
utils_math.py
utils_mesh.py
utils_poly.py
utils_vertex.py

[modellstadt]

Amazing! We will check this.

[modellstadt]

@worbit @demsham Shall we take over this?

[modellstadt]

@tetov what was the trick with the doc creation?

[tetov]

Not much of a trick, it's mainly moving python modules to a folder to make it more of a typical package.

Try running it with the folder structure in this branch:

git clone https://github.com/tetov/mola/
git checkout docs_demo
pip install pdoc3 compas
pdoc3 --html mola

compas is only there so it can be imported for module_compas. Could be skipped if the compas import in that file is also wrapped in try/except like I did for module_blender.

I would set up a travis job or github action to generate the docs and push them to branch gh-pages and publish them from there. Or do it locally, something like this:

rm -rf html && pdoc3 --html mola && \
cd html/mola && git init && git add . \
&& git commit -m "Pushing docs" && \
git remote add origin https://github.com/dbt-ethz/mola && \
git push --force origin master:gh-pages

Code above would build docs, create a new repository inside the folder with the documentation and then force push that to branch gh-pages.

[worbit]

hey anton @tetov

thank you very much for the proposal. i cloned your fork and tested building the docs. works smoothly, great! before we publish it to gh-pages, i would have a question regarding layout: do you know how we would have to either change the docstrings or the css to have the html doc more closely reflecting the source code, i.e. have x2,y2,zy2 on a new line? is it interpreting it as MD and a simple <br> would do the job?

[worbit]

yes, does the job!

[demsham]

cool. will the break be visible in the colab popup window?

[tetov]

In markdown one linebreak on it’s own is ignored, you need two.

I would change it to use NumpyDoc/ReST-like instead since a lot of docs are quite close to that (or maybe closer to Google style).

If you are ok with this folder structure change, then I can submit a PR to make mola pip installable. That will make integration into Rhino easier (python -m compas_rhino.install -p mola).

[worbit]

thanks. i would strongly suggest we keep different ideas separate. for the ACDC course (the primary use of mola) we don't need rhino integration and also no compas. so ideally we find a good solution for the docs first, as a reference for all students and pure mola only. compas/rhino/xy?? integration should be handled separately

[tetov]

Haven't used colab much at all, but making it a package (and potentially publishing it to pypi) would make it easier to install there as well, right? !pip install git+https://github.com/dbt-ethz/mola or !pip !install mola.

[worbit]

how do you mean "instead", instead of pdoc3? or as a formatting style for docstrings like here?

furthermore, i would suggest to have mola/src/some_file.py instead of mola/mola/some_file.py

what do you think?

[worbit]

Haven't used colab much at all, but making it a package (and potentially publishing it to pypi) would make it easier to install there as well, right? !! !pip !install mola

yes, but as we keep adding features (and bugfixes) during the courses, it is even easier to just run as a first cell

!rm -rf mola
!git clone https://github.com/dbt-ethz/mola.git

then they always have the latest version

[tetov]

This is the first time I've worked with pdoc3, so I haven't tested this but my assumption would be that adhering to a standard docstring format would make the docs render nicer output that just parsing it as markdown. And yes, I mean numpy docstyle or google docstyle (both are supported :). So far I haven't found a config option in pdoc3 to set the style so maybe this is a moot point though.

[tetov]

Haven't used colab much at all, but making it a package (and potentially publishing it to pypi) would make it easier to install there as well, right? !! !pip !install mola

yes, but as we keep adding features (and bugfixes) during the courses, it is even easier to just run as a first cell

!rm -rf mola
!git clone https://github.com/dbt-ethz/mola.git

then they always have the latest version

Sorry, I managed to post before I finished the comment. I understand that workflow, but a) same could be done using !pip install --upgrade https://github.com/dbt-ethz/mola and b) having it installable by pip doesn't break that workflow.

[tetov]

[...]

furthermore, i would suggest to have mola/src/some_file.py instead of mola/mola/some_file.py

what do you think?

Then it needs to be mola/src/mola/some_file.py, since the directory name of the top level __init__.py sets the package name in terms of importing it. Your suggestion would mean that you would need to do import src.core_vertex.Vertex which is very confusing.

And if you have only one package in your repo you can skip having src.

[worbit]

ok, i see and agree. mola/mola is better then, as we don't have any sub-packages and src would only contain mola anyway. and as for the documentation: i suggest to name this folder docs instead of html. i will set up the repo to point to master/docs then!

so:

• mola
  • mola
  • some_file.py
  • docs
  • some_file.html
  • readme.md

[tetov]

I agree, naming the folder html is pdoc3's default but I don't think it makes much sense.

[worbit]

having a pip installable package is a nice idea to definitely keep in mind for the (near) future as well! but first and foremost, let's build those docs.

[tetov]

having a pip installable package is a nice idea to definitely keep in mind for the (near) future as well! but first and foremost, let's build those docs.

Yeah, I agree. I meant that more as an additional reason to restructure all files ;). (Only extra thing needed would be a short setup.py file)

Let me know if you need my help with setting up CI.

[tetov]

hey anton @tetov

thank you very much for the proposal. i cloned your fork and tested building the docs. works smoothly, great! before we publish it to gh-pages, i would have a question regarding layout: do you know how we would have to either change the docstrings or the css to have the html doc more closely reflecting the source code, i.e. have x2,y2,zy2 on a new line? is it interpreting it as MD and a simple <br> would do the job?

Looking closer at this, I'd guess that the docstring parser is confused by the variable name "x1,y1,z1" as it contains commas (because I don't think it will parse it as three different variables with the same description).

[worbit]

indeed, this seems to be the reason as for single variables it works. good catch! i put one version of the docs online, see here: https://dbt-ethz.github.io/mola/ i left the folder structure of the master branch still the same though, as otherwise the imports in the colab notebooks wouldn't work anymore as they used to. because it is mola/mola/some_file.py, we can no longer do import mola.colab3D for example. we will have to "pull these up" to top level through the __init__.py files i guess (as COMPAS does)?

[tetov]

No, that would not really be good practice. You should instead put mola top level directory in your python search path.

So generic way would be:

import sys
sys.path.insert(1, path/to/mola’)
import mola.colab_3d

Pip install would put the mola/mola folder in the environments site-packages folder which is in the python search path by default (which is how compas gets the packages under src on path).

I’ll have to look into how colab does this. I’ll post that in a moment.

EDIT: @worbit could you please share a colab notebook where mola is imported using the current setup?

[worbit]

sure, see here: https://colab.research.google.com/drive/1rg9GwBLKw1w3G7I6YIu6IrNkZ_t-2cY5

[tetov]

Ok! The current directory is always part of the python search path, so in colab you currently don’t have to worry about search paths at all..

I would suggest then that for the colab use case you should change the first import statement to from mola import mola or import mola.mola as mola. I’d probably also make sure to include a comment for curious students and future users. So maybe:

# import source folder from repository
from mola import mola 

In other circumstances you either add the path to the repository to the search path or you install it using pip.

[tetov]

Now I understand why the current repo has the python modules in the root folder :). It makes a lot of sense for a setup like colabs’.

I’m sure we could get pdoc3 or another tool to work with current setup, but using a more conventional structure will make this tool as well as future tools easier to integrate. And it only requires changing one line in the colab environment :).

[modellstadt]

From a user perspective, we believe that for auto-complete the separation of parts of the code via dot is most convenient. So if one writes mola. some (but not all) functions appear and if someone writes mola.colab3D. functions related to those render methods appear. I believe this aspect has the highest priority. Changing one or two lines in the import statement and changing folder structures within mola would be acceptable, if we, therefore, gain nice documentation and compatibility with other tools. To me, one design goal of this lib is in how easy it is to use (for our course students for example)