OpenVDB: Rewrite the Makefile and submit to upstream

[dracwyrm]

Currently we are doing so many hacks on that Makefile that's it's just plain ridiculous. I know how to rewrite it, it's just I'm a bit torn about backporting to 3.1, because upstream says that 3.2 is supposed to be out "soon".

The rewrite will break up the massive install section to have individual module installs. Then we will be able to have a proper install path: src_compile will compile with emake module1 module2.... instead of make all, then we can use the use flags to target each install instead of blanking out the options to disable install

src_install then then be emake module1_install module2_install... no more hacks.

I don't know when 3.2 will be out, so it might be worth backporting. I might even be able to use the 3.2 Makefile in 3.1 directly, which would be insanely nice as they did do a few touch ups on it, so it's a good base.

[redchillipadi]

I had the same feeling when working through the build system, it might be easier to rewrite it from scratch. But I think it is important to see whether upstream are interested. Someone submitted a complete rewrite in cmake in 2014 and it is still an open pull request with no comments given. If they only wish to maintain the fragile handcrafted makefile system currently in use then we will just be rewriting the build system with each release. Although it would make our ebuild more robust.

[dracwyrm]

@redchillipadi I'm still debating about what to do about python. On one hand, it's really good that multiple pythons are supported. Users could have legacy programs that use python 2.x to connect to an OpenVDB python module. However, Blender requires version 3.5. It's easier for us to support if we just have one, but, this is a big but, from a user perspective, having multiple versions might be a good thing.

I can fix up the ebuild and the build order to make everything compile once. :)

However, there is still one glaring thing that cannot be fixed. There is no configure, which means that once make stops, all variables are forgotten. Hence, the src_install section cannot actually call emake install, unless you pass it every single one of the previous variables from the src_compile section, which balloons the ebuild, so no matter what I do, compile and install need to occur in the same section. It's not ideal, but it's the only way to go with this, and that's the way it is now.

However, we can compile all the modules without those huge compile times as is now. I can fix it, so it will build cleanly build the modules and we may not need to call emake clean at all anywhere.

I will attempt this now. Wish me luck. :P

[redchillipadi]

There is an issue with compiling everything in one go, the call to pdoc needs to use the python specific version EPYDOC="${EPREFIX}"/usr/lib/python-exec/python${EPYTHON/python/}/pdoc rather than /usr/bin/pdoc (which defaults to python 2.7 on my system).

Otherwise I get an error compiling python 3.5 documentation with the python 2.7 pydoc:

export PYTHONPATH=:/var/tmp/portage/media-gfx/openvdb-3.1.0/work/openvdb; \
    pdoc --html --html-dir ${pydocdir} pyopenvdb > /dev/null
Created doc/html/python
Traceback (most recent call last):
  File "/usr/lib/python-exec/python2.7/pdoc", line 463, in <module>
    module = pdoc.import_module(args.module_name)
  File "/usr/lib64/python2.7/site-packages/pdoc/__init__.py", line 326, in import_module
    __import__(module_name)
ImportError: dynamic module does not define init function (initpyopenvdb)
Makefile:799: recipe for target 'pydoc' failed
make: *** [pydoc] Error 1

[dracwyrm]

I'll look into if EPYTHON is still set after the loop. I think it is, then it will work fine. :)

[redchillipadi]

Thanks, once that is done I will test it out and send you the build log for the pdf compilation

[dracwyrm]

I think this resolves it: In python_module_compile:

# This is so the correct version of pdoc is used
mypyscriptdir=$(python_get_scriptdir)

In src_compile:

local myprefix="${EPREFIX}"/usr
local myinstallbase="${WORKDIR}"/install
local myinstalldir="${myinstallbase}${myprefix}"
local mypyscriptdir

Then:

if use python && use doc; then
     mypythonargs+=( EPYDOC="${mypyscriptdir}"/pdoc )
else
     mypythonargs+=( EPYDOC= )
fi

The local variable declared in src_compile so it's scope is in both functions. :)

Now the right python is being used.

[dracwyrm]

This is a copy of the ebuild and patch I am using to test. That patch is modified to give more detailed output on pdf compiling.

It will only compile the pdf and then it will die so the build logs aren't lost.

openvdb-3.1.0-makefile-fixes.patch.txt openvdb-3.1.0.ebuild.txt

[redchillipadi]

OK, I have pulled the latest commit from master and will add in your openvdp patch above and test using sphinx 1.3.1 and texlive 2014. This produces lots of errors like on your system!

[dracwyrm]

Don't forget the attached ebuild as well. :) The only error I get with that patch and ebuild is the one for pdfdoc. regular doc works and everything else works as well. I recompiled sphinx with latex support even. Nothing I do seems to do anything.

[dracwyrm]

PDFDoc doesn't compile for me with the old ebuilds. The new makefile patch is just slightly modified previous ones put into one patch.

I added the removal of python mods which means no more emake clean. Had to change the location of the pdfdoc install because when pdfdoc useflag was unset, it still tried to install it, and thus failed since the file didn't exist. I moved it to the pdfdoc section of the makefile and installs right after making the pdf.

And the few other tweaks I did.

Basically, this is simple changes that are easy to maintain. I'm rewriting the makefile to be better and submit that to upstream, but that will take time and I don't want to slow the submission to the portage tree. I will have time to make it really nice and I hope they accept it, as it would greatly improve everything.

So, this is half done.

[dracwyrm]

Upstream hasn't touched my PR. :(