blender python api documentation fails

[redchillipadi]

Snice downgrading my texlive to 2014, and sphinx to stable, I now have the following error generating the blender python API. I noticed it once before but it disappeared after a reboot

 * Generating (BPY) Blender Python API docs ...
addon not found: 'cycles'
/var/tmp/portage/media-gfx/blender-2.77a/work/blender-2.77a/doc/python_api/rst/bmesh.ops.rst
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'addon_filter', 'default'
NVIDIA: failed to execute `/usr/bin/nvidia-modprobe -u`: Permission denied.
Writing: /var/tmp/portage/media-gfx/blender-2.77a/temp/blender.crash.txt
CUDA cuInit: Unknown error
 * ACCESS DENIED:  open_wr:      /dev/nvidiactl
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'modifier', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'clip', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'action', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'pose', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'pose', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'group_select_mode', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'type', 'default'
RNA Warning: Current value "1" matches no enum in 'EnumProperty', 'group_select_mode', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'type', 'default'
RNA Warning: Current value "1" matches no enum in 'EnumProperty', 'group_select_mode', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'modifier', 'default'
RNA Warning: Current value "1" matches no enum in 'EnumProperty', 'group_select_mode', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'shape', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'modifier', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'group', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'property', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'type', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'group', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'image', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'group_select_mode', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'scene', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'group', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'group', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'modifier', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'modifier', 'default'
RNA Warning: Current value "1" matches no enum in 'EnumProperty', 'group_select_mode', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'group', 'default'
RNA Warning: Current value "1" matches no enum in 'EnumProperty', 'group_select_mode', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'scene', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'name', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'mask', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'scene', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'node_item', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'action', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'group', 'default'
Error: Not freed memory blocks: 364, total unfreed memory 0.051491 MB
Running Sphinx v1.3.1
making output directory...

...

 * --------------------------- ACCESS VIOLATION SUMMARY ---------------------------
 * LOG FILE: "/var/log/sandbox/sandbox-29452.log"
 * 
VERSION 1.0
FORMAT: F - Function called
FORMAT: S - Access Status
FORMAT: P - Path as passed to function
FORMAT: A - Absolute Path (not canonical)
FORMAT: R - Canonical Path
FORMAT: C - Command Line

F: open_wr
S: deny
P: /dev/nvidiactl
A: /dev/nvidiactl
R: /dev/nvidiactl
C: /var/tmp/portage/media-gfx/blender-2.77a/work/blender-2.77a_build/bin/blender --background --python doc/python_api/sphinx_doc_gen.py -noaudio 
 * --------------------------------------------------------------------------------

[dracwyrm]

Did it work with texlive 2015? If so, then we need to hard depend on >=2015. A lot of things need latex for docs. :P

[redchillipadi]

I remember upgrading texlive and sphinx around that time. Both of which are stable now. I will try upgrading them one by one to find the missing dependency.

[redchillipadi]

I have updated to texlive 2015 and sphinx independently and it does not fix it. I have also tried with FEATURES="-sandbox" but it still fails.

I think it must be related to trying to access nvidia-modprobe/nvidiactl from portage, which are owned by root:video, But if this is the case I am not sure why it worked previously

[dracwyrm]

The main concern is why is it trying to use /dev/nvidiactl? That's a video card driver.

This and the pdfdoc bug, I think are the remaining ones. I uploaded the ebuilds to that bug because I didn't want them readily available in case someone clones the repo and I have die statements in it. I'll look in to this because I don't have an nVidia card on my Linux side. The nVidia one is passed to the Windows KVM guest. :)

[dracwyrm]

Update: you aren't the only one: https://gist.github.com/rhaamo/8742889

Maybe it's an nVidia thing?

Do you think it's trying to access the GPU for CUDA support?

[dracwyrm]

@redchillipadi Try this in the beginning of the src_compile section:

# Binary drivers need a workaround. FOSS Drivers are fine.
cards=$(echo -n /dev/ati/card* /dev/nvidiactl* | sed 's/ /:/g')
      if test -n "${cards}"; then 
         addpredict "${cards}"
      fi

There's a cleaner solution, but this is just a test. :)

[redchillipadi]

Great, your workaround stops blender from dying during the ebuild.

 * Generating (BPY) Blender Python API docs ...
addon not found: 'cycles'
/var/tmp/portage/media-gfx/blender-2.77a/work/blender-2.77a/doc/python_api/rst/bmesh.ops.rst
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'addon_filter', 'default'
NVIDIA: failed to execute `/usr/bin/nvidia-modprobe -u`: Permission denied.
Writing: /var/tmp/portage/media-gfx/blender-2.77a/temp/blender.crash.txt
CUDA cuInit: Unknown error
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'modifier', 'default'
RNA Warning: Current value "0" matches no enum in 'EnumProperty', 'type', 'default'

However the installation completes and installs documentation for the blender and python api again

[dracwyrm]

Why the hell is it calling modprobe? Can't it see that nvidia is there from the loaded modules? I think this should be filed upstream as there were no bug reports on it that I could find.

I think it's only on CUDA? right? So I can CUDA wrap this?

[redchillipadi]

I think so, I will try removing the CUDA use flag now. Oh, there is no CUDA use flag in the blender ebuild.

[redchillipadi]

Calls to cuInit() are made from the CUDADevice constructor and device_cuda_info which are accesed from device_cuda.cpp, device.cpp and device_intern.h in intern/cycles/device

There are also a few mentions in intern/opensubdiv/opensubdiv_device_context_cuda and cuew (only compiled using WITH_CUDA_DYNLOAD)

[dracwyrm]

Two questions then: 1) If I put that "hack" in the ebuild and under a cuda use flag, then everything would compile right?

2) If WITH_CUDA_DYNLOAD was set to on, that sounds like "Dynamic Loading", so to me that means it wouldn't care about Cuda on compile, so would it still access nvidiactl and nvidia-modprobe?

[dracwyrm]

Just realised there is no cuda use flag. -_- I'm really tired.

[redchillipadi]

If you put the hack under a cuda use flag and I didn't read the euse, I would expect that the flag turns cuda on or off. In reality I could set -cuda -doc and blender would compile and cuda would be available, or +cuda -cycles and cuda would not be available.

Why not just hide the hack under the if statement for the doc use flag, as its only needed when the blender -b ... command is used to create the python documentation.

src_compile() {
        cmake-utils_src_compile

        if use doc; then
                einfo "Generating Blender C/C++ API docs ..."
                cd "${CMAKE_USE_DIR}"/doc/doxygen || die
                doxygen -u Doxyfile || die
                doxygen || die "doxygen failed to build API docs."

                cd "${CMAKE_USE_DIR}" || die
                einfo "Generating (BPY) Blender Python API docs ..."
                # Binary drivers need a workaround. FOSS Drivers are fine.
                cards=$(echo -n /dev/ati/card* /dev/nvidiactl* | sed 's/ /:/g')
                if test -n "${cards}"; then
                        addpredict "${cards}"
                fi
                "${BUILD_DIR}"/bin/blender --background --python doc/python_api/sphinx_doc_gen.py -noaudio || die "blender failed."

                cd "${CMAKE_USE_DIR}"/doc/python_api || die
                sphinx-build sphinx-in BPY_API || die "sphinx failed."
        fi
}

It looks when you start blender, cycles calls Device::available_devices which queries the device capabilities and calls cuInit(). For all blender knows it needs this functionality when processing the python script which could be commands to render a scene rather than just generate the background documentation. So I think we are better using your hack rather than patching the source.

I will try setting WITH_CUDA_DYNLOAD and let you know.

[dracwyrm]

Good point! It can be put under doc since it's only used there which is really extremely odd. Then the docs will compile correctly and the main part compile correctly and all is good.

[redchillipadi]

Well, that was interesting. I added -DWITH_CUDA_DYNLOAD and -DWITH_CYCLES_STANDALONE and blender failed to link cycles due to undefined references to GLEW.

[redchillipadi]

I am closing this issue as it has been resolved by your hack, thanks!

[dracwyrm]

That is a bit strange to fail like that. GLEW is passed in to it. We might need to put some of these bugs up on Blenders tracker.

No problem with the bug help. :) Just glad it works now.

[redchillipadi]

For some reason I am getting this error again today

 * Generating (BPY) Blender Python API docs ...
addon not found: 'cycles'
/var/tmp/portage/media-gfx/blender-2.77a/work/blender-2.77a/doc/python_api/rst/bmesh.ops.rst
 * ACCESS DENIED:  open_wr:      /dev/nvidia-uvm
CUDA cuInit: Unknown error
 * ACCESS DENIED:  open_wr:      /dev/nvidiactl

[redchillipadi]

Do we need to add /dev/nvidia-uvm to the whitelist as well?

 --------------------------- ACCESS VIOLATION SUMMARY ---------------------------  LOG FILE: "/var/log/sandbox/sandbox-2151.log" * VERSION 1.0 FORMAT: F - Function called FORMAT: S - Access Status FORMAT: P - Path as passed to function FORMAT: A - Absolute Path (not canonical) FORMAT: R - Canonical Path FORMAT: C - Command Line

F: open_wr S: deny P: /dev/nvidia-uvm A: /dev/nvidia-uvm R: /dev/nvidia-uvm C: /var/tmp/portage/media-gfx/blender-2.77a/work/blender-2.77a_build/bin/blender --background --python doc/python_api/sphinx_doc_gen.py -noaudio

F: open_wr S: deny P: /dev/nvidiactl A: /dev/nvidiactl R: /dev/nvidiactl C: /var/tmp/portage/media-gfx/blender-2.77a/work/blender-2.77a_build/bin/blender --background --python doc/python_api/sphinx_doc_gen.py -noaudio * --------------------------------------------------------------------------------

build.log.txt

[dracwyrm]

Okay. Let's try this one. Change:

cards=( /dev/ati/card* /dev/nvidiactl* )
if test -e "${cards[0]}"; then
    addpredict "${cards}"
fi

To:

cards=( /dev/ati/card* /dev/nvidia* )
if test -e "${cards[0]}"; then
    addpredict "${cards[@]}"
fi

I'm really hoping addpredict can take multiple arguments, if not, I will need to do a bash for loop. I now hope that addpredict will work in a loop.

[dracwyrm]

Scratch the above. I looked up addpredict in the gentoo manual and it only takes one argument. This is the solution:

# Workaround for binary drivers.
cards=( /dev/ati/card* /dev/nvidia* )
for card in "${cards[@]}"; do addpredict ${card}; done

It's a lot more simple now. :) Just two lines and a comment replacing the big if statement.

Can you test this, please?

[redchillipadi]

Oops I mistyped it. I have copied the above version in and am testing it now. With the version from your post above it works.

[redchillipadi]

@dracwyrm I have added the version that works on my system into master. Thanks for your help patching it.

[dracwyrm]

So this is re-closed? haha.

[redchillipadi]

It works again now, I will let you know if I see it again. Thats it from me for today, Adrian

[dracwyrm]

Good work. :) and good night.