search

AcademySoftwareFoundation / openvdb

OpenVDB - Sparse volume data structure and tools
http://www.openvdb.org/
Apache License 2.0
2.69k stars 659 forks source link

Improve installation layout on Windows for openvdb_houdini and python #624

Open e4lam opened 4 years ago

e4lam commented 4 years ago

In writing the Install steps in my comment for #603, I needed to rejig the installation layout and would need to do less with some improvements:

Idclip commented 4 years ago

https://github.com/AcademySoftwareFoundation/openvdb/pull/646 will address (2)

Idclip commented 4 years ago

Hey @e4lam regarding (1), I was able to get this working with no installation changes, provided I have the following set:

# Assumes vcpkg/installed/blah is already on path
1 > set PATH=%PATH%:C:\Users\Me\houdini18.0\lib
2 > set HOUDINI_DSO_PATH=@/dso_^;@/dso;C:\Users\Me\houdini18.0\dso"

My thought process here is to try and keep the installation dirs the same. The dso folder contains "module" libraries which should never exist on PATH but can be opened by Houdini, where as the lib is "linkable" and should exist on PATH. I wasn't able to confirm whether or not Houdini on Windows does automatically scan C:\Users\Me\houdini18.0\dso? Additionally, I obviously also need to set PATH to the lib installation; would Houdini do this automatically were I to install to C:\Users\Me\houdini18.0\bin? I could also just copy everything as per https://github.com/AcademySoftwareFoundation/openvdb/issues/603#issuecomment-580097614 Any thoughts on the best approach?

e4lam commented 4 years ago

@Idclip wrote:

I wasn't able to confirm whether or not Houdini on Windows does automatically scan C:\Users\Me\houdini18.0\dso ?

Houdini on Windows scans HOUDINI_DSO_PATH in the same way as all the platforms. So yes according to your example setup.

Additionally, I obviously also need to set PATH to the lib installation; would Houdini do this automatically were I to install to C:\Users\Me\houdini18.0\bin ?

No, which is why I mentioned in that comment to also do: export PATH="$PATH:$HOME/houdini18.0/bin"

The dso folder contains "module" libraries which should never exist on PATH but can be opened by Houdini, where as the lib is "linkable" and should exist on PATH.

But this is non-standard. See https://cmake.org/cmake/help/latest/command/install.html for example and look for RUNTIME DESTINATION. I quote:

Consider hypothetical targets myExe, mySharedLib, and myStaticLib. The code:

install(TARGETS myExe mySharedLib myStaticLib
       RUNTIME DESTINATION bin
       LIBRARY DESTINATION lib
       ARCHIVE DESTINATION lib/static)
install(TARGETS mySharedLib DESTINATION /some/full/path)

will install myExe to /bin and myStaticLib to /lib/static. On non-DLL platforms mySharedLib will be installed to /lib and /some/full/path. On DLL platforms the mySharedLib DLL will be installed to /bin and /some/full/path and its import library will be installed to /lib/static and /some/full/path

The standard is for lib to contain files used by the linker, not the loader. Recall again that on Windows, a shared library has 2 parts to it: the .dll used by the loader at run time, and the proxy .lib used at compile/link time.

Idclip commented 4 years ago

Houdini on Windows scans HOUDINI_DSO_PATH in the same way as all the platforms. So yes according to your example setup.

Sorry, bit confused here - do you mean to say that C:\Users\<USER>\houdini<VER>\dso is automatically scanned on windows irrespective of the value of HOUDINI_DSO_PATH? As I'm unable to get Houdini to pick up my installation unless I manually set it. I was anticipating this to be the case based on the default installation path returned from the Houdini CMake, though the docs here don't mention windows paths:

https://www.sidefx.com/docs/hdk/_h_d_k__intro__creating_plugins.html

Regarding the runtime installation, that all seems good to me. I've set the runtime component of the shared lib to /bin and left the modules in dso.

e4lam commented 4 years ago

@Idclip wrote:

Sorry, bit confused here - do you mean to say that C:\Users\<USER>\houdini<VER>\dso is automatically scanned on windows irrespective of the value of HOUDINI_DSO_PATH? As I'm unable to get Houdini to pick up my installation unless I manually set it. I was anticipating this to be the case based on the default installation path returned from the Houdini CMake, though the docs here don't mention windows paths:

Sorry, I didn't read your path carefully there. You're missing the Documents subdirectory. The default on Windows should be C:\Users\<USER>\Documents\houdini<VER>\dso. The actual rules are unfortunately complicated depending on platform. In $HFS/toolkit/cmake/HoudiniConfig.cmake, you should note that the houdini_get_default_install_dir cmake function actually runs hython to use the value from hou.homeHoudiniDirectory() and then appends /dso to it. Also note that HOUDINI_DSO_PATH depends on HOUDINI_PATH and you can find out the paths resolved for it by running hconfig -ap | less.

How we find the user prefs for HOUDINI_PATH in turn depends on $HOUDINI_USER_PREF_DIR which by default depends on $HOME. On Windows, $HOME is not defined by default so it Houdini falls back to using the system value for a user's documents directory. This itself may change but defaults to $USERPROFILE/Documents. On macOS, it has a different app preferences convention so Houdini first looks to see if $HOME/houdini<ver> exists. If it doesn't (ie. the default case), then it falls back to $HOME/Library/Preferences/houdini/<VER>.

Idclip commented 4 years ago

Thanks for the explanation @e4lam I'll try installing to the path you've suggested. Interestingly it seems the reason I'm missing the Documents dir is because I'm running cmake through bash... It may be an environment thing, but if I run cmake through my bash terminal on Windows houdini_get_default_install_dir is evaluating to C:/Users/Me/houdini18.0, but in command prompt, I'm getting C:/Users/Me/Documents/houdini18.0. USERPROFILE is coming out as C:\Users\Me in both (without Documents). I've double checked by running hython directly too.

Idclip commented 4 years ago

Running hconfig in a fresh command prompt gives me the following: HOUDINI_USER_PREF_DIR=C:/Users/Me/Documents/houdini18.0 and through bash: HOUDINI_USER_PREF_DIR=C:/Users/Me/houdini18.0

e4lam commented 4 years ago

@Idclip When you run bash, it usually sets $HOME.

Idclip commented 4 years ago

Right I'm with you - confirmed that as long as I launch houdini from the same terminal env that the plugins were built from that everything works as expected without me having to change HOUDINI_DSO_PATH. This env behaviour is a bit odd but now I'm aware of it it's not a big issue.

Idclip commented 4 years ago

https://github.com/AcademySoftwareFoundation/openvdb/pull/657 will address (1) - still need to fix (3)

e4lam commented 4 years ago

@Idclip Was (2) the python extension dll filename suffix addressed elsewhere?

Idclip commented 4 years ago

Yes, here https://github.com/AcademySoftwareFoundation/openvdb/pull/646/files#diff-560924d847c60c0decf045019fc7d739R274

e4lam commented 4 years ago

Ah, thanks!

Idclip commented 4 years ago

The first two issues have now been fixed and merged! Still outstanding for this issue to be completed:

For Houdini, the python extension modules need to be installed into $HOUDINI_USER_PREF_DIR/python2.7libs which corresponds to OPENVDB_HOUDINI_PYTHONRC_DIR in openvdb_houdini/CMakeLists.txt

e4lam commented 4 years ago

Thanks! The issue here is that HOUDINI_USER_PREF_DIR varies by platform so again we might consider doing what HoudiniConfig.cmake does, which is run hython to figure it out.