Houdini in Windows 10 - Githubissues

Idclip commented 4 years ago

I'm trying to build openvdb against Houdini in Windows 10, using vcpkg (and then Visual Studio 2019).

I've run into the same error, which I haven't been able to resolve:

CMake Error at C:/Program Files/CMake/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
  Could NOT find IlmBase (missing: IlmBase_LIB_COMPONENTS Half) (found
  suitable version "2.2", minimum required is "2.2")
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindIlmBase.cmake:285 (find_package_handle_standard_args)
  C:/vcpkg/scripts/buildsystems/vcpkg.cmake:263 (_find_package)
  openvdb/CMakeLists.txt:61 (find_package)

Firstly, it seems odd that it says "found suitable version" when it's also saying "could not find" it.

Also, inside C:\vcpkg\installed\x64-windows\lib I can see Half:

Half-2_3.lib
Half-s2_3_s.lib

So, it seems that it's there but CMake just can't find it. I get the same error even if I explicitly point to this lib directory with my CMake command:

-DILMBASE_ROOT=”C:/vcpkg/installed/x64-windows/lib”

I've been making a guide document for myself as I go, which I've attached to show the detail of what I've done so far. OpenVDB installation using vcpkg - github query.pdf

In summary, my steps have been:

.\vcpkg install --triplet x64-windows ilmbase blosc boost zlib openexr tbb boost-python cppunit 
.\vcpkg integrate install

If I run vcpkg list, it shows that ilmbase is installed:

ilmbase:x64-windows              2.3.0            empty package, linking to newer one

I've cloned the openvdb git:

git clone git@github.com:AcademySoftwareFoundation/openvdb.git

My CMake command in Windows Powershell is:

cmake `
-S"C:/vcpkg/openvdb" `
-G"Visual Studio 16 2019" `
-DOPENVDB_ABI_VERSION_NUMBER=5 `
-DCMAKE_TOOLCHAIN_FILE="C:/vcpkg/scripts/buildsystems/vcpkg.cmake" `
-DHoudini_ROOT="C:/Program Files/Side Effects Software/Houdini 17.5.425" `
-DUSE_HOUDINI=ON `
-DOPENVDB_BUILD_HOUDINI_PLUGIN=ON `
-DUSE_DEFAULT_HOUDINI_INSTALL=ON `
-DOPENVDB_BUILD_PYTHON_MODULE=ON `
-DOPENVDB_BUILD_UNITTESTS=ON `
-DOPENEXR_ROOT="C:/vcpkg/installed/x64-windows" `
-DILMBASE_ROOT=”C:/vcpkg/installed/x64-windows/lib” `
-DTBB_ROOT="C:/vcpkg/installed/x64-windows" `
-DUSE_BLOSC=OFF `
-DCPPUNIT_ROOT="C:/vcpkg/installed/x64-windows" `
-DBOOST_ROOT="C:/vcpkg/installed/x64-windows" `
-DBOOST-PYTHON_ROOT="C:/vcpkg/installed/x64-windows" `
-DBOOST_LIBRARYDIR="C:/vcpkg/installed/x64-windows" `
-DZLIB_ROOT="C:/vcpkg/installed/x64-windows" `
..

I'm wondering if the error is because CMake is expecting to find Half in a discrete ilmbase libary but Half is now actually built inside OpenEXR, which the empty ilmbase package should be pointing to but isn't.

I'd really appreciate some guidance on resolving this!

Originally posted by @ianww in https://github.com/AcademySoftwareFoundation/openvdb/issues/429#issuecomment-571989625

e4lam commented 4 years ago

@ianww I don't see anything wrong with them except I'd double-check whether it removes any macros that are required by the OpenVDB build. An example command line of compiling one of those problem .cc files would be useful to see what's actually happening in your build.

If we're getting the same linker problems regardless on VS2017 or VS2019, it may very well be that the approach in openvdb_houdini for being able to use a custom OpenVDB library in Houdini doesn't work on Windows. I've never heard of anyone doing this on Windows. Having said that, I do vaguely recall being able to compile openvdb_houdini on Windows before though but that was a very long time ago (4+ years) and in our own forked Makefile build system.

I think a question I have at this point, is what are your aims for compiling OpenVDB yourself?

ianww commented 4 years ago

I want to use Houdini for the visualisation of astronomical data, eg. (http://www.ytini.com/).

In the first instance I'll only actually need the pyopenvdb component but ultimately I may want to use openvdb SOPS that aren't already native to Houdini.

In the build attempts so far, I've turned off the python module build because I was getting a CMake error with that and I didn't want to further complicate the troubleshooting that you guys (@e4lam, @Idclip) have been so kindly been helping me with. The error was:

CMake Error at C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.15/Modules/FindPython/Support.cmake:1550 (list):
1> [CMake]   list index: 2 out of range (-2, 1)

If solving that python error is simple (any thoughts on it?), I might try just a python only build, which would allow me to at least get going with ytini for now, in the hope that a full Windows build might become trouble free in later versions.

Another option I've been contemplating is to make my machine dual boot and use Linux. I presume that building with Linux would (perhaps always?) be much more straightforward (although I'll need to learn a new suite of build software from scratch).

What's your advice for the long term - will openvdb always be more Linux-centric and easier than Windows?

(I do hate giving up though - I feel like we're so close to success with Windows, and I'd love to make it through to a successful end. From my novice perspective it just seems that the last remaining bug is a wrong path APPEND being made to the build paths, as per https://github.com/AcademySoftwareFoundation/openvdb/issues/603#issuecomment-576677227.)

EDIT: Regarding that earlier comment, I've since realised/worked out that C:\vcpkg\openvdb\out\build\x64-Release\openvdb_houdini\GEO_VDBTranslator.obj is not where CMake is finding GEO_VDBTranslator.obj but rather where it is trying to build it to, and that ...\openvdb_houdini.dir is an intermediate directory in that build process. So, the error is more fundamental than a simple path error.

ianww commented 4 years ago

@e4lam, @Idclip

Almost all solved! I've managed to get rid of all but one of the LINK errors.

All the GEO_PrimVDB errors have gone, and only a single GU_PrimVDB LINK error is left.

The GEO_PrimVDB errors were due to my missing some places where SYS_FORCE_INLINE needed to be added to the Houdini 17.5 GEO_PrimVDB.h file. I went back through that file and the Houdini 18.0.352 daily build version of it side by side to match up occurrences, and added those that I'd missed. Sorry about wasting your time by those misses.

The remaining error is:

C:\vcpkg\openvdb\out\build\x64-Release\openvdb_houdini\Utils.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: static class GU_PrimVDB * __cdecl GU_PrimVDB::buildFromGrid(class GU_Detail &,class std::shared_ptr<class openvdb::v6_2abi5::GridBase>,class GEO_PrimVDB const *,char const *)" (__imp_?buildFromGrid@GU_PrimVDB@@SAPEAV1@AEAVGU_Detail@@V?$shared_ptr@VGridBase@v6_2abi5@openvdb@@@std@@PEBVGEO_PrimVDB@@PEBD@Z) referenced in function "class GU_PrimVDB * __cdecl openvdb_houdini::createVdbPrimitive(class GU_Detail &,class std::shared_ptr<class openvdb::v6_2abi5::GridBase>,char const *)" (?createVdbPrimitive@openvdb_houdini@@YAPEAVGU_PrimVDB@@AEAVGU_Detail@@V?$shared_ptr@VGridBase@v6_2abi5@openvdb@@@std@@PEBD@Z) 
C:\vcpkg\openvdb\out\build\x64-Release\openvdb_houdini\Release\openvdb_houdini.dll : fatal error LNK1120: 1 unresolved externals

I presume that this will also need SYS_FORCE_INLINE to be added in one or more places in GU_PrimVDB but I've not yet managed to work out where - the potential locations are harder to determine than for GEO_PrimVDB. There don't seem to be as clear-cut (for a non-expert like me) error-matching method declarations in GU_PrimVDB as there were in GEO_PrimVDB.

I'll keep trying but if you have some ideas I'd be grateful!

e4lam commented 4 years ago

@ianww Perhaps you missed adding SYS_FORCE_INLINE for that method in GU_PrimVDB.h? This was part of the changes that I mentioned in my comment about fixing in 18.0.353.

This is how it looks like for me in $HFS/toolkit/include/GU/GU_PrimVDB.h:

    static SYS_FORCE_INLINE
    GU_PrimVDB* buildFromGrid(GU_Detail& gdp, openvdb::GridBase::Ptr grid,
    const GEO_PrimVDB* src = NULL, const char* name = NULL)
    {
    return GU_PrimVDB::buildFromGridAdapter(gdp, &grid, src, name);
    }

e4lam commented 4 years ago

PS. It's also not just that method in GU_PrimVDB.h that was modified, there were at least a few of them.

ianww commented 4 years ago

Many thanks @e4lam. Sorry, I'd forgotten about your GU_PrimVDB.diff.txt document in https://github.com/AcademySoftwareFoundation/openvdb/issues/603#issuecomment-575861498. I'd set it aside while I worked on GEO_PrimVDB and forgot to return to it.

I've now made those SYS_FORCE_INLINE changes and all the GEO_PrimVDB and GU_PrimVDB errors appear to have been fixed, which is a truly excellent result!

But, as seems to be an inevitability, another new lot of similar LINK errors have appeared as the build reaches further, this time in GR_PrimVDBPoints:

C:\vcpkg\openvdb\out\build\x64-Release\openvdb_houdini\GR_PrimVDBPoints.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: class openvdb::v6_2abi5::GridBase const * __cdecl GT_PrimVDB::getGrid(void)" (__imp_?getGrid@GT_PrimVDB@@QEAAPEBVGridBase@v6_2abi5@openvdb@@XZ) referenced in function "public: virtual void __cdecl GR_PrimVDBPoints::update(class RE_Render *,class UT_IntrusivePtr<class GT_Primitive> const &,class GR_UpdateParms const &)" (?update@GR_PrimVDBPoints@@UEAAXPEAVRE_Render@@AEBV?$UT_IntrusivePtr@VGT_Primitive@@@@AEBVGR_UpdateParms@@@Z) 
C:\vcpkg\openvdb\out\build\x64-Release\openvdb_houdini\GR_PrimVDBPoints.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: __cdecl GUI_PrimVDBPointsHook::GUI_PrimVDBPointsHook(void)" (__imp_??0GUI_PrimVDBPointsHook@@QEAA@XZ) referenced in function newRenderHook 
C:\vcpkg\openvdb\out\build\x64-Release\openvdb_houdini\GR_PrimVDBPoints.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: virtual __cdecl GUI_PrimVDBPointsHook::~GUI_PrimVDBPointsHook(void)" (__imp_??1GUI_PrimVDBPointsHook@@UEAA@XZ) referenced in function "public: virtual void * __cdecl GUI_PrimVDBPointsHook::`scalar deleting destructor'(unsigned int)" (??_GGUI_PrimVDBPointsHook@@UEAAPEAXI@Z) 
C:\vcpkg\openvdb\out\build\x64-Release\openvdb_houdini\GR_PrimVDBPoints.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: virtual __cdecl GR_PrimVDBPoints::~GR_PrimVDBPoints(void)" (__imp_??1GR_PrimVDBPoints@@UEAA@XZ) referenced in function "public: virtual void * __cdecl GR_PrimVDBPoints::`scalar deleting destructor'(unsigned int)" (??_GGR_PrimVDBPoints@@UEAAPEAXI@Z) 
C:\vcpkg\openvdb\out\build\x64-Release\openvdb_houdini\GR_PrimVDBPoints.obj : error LNK2001: unresolved external symbol "public: virtual char const * __cdecl GR_PrimVDBPoints::className(void)const " (?className@GR_PrimVDBPoints@@UEBAPEBDXZ) 
C:\vcpkg\openvdb\out\build\x64-Release\openvdb_houdini\GR_PrimVDBPoints.obj : error LNK2001: unresolved external symbol "public: virtual void __cdecl GR_PrimVDBPoints::renderDecoration(class RE_Render *,enum GR_Decoration,class GR_DecorationParms const &)" (?renderDecoration@GR_PrimVDBPoints@@UEAAXPEAVRE_Render@@W4GR_Decoration@@AEBVGR_DecorationParms@@@Z) 
C:\vcpkg\openvdb\out\build\x64-Release\openvdb_houdini\GR_PrimVDBPoints.obj : error LNK2001: unresolved external symbol "public: virtual int __cdecl GR_PrimVDBPoints::renderPick(class RE_Render *,class GR_DisplayOption const *,unsigned int,enum GR_PickStyle,bool)" (?renderPick@GR_PrimVDBPoints@@UEAAHPEAVRE_Render@@PEBVGR_DisplayOption@@IW4GR_PickStyle@@_N@Z) 
C:\vcpkg\openvdb\out\build\x64-Release\openvdb_houdini\GR_PrimVDBPoints.obj : error LNK2001: unresolved external symbol "public: virtual void __cdecl GR_PrimVDBPoints::resetPrimitives(void)" (?resetPrimitives@GR_PrimVDBPoints@@UEAAXXZ) 
C:\vcpkg\openvdb\out\build\x64-Release\openvdb_houdini\GR_PrimVDBPoints.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) const GR_PrimVDBPoints::`vftable'" (__imp_??_7GR_PrimVDBPoints@@6B@) referenced in function "public: __cdecl GR_PrimVDBPoints::GR_PrimVDBPoints(class GR_RenderInfo const *,char const *,class GEO_Primitive const *)" (??0GR_PrimVDBPoints@@QEAA@PEBVGR_RenderInfo@@PEBDPEBVGEO_Primitive@@@Z) 
C:\vcpkg\openvdb\out\build\x64-Release\openvdb_houdini\GR_PrimVDBPoints.dll : fatal error LNK1120: 9 unresolved externals

GR_PrimVDBPoints is different than GEO_PrimVDB and GU_PrimVDB because as far as I can tell it's only present as an _openvdbhoudini cc file, whereas the others had Houdini h files to edit (but with their cc counterparts in _openvdbhoudini).

I've tried doing a similar thing, putting SYS_FORCE_INLINE into GR_PrimVDBPoints.cc where I think the errors are occurring but so far it hasn't helped.

I'm not enough of a programmer to know whether the absence of a GR_PrimVDBPoints.h file is an error in itself.

In the absence of an GR_PrimVDBPoints.h to edit, would the same approach of adding SYS_FORCE_INLINE apply for an _openvdbhoudini cc file as it did for the Houdini h files?

e4lam commented 4 years ago

@ianww GR_PrimVDBPoints is just the plugin for displaying point primitives. You can probably just take it out in the mean time.

e4lam commented 4 years ago

@ianww For the missing references to methods inside GR_PrimVDBPoints.ccthemselves, I actually had ran into this from my clang-cl build and had a local fix for this. I removed the OPENVDB_HOUDINI_API from the class declarations in GR_PrimVDBPoints.cc. I think this is before @Idclip fixed the missing -DOPENVDB_HOUDINI_PRIVATE compiler option (in #606?) so we were trying to import these classes when we should be exporting. However It doesn't make sense to export these classes in the first place because they're registered through an exported plugin installation method already. @Idclip can you perhaps make this fix? I've been kinda waiting for all the Windows related fixes to be merged upstream before I try to make the clang-cl pass changes.

For the link problems with Houdini's GT_PrimVDB.h, I've got a fix that may help. As in GEO_PrimVDB and GU_PrimVDB, any methods with openvdb::GridBase in their signature need to be SYS_FORCE_INLINE. This is fixed in Houdini 18.0.360.

e4lam commented 4 years ago

PS. I forgot to mention that the GT_PrimVDB.h requires changes in the implementation side as well. So unless we change the calling code to GT_PrimVDB::getGrid() right now in openvdb_houdini, Houdini 18 must be used.

ianww commented 4 years ago

Thanks @e4lam.

As you suggested, I took GR_PrimVDBPoints.cc out of the build, so its LINK errors went. On then running a build I didn't actually get any LINK errors with Houdini's GT_PrimVDB.h but I subsequently added SYS_FORCE_IN_LINE's to that too anyway, just to be sure.

So, all the LINK errors have been fixed!

Two new errors emerged.

The first one was:

C:\vcpkg\openvdb\openvdb_houdini\SOP_OpenVDB_Visualize.cc(894): error C2668: 'TreeVisualizer::addPoint': ambiguous call to overloaded function 
    C:\vcpkg\openvdb\openvdb_houdini\SOP_OpenVDB_Visualize.cc(953): note: could be 'void TreeVisualizer::addPoint<openvdb::v6_2abi5::PointIndex<openvdb::v6_2abi5::Index32,0>>(const openvdb::v6_2abi5::math::CoordBBox &,const UT_Vector3 &,ValType,bool)'

I fixed this by parasitisting some code from the git master's SOP_OpenVDB_Visualize.cc file (I'm necessarily working with version 6.2 for Houdini 17.5 compatibility reasons).

Specifically, I first added this namespace code near the head of the file:

Namespace code added

``` namespace { template struct IsGridTypeIntegral : std::conditional_t< std::is_integral::value || std::is_same::value || std::is_same::value || std::is_same::value || std::is_same::value , std::true_type , std::false_type> { }; template struct IsGridTypeArithmetic : std::conditional_t< IsGridTypeIntegral::value || std::is_floating_point::value , std::true_type , std::false_type> { }; } // anonymous namespace ```

and then at around original line 955 (line numbers before the above addition), I replaced

typename std::enable_if<!std::is_arithmetic<ValType>::value>::type

with

typename std::enable_if<!IsGridTypeArithmetic<ValType>::value>::type

in this code block:

Code block edited

``` template ///typename std::enable_if::value>::type /// ianww deletion typename std::enable_if::value>::type ///ianww addition based on git master TreeVisualizer::addPoint(const openvdb::CoordBBox& bbox, const UT_Vector3& color, ValType v, bool staggered) { if (!staggered) { mVec3fHandle.set(createPoint(bbox, color), UT_Vector3(float(v[0]), float(v[1]), float(v[2]))); } else { openvdb::Vec3d pos = openvdb::Vec3d(0.5*(bbox.min().x()+bbox.max().x()), 0.5*(bbox.min().y()+bbox.max().y()), 0.5*(bbox.min().z()+bbox.max().z())); pos[0] -= 0.5; // -x GA_Offset offset = createPoint(pos); if (mCdHandle.isValid()) mCdHandle.set(offset, UT_Vector3(1.0, 0.0, 0.0)); // r mVec3fHandle.set(offset, UT_Vector3(float(v[0]), 0.0, 0.0)); pos[0] += 0.5; pos[1] -= 0.5; // -y offset = createPoint(pos); if (mCdHandle.isValid()) mCdHandle.set(offset, UT_Vector3(0.0, 1.0, 0.0)); // g mVec3fHandle.set(offset, UT_Vector3(0.0, float(v[1]), 0.0)); pos[1] += 0.5; pos[2] -= 0.5; // -z offset = createPoint(pos); if (mCdHandle.isValid()) mCdHandle.set(offset, UT_Vector3(0.0, 0.0, 1.0)); // b mVec3fHandle.set(offset, UT_Vector3(0.0, 0.0, float(v[2]))); } } ```

That got rid of the first error.

The second (and it may be the very last remaining error!) is:

C:\vcpkg\openvdb\openvdb_houdini\VRAY_OpenVDB_Points.cc(455): error C2440: 'static_cast': cannot convert from '_Ty' to 'UT_SPLINE_BASIS' 
            with
            [
                _Ty=fpreal32
            ]
    C:\vcpkg\openvdb\openvdb_houdini\VRAY_OpenVDB_Points.cc(455): note: Conversions between enumeration and floating point values are no longer allowed

From googling, it appears that this is a Visual Studio restriction. I'm trying to work out a way to fix it but no success yet.

e4lam commented 4 years ago

@ianww VRAY_OpenVDB_Points.cc is for the renderer to render vdb points primitives. I think you can probable take that out too. I recall running into this error but can't seem to find what I did for this now. This might have as far as I've got in my VS2019 build as well.

To fix the error, we can force the compiler by first assigning into an int.

            int basis = rampVals[n];
            mFunctionRamp.addNode(rampVals[n-4],
                UT_FRGBA(rampVals[n-3], rampVals[n-2], rampVals[n-1], 1.0f),
                static_cast<UT_SPLINE_BASIS>(basis));

ianww commented 4 years ago

I made those edits, and...

Errors = zero!! Build succeeded!!

Fantastic - thanks @e4lam and @Idclip for all your help! That was a journey and a half!

I've only one final thing to do: build and install the python module.

To start with, I tried enabling the python module in the CMake build but I got quite a few syntax errors, which seems peculiar for well established code. Before I look into them any further, can I please first ask whether they might be because as part of my overcoming the earlier build errors I have forced my build to use Python 2.7 (rightly or wrongly - I can't recall my exact rationale for doing this but it seemed like a good idea at the time and 2.7 is the version recommended by openvdb https://www.openvdb.org/documentation/doxygen/dependencies.html).

Could the Python module in OpenVDB 6.2 require Python 3? Or is there some other explanation and solution?

FYI the python module build errors were:

Error   C2059   syntax error: '('   C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj c:\vcpkg\openvdb\openvdb\python\pyGrid.h    1843    
Error   C2059   syntax error: '('   C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj c:\vcpkg\openvdb\openvdb\python\pyGrid.h    1843    
Error   C2059   syntax error: '('   C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj c:\vcpkg\openvdb\openvdb\python\pyGrid.h    1843    
Error   C2589   '(': illegal token on right side of '::'    C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj C:\vcpkg\openvdb\openvdb\python\pyOpenVDBModule.cc  475 
Error   C2062   type 'unknown-type' unexpected  C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj C:\vcpkg\openvdb\openvdb\python\pyOpenVDBModule.cc  475 
Error   C2589   '(': illegal token on right side of '::'    C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj C:\vcpkg\openvdb\openvdb\python\pyOpenVDBModule.cc  476 
Error   C2059   syntax error: ')'   C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj C:\vcpkg\openvdb\openvdb\python\pyOpenVDBModule.cc  476 
Error   C2143   syntax error: missing ';' before '{'    C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj C:\vcpkg\openvdb\openvdb\python\pyOpenVDBModule.cc  477 
Error   C2181   illegal else without matching if    C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj C:\vcpkg\openvdb\openvdb\python\pyOpenVDBModule.cc  479 
Error   C2375   'initpyopenvdb': redefinition; different linkage    C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj C:\vcpkg\openvdb\openvdb\python\pyOpenVDBModule.cc  840 
Error   C2589   '(': illegal token on right side of '::'    C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj C:\vcpkg\openvdb\openvdb\python\pyOpenVDBModule.cc  978 
Error   C2062   type 'unknown-type' unexpected  C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj C:\vcpkg\openvdb\openvdb\python\pyOpenVDBModule.cc  978 
Error   C2059   syntax error: ')'   C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj C:\vcpkg\openvdb\openvdb\python\pyOpenVDBModule.cc  978 
Error   C2589   '(': illegal token on right side of '::'    C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj C:\vcpkg\openvdb\openvdb\python\pyOpenVDBModule.cc  979 
Error   C2062   type 'unknown-type' unexpected  C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj C:\vcpkg\openvdb\openvdb\python\pyOpenVDBModule.cc  979 
Error   C2059   syntax error: ')'   C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj C:\vcpkg\openvdb\openvdb\python\pyOpenVDBModule.cc  979 
Error   C2059   syntax error: '('   C:\vcpkg\openvdb\out\build\x64-Release\openvdb\python\pyopenvdb.vcxproj c:\vcpkg\openvdb\openvdb\python\pyGrid.h    1843

Idclip commented 4 years ago

@ianww I would suggest opening a different issue to track the python work as this has become pretty big.

To clarify (to my future self and the rest of the TSC), the remaining Houdini errors on Windows that need fixing are:

GR_PrimVDBPoints.cc: https://github.com/AcademySoftwareFoundation/openvdb/issues/603#issuecomment-578361888
SOP_OpenVDB_Visualize.cc : https://github.com/AcademySoftwareFoundation/openvdb/issues/603#issuecomment-578406885
VRAY_OpenVDB_Points.cc https://github.com/AcademySoftwareFoundation/openvdb/issues/603#issuecomment-578409135

@e4lam @ianww is this correct? This issue should remain open until these are resolved.

ianww commented 4 years ago

Yep @e4lam and @Idclip, that's correct.

ianww commented 4 years ago

After the successful CMake build (omiting the python module), I then opened the OpenVDB.sln in Visual Studio and ran Build INSTALL.

VS created 3 directories within C:/Users/Owner/Documents/houdini17.5 - a dso, an include and a lib directory.

Inside the dso directory, there are 50 or so dll files and their matching lib files (eg. SOP_OpenVBD_Scatter.dll and SOP_OpenVDB_Scatter.lib).

In my houdini.env file, I've set HOUDINI_DSO_PATH ="C:/Users/Owner/Documents/houdini17.5/dso;&"

I've confirmed this path by running hconfig -ap in Houdini's Shell.

When I run hscript>dsoinfo in Shell, I get a list of all the dso's that Houdini can see.

None of my dso's show up.

Is there something wrong with my approach or does it suggest that something is wrong with the compiles, causing Houdini to not recognise them?

e4lam commented 4 years ago

@ianww wrote:

In my houdini.env file, I've set HOUDINI_DSO_PATH ="C:/Users/Owner/Documents/houdini17.5/dso;&"

I've confirmed this path by running hconfig -ap in Houdini's Shell.

By Houdini's "Shell", you mean the Houdini "Command Line Tools" from the Start menu, right? I'd try starting that and then running set HOUDINI_DSO_ERROR=3 first to enable DSO debugging. Then run houdini > log.txt to see if it even tries to load your extra DSOs and if it encountered errors while doing so.

e4lam commented 4 years ago

FWIW, I took a crack at updating vcpkg to build the latest openvdb with python module enabled.

First, here's the patch for boost-python so that it builds against python2: vcpkg_ports_boost-python.patch.txt

And then here's the changes for openvdb: vcpkg_ports_openvdb.patch.txt

I'm not sure how you normally do things but I run cygwin on Windows so convert the following steps to whatever is the equivalent (Power Shell?) for you.

Apply both patches. cd to wherever you have vcpkg and then run patch -p1 < /PATH/TO/PATCH_FILE
First ensure that you remove the packages before installing them. cd to vcpkg and run ./vcpkg remove --triplet x64-windows boost-python openvdb
Now install openvdb: ./vcpkg --triplet x64-windows openvdb
Copy the created files into all the required places

e4lam commented 4 years ago

PS. I'm not sure why you were running into those pyGrid.h problems as they look like there were missing #includes or something otherwise seriously wrong.

e4lam commented 4 years ago

To clarify (to my future self and the rest of the TSC), the remaining Houdini errors on Windows that need fixing are:

GR_PrimVDBPoints.cc: #603 (comment)

SOP_OpenVDB_Visualize.cc : #603 (comment)

VRAY_OpenVDB_Points.cc #603 (comment)

@e4lam @ianww is this correct? This issue should remain open until these are resolved.

@Idclip This is correct except that the SOP_OpenVDB_Visualize.cc change was in #601 which is already merged.

ianww commented 4 years ago

First, here's the patch for boost-python so that it builds against python2: vcpkg_ports_boost-python.patch.txt

@e4lam I already had edited boost-python's CONTROL file by changing python3 to python2 in its Build-Depends as part of my earlier CMake build troubleshooting.

This is the python output I get from CMake as it's generating the cache:

1> [CMake] This warning is for project developers.  Use -Wno-dev to suppress it.
1> [CMake] -- ----------------------------------------------------
1> [CMake] -- ------------ Configuring OpenVDBPython -------------
1> [CMake] -- ----------------------------------------------------
1> [CMake] -- Found Python: C:/Program Files/Houdini17.5.425/python27/python.exe (found suitable version "2.7.15", minimum required is "2.7") found components:  Interpreter Development 
1> [CMake] CMake Warning at C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.15/Modules/FindBoost.cmake:1125 (message):
1> [CMake]   New Boost version may have incorrect or missing dependencies and imported
1> [CMake]   targets
1> [CMake] Call Stack (most recent call first):
1> [CMake]   C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.15/Modules/FindBoost.cmake:1247 (_Boost_COMPONENT_DEPENDENCIES)
1> [CMake]   C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.15/Modules/FindBoost.cmake:1885 (_Boost_MISSING_DEPENDENCIES)
1> [CMake]   C:/vcpkg/scripts/buildsystems/vcpkg.cmake:227 (_find_package)
1> [CMake]   openvdb/python/CMakeLists.txt:121 (find_package)
1> [CMake] -- Found boost_python27

Would I still need your patch 1?

e4lam commented 4 years ago

As requested, @Idclip, I've created 2 new issues that I encountered while building the openvdb vcpkg (#620 and #621) and commented on an old pre-existing issue that still persists (#527).

e4lam commented 4 years ago

@e4lam I already had edited boost-python's CONTROL file by changing python3 to python2 in its Build-Depends as part of my earlier CMake build troubleshooting.

@ianww That's not enough. You need to patch the python3 references in boost-python's portfile.cmake as well. The first patch I posted has changes in both CONTROL and portfile.cmake

e4lam commented 4 years ago

@ianww PS. I fixed my original comment about where to cd for applying the patches.

ianww commented 4 years ago

By Houdini's "Shell", you mean the Houdini "Command Line Tools" from the Start menu, right? I'd try starting that and then running set HOUDINI_DSO_ERROR=3 first to enable DSO debugging. Then run houdini > log.txt to see if it even tries to log your extra DSOs and if it encountered errors doing so.

@e4lam Houdini loaded all its own dso's from C:\Program Files\Houdini17.5.425\houdini\dso and then as its very last step tried:

DSO_INFO:C:/Users/Owner/Documents/houdini17.5/dso/SOP_OpenVDB_Write.dll:Attempt load

and then stopped.

SOP_OpenVDB_Write.dll is alphabetically the last of the dll's - it seems to be working backwards and fails at the first one, and it threw up this error:

Clicking this error away leads to the same error coming up for the next SOP, and so on all the way through them all.

Looking through my CMake files, there doesn't seem to be any explicit setting of Houdini's version to be the 17.5.425 that I'm using but I've presumed that these settings implicitly set the version:

HOUDINI_CMAKE_LOCATION=C:/Program Files/Houdini17.5.425/toolkit/cmake
Houdini_DIR=C:/Program Files/Houdini17.5.425

and the Cmake cache generation output does say:

Found Houdini: C:/Program Files/Houdini17.5.425 (found version "17.5.425")

But perhaps my presumption is wrong, and I should be doing an explicit set somewhere too?

I do have a Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE) that pops up several times in the cache generation output but after googling that I've been ignoring it. Maybe I do need that?

ianww commented 4 years ago

@e4lam

First, here's the patch for boost-python so that it builds against python2: vcpkg_ports_boost-python.patch.txt

I installed cygwin to do this (Powershell doesn't have an equivalent patch command).

While the first diff worked (replacing python3 with python2 in the Build-Depends line), the second diff failed due to different line endings.

In my original portfile.cmake file, the line reads:

file(GLOB PYTHON_INCLUDE_PATH "${CURRENT_INSTALLED_DIR}/include/python[0-9.]*")

but your patch is looking for:

file(GLOB PYTHON_INCLUDE_PATH "${CURRENT_INSTALLED_DIR}/include/python3.*")

I therefore made the change from python[0-9.] to python2. manually.

FYI also, my portfile.cmake file has 4 additional lines (the last 4 here) to what I think you have (they did not appear in your patch):

# Find Python. Can't use find_package here, but we already know where everything is
file(GLOB PYTHON_INCLUDE_PATH "${CURRENT_INSTALLED_DIR}/include/python2*")
set(PYTHONLIBS_RELEASE "${CURRENT_INSTALLED_DIR}/lib")
set(PYTHONLIBS_DEBUG "${CURRENT_INSTALLED_DIR}/debug/lib")
string(REGEX REPLACE ".*python([0-9\.]+)$" "\\1" PYTHON_VERSION "${PYTHON_INCLUDE_PATH}")
include(${CURRENT_INSTALLED_DIR}/share/boost-build/boost-modular-build.cmake)
boost_modular_build(SOURCE_PATH ${SOURCE_PATH})
include(${CURRENT_INSTALLED_DIR}/share/boost-vcpkg-helpers/boost-modular-headers.cmake)
boost_modular_headers(SOURCE_PATH ${SOURCE_PATH})

Not knowing otherwise, I've left those 4 additional lines in.

And then here's the changes for openvdb: vcpkg_ports_openvdb.patch.txt

This patch seemed to work fine.

Now install openvdb: ./vcpkg --triplet x64-windows openvdb

vcpkg installed 96 out of 97 packages OK.

It then gave a CMake error on package 97. I think this was because vcpkg was actually trying to run CMake as its final step. I've been using Visual Studio's CMake engine instead, so I don't think a vcpkg CMake error matters to me.

However, it may be of note that when I later tried as an experiment .\vcpkg remove --triplet x64-windows --recurse openvdb, it said Package openvdb:x64-windows is not installed.

FYI the error messages were:

Error message

``` Starting package 97/97: openvdb:x64-windows Building package openvdb[core]:x64-windows... -- Downloading https://github.com/AcademySoftwareFoundation/openvdb/archive/5496aab7af43d41196f6b5cdb4288da0d51d97d8.tar.gz... -- Extracting source C:/vcpkg/downloads/AcademySoftwareFoundation-openvdb-5496aab7af43d41196f6b5cdb4288da0d51d97d8.tar.gz -- Applying patch 0001-remove-pkgconfig.patch -- Applying patch 0002-fix-cmake-modules.patch -- Applying patch 0003-fix-cmake.patch -- Applying patch 0004-remove-python-interpreter.patch -- Applying patch 0005-fix-pyopenvdb.patch -- Using source at C:/vcpkg/buildtrees/openvdb/src/a0d51d97d8-5a94ba1850 -- Configuring x64-windows .\vcpkg : CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:72 (message): At line:1 char:1 + .\vcpkg install --triplet x64-windows boost-python openvdb + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (CMake Error at ...e:72 (message)::String) [], RemoteException + FullyQualifiedErrorId : NativeCommandError Command failed: ninja -v Working Directory: C:/vcpkg/buildtrees/openvdb/x64-windows-rel/vcpkg-parallel-configure Error code: 1 See logs for more information: C:\vcpkg\buildtrees\openvdb\config-x64-windows-out.log Call Stack (most recent call first): scripts/cmake/vcpkg_configure_cmake.cmake:293 (vcpkg_execute_required_process) ports/openvdb/portfile.cmake:33 (vcpkg_configure_cmake) scripts/ports.cmake:94 (include) Error: Building package openvdb:x64-windows failed with: BUILD_FAILED Please ensure you're using the latest portfiles with `.\vcpkg update`, then submit an issue at https://github.com/Microsoft/vcpkg/issues including: Package: openvdb:x64-windows Vcpkg version: 2019.09.12-nohash-external Additionally, attach any relevant sections from the log files above. PS C:\vcpkg> .\vcpkg update Using local portfile versions. To update the local portfiles, use `git pull`. No packages need updating. ```

After the package installations, I went back to my Visual Studio CMake project from before. The cache built OK straight off - none of my dependency paths required updating (ie. CMake could see in the new installation directory all the same files I'd previously specified), so it looks like I had managed previously to find all the right packages anyway . No doubt your patches would save a lot of path setting if I was starting from scratch though.

My CMake build (omitting the python module) completed successfully, as before. But also as before, after building the install using the OpenVDB.sln, Houdini can't load the resulting dso's.

I tried a CMake build with the python module enabled, and got the same errors as before. I've started another separate new issue #622 for that problem.

So, for the current issue, my remaining error to fix is to get Houdini to be able to load the openvdb dso's that I've created:

From googling I found a suggested fix for someone getting a similar error was to update their (in their case) 2015 C++ redistributable. I've done that for my Microsoft Visual C++ 2015-2019 Redistributable (x64) - 14.24.28127 but the load failure's still there.

e4lam commented 4 years ago

@iamww It probably can't find some of the DLL dependencies. Try adding the full path of the vcpkg/installed/x64-windows/bin as the LAST entry of your PATH environment variable. I'll need to give this another try when I'm next at my desk.

On Tue, Jan 28, 2020, 3:15 AM Ian Woodward, notifications@github.com wrote:

@e4lam https://github.com/e4lam

First, here's the patch for boost-python so that it builds against python2: vcpkg_ports_boost-python.patch.txt

I installed cygwin to do this (Powershell doesn't have a similar patch command).

While the first diff worked (replacing python3 with python2 in the Build-Depends line), the second diff failed due to different line endings.

In my original CONTROL file, the line reads:

file(GLOB PYTHON_INCLUDE_PATH "${CURRENT_INSTALLED_DIR}/include/python[0-9.]*")

but your patch is looking for:

file(GLOB PYTHON_INCLUDE_PATH "${CURRENT_INSTALLED_DIR}/include/python3.*")

I therefore made that change manually.

FYI also, my CONTROL file has 4 additional lines (the last 4 here) to what I think you have:

Find Python. Can't use find_package here, but we already know where everything is

file(GLOB PYTHON_INCLUDE_PATH "${CURRENT_INSTALLED_DIR}/include/python2") set(PYTHONLIBS_RELEASE "${CURRENT_INSTALLED_DIR}/lib") set(PYTHONLIBS_DEBUG "${CURRENT_INSTALLED_DIR}/debug/lib") string(REGEX REPLACE ".python([0-9.]+)$" "\1" PYTHON_VERSION "${PYTHON_INCLUDE_PATH}") include(${CURRENT_INSTALLED_DIR}/share/boost-build/boost-modular-build.cmake) boost_modular_build(SOURCE_PATH ${SOURCE_PATH}) include(${CURRENT_INSTALLED_DIR}/share/boost-vcpkg-helpers/boost-modular-headers.cmake) boost_modular_headers(SOURCE_PATH ${SOURCE_PATH})

Not knowing otherwise, I've left those 4 additional lines in.

And then here's the changes for openvdb: vcpkg_ports_openvdb.patch.txt

This patch worked fine.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/AcademySoftwareFoundation/openvdb/issues/603?email_source=notifications&email_token=AALXQHLHJZFW2P7TFOQ7AULQ77SR5A5CNFSM4KEG667KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKCND4A#issuecomment-579129840, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALXQHJN2TNJRAMKV27PCATQ77SR5ANCNFSM4KEG667A .

e4lam commented 4 years ago

In my original portfile.cmake file, the line reads:
file(GLOB PYTHON_INCLUDE_PATH "${CURRENT_INSTALLED_DIR}/include/python[0-9.]*")
but your patch is looking for:
file(GLOB PYTHON_INCLUDE_PATH "${CURRENT_INSTALLED_DIR}/include/python3.*")
I therefore made the change from python[0-9.] to python2. manually.

PS. I think this it because I'm working off of the latest vcpkg git code. If you run ./vcpkg update or git pull, then I think your "original" files here will change.

ianww commented 4 years ago

@e4lam @Idclip FYI - new issue created for python errors: #622

EDIT: This issue is now solved.

danrbailey commented 4 years ago

@ianww - I wanted to pick out one comment and add a clarification:

I'm necessarily working with version 6.2 for Houdini 17.5 compatibility reasons

I assume that you're back-porting changes from master to a 6.2 branch, but I wanted to make sure that you're aware that you can build 7.0 or the latest master for Houdini 17.5 providing that you use the correct ABI. The CMake build system should detect the Houdini install and use the appropriate ABI version for you, which for H17.5 is ABI=5.

ianww commented 4 years ago

Thanks @danrbailey - I'll give that a go later, once I've got my current build with 6.2 sorted. I'm so close....!

ianww commented 4 years ago

Going back to these dso errors:

I'm trying to work out if my compiler settings are wrong, and specifically whether they are incompatible with Houdini's own build.

My Houdini download was houdini-17.5.425-win64-vc141. That means that it is compatible with msvc14.1 (which is Visual Studio 2017):

https://www.openvdb.org/documentation/doxygen/build.html says:

When downloading Houdini, take note of the file version information. The version you install will determine the compiler version you should try and use to build OpenVDB. For example, for Linux Houdini 17.5, with a file name of houdini-17.5.219-linux_x86_64_gcc6.3.tar.gz, GCC 6.3 should be used.

In Visual Studio 2019, I therefore installed the MSVC v141 – VS 2017 C++ x64/x86 build tools.

My computer is 64 bit.

In my CMake settings, I selected Visual Studio 15 2017 Win64 as the CMake generator:

For the toolset setting I have choices of msvc_x86, msvc_x64, msvc_x64_x64 and msvc_x86_x64. These are explained here: https://software.intel.com/en-us/forums/intel-c-compiler/topic/806872.

When I open up Houdini's Command Line Tools terminal from the Windows start menu, it says:

which to me suggests that Houdini 17.5 was compiled with the 19.16 compiler for an msvc_x86_64 configuration.

I therefore set my CMake toolset to be msvc_x86_x64 and to match this I set my CMAKE_LINKER to be: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe.

The tool tip properties indicate that this compiler matches what was shown in the Command Line Tools terminal, ie. cl19.16:

When I generate my CMake cache, I get these output messages relevant to CXX:

 [CMake] -- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.18363.
1> [CMake] -- The CXX compiler identification is MSVC 19.16.27035.0
1> [CMake] -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe
1> [CMake] -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe -- works

which to me indicates that the compiler being used to do the build work is msvc19.16.27035.0, provided by msvc14.16 (Visual Studio 2017) for an x86_x64 compile, which matches expectations.

The output also confirms that CMake is building against Houdini 17.5.425:

1> [CMake] -- Found Houdini: C:/Program Files/Houdini17.5.425 (found version "17.5.425")

At the start of my CMake build, I get a message:

which says the build engine being used is 15.9.21, which is Visual Studio 2017, as specified.

My CMake build is successful. In Visual Studio 2019 I then open up the OpenVDB.sln:

Selecting INSTALL, I choose Project>Properties and confirm that the platform toolset is Visual Studio 2017 (v141):

In the Release>Configuration Manager, I make sure that all the openvdb projects are selected:

(Note: All the SOPS are selected, all the way down)

I choose Build>Build INSTALL. When it's finished I get a message Build All: 54 succeeded, 0 failed, 0 skipped.

In C:\Users\Owner\Documents\houdini17.5, 4 new directories are created: dso, include, lib and scripts.

dso contains 46 pairs of dll and lib files, eg:

include contains h files (by their dates, none of which were actually generated by the build or install):

lib contains a dll and lib file:

scripts contains a single cmd file:

When I open Houdini, with HOUDINI_DSO_ERROR=3, I get these errors messages for every one of the dso's:

In all the above, I can't see any incompatibilities (but I may be blinded to them).

Can anyone see anything I've done wrong?

If it's not a compiler issue, maybe it's a dependency issue, although my dependencies come from vcpkg openvdb (modified as per https://github.com/AcademySoftwareFoundation/openvdb/issues/603#issuecomment-579069357).

As a final piece of information, here's my json file, showing the dependencies I've linked to:

json file

``` "configurations": [ { "name": "x64-Release", "generator": "Visual Studio 15 2017 Win64", "configurationType": "Release", "buildRoot": "${projectDir}\\out\\build\\${name}", "installRoot": "C:/vcpkg/openvdb/out/install/x64-Release", "cmakeCommandArgs": "", "buildCommandArgs": "", "ctestCommandArgs": "", "inheritEnvironments": [ "msvc_x86_x64" ], "variables": [ { "name": "Houdini_DIR", "value": "C:/Program Files/Houdini17.5.425", "type": "PATH" }, { "name": "IlmBase_Half_DLL", "value": "C:/Program Files/Houdini17.5.425/bin/Half.dll", "type": "FILEPATH" }, { "name": "IlmBase_Half_LIBRARY", "value": "C:/Program Files/Houdini17.5.425/custom/houdini/dsolib/Half.lib", "type": "FILEPATH" }, { "name": "IlmBase_Iex_DLL", "value": "C:/Program Files/Houdini17.5.425/bin/Iex_sidefx.dll", "type": "FILEPATH" }, { "name": "IlmBase_Iex_LIBRARY", "value": "C:/Program Files/Houdini17.5.425/custom/houdini/dsolib/Iex_sidefx.lib", "type": "FILEPATH" }, { "name": "IlmBase_IexMath_DLL", "value": "C:/Program Files/Houdini17.5.425/bin/IexMath_sidefx.dll", "type": "FILEPATH" }, { "name": "IlmBase_IexMath_LIBRARY", "value": "C:/Program Files/Houdini17.5.425/custom/houdini/dsolib/IexMath_sidefx.lib", "type": "FILEPATH" }, { "name": "IlmBase_IlmThread_DLL", "value": "C:/Program Files/Houdini17.5.425/bin/IlmThread_sidefx.dll", "type": "FILEPATH" }, { "name": "IlmBase_IlmThread_LIBRARY", "value": "C:/Program Files/Houdini17.5.425/custom/houdini/dsolib/IlmThread_sidefx.lib", "type": "FILEPATH" }, { "name": "IlmBase_Imath_DLL", "value": "C:/Program Files/Houdini17.5.425/bin/Imath_sidefx.dll", "type": "FILEPATH" }, { "name": "IlmBase_Imath_LIBRARY", "value": "C:/Program Files/Houdini17.5.425/custom/houdini/dsolib/Imath_sidefx.lib", "type": "FILEPATH" }, { "name": "ZLIB_LIBRARY", "value": "C:/Program Files/Houdini17.5.425/custom/houdini/dsolib/zdll.lib", "type": "FILEPATH" }, { "name": "OpenEXR_IlmImf_DLL", "value": "C:/Program Files/Houdini17.5.425/bin/IlmImf_sidefx.dll", "type": "FILEPATH" }, { "name": "IlmBase_INCLUDE_DIR", "value": "C:/Program Files/Houdini17.5.425/toolkit/include/OpenEXR", "type": "PATH" }, { "name": "OpenEXR_IlmImf_LIBRARY", "value": "C:/Program Files/Houdini17.5.425/custom/houdini/dsolib/IlmImf_sidefx.lib", "type": "FILEPATH" }, { "name": "OpenEXR_IlmImfUtil_DLL", "value": "C:/Program Files/Houdini17.5.425/bin/IlmImfUtil_sidefx.dll", "type": "FILEPATH" }, { "name": "OpenEXR_IlmImfUtil_LIBRARY", "value": "C:/Program Files/Houdini17.5.425/custom/houdini/dsolib/IlmImfUtil_sidefx.lib", "type": "FILEPATH" }, { "name": "OpenEXR_INCLUDE_DIR", "value": "C:/Program Files/Houdini17.5.425/toolkit/include/OpenEXR", "type": "PATH" }, { "name": "Tbb_tbb_LIBRARY", "value": "C:/Program Files/Houdini17.5.425/custom/houdini/dsolib/tbb.lib", "type": "FILEPATH" }, { "name": "Blosc_LIBRARY", "value": "C:/Program Files/Houdini17.5.425/custom/houdini/dsolib/blosc.lib", "type": "FILEPATH" }, { "name": "Blosc_INCLUDE_DIR", "value": "C:/Program Files/Houdini17.5.425/toolkit/include", "type": "PATH" }, { "name": "Boost_INCLUDE_DIR", "value": "C:/vcpkg/installed/x64-windows/include", "type": "PATH" }, { "name": "Boost_IOSTREAMS_LIBRARY_RELEASE", "value": "C:/vcpkg/installed/x64-windows/lib/boost_iostreams-vc140-mt.lib", "type": "FILEPATH" }, { "name": "Boost_SYSTEM_LIBRARY_RELEASE", "value": "C:/vcpkg/installed/x64-windows/lib/boost_system-vc140-mt.lib", "type": "FILEPATH" }, { "name": "Tbb_tbbmalloc_LIBRARY", "value": "C:/Program Files/Houdini17.5.425/custom/houdini/dsolib/tbbmalloc.lib", "type": "FILEPATH" }, { "name": "Tbb_INCLUDE_DIR", "value": "C:/Program Files/Houdini17.5.425/toolkit/include", "type": "PATH" }, { "name": "Tbb_tbbmalloc_proxy_LIBRARY", "value": "C:/Program Files/Houdini17.5.425/custom/houdini/dsolib/tbbmalloc_proxy.lib", "type": "FILEPATH" }, { "name": "USE_DEFAULT_HOUDINI_INSTALL", "value": "True", "type": "BOOL" }, { "name": "Boost_IOSTREAMS_LIBRARY_DEBUG", "value": "Boost_IOSTREAMS_LIBRARY_DEBUG-NOTFOUND", "type": "FILEPATH" }, { "name": "Boost_LIBRARY_DIR_RELEASE", "value": "C:/vcpkg/installed/x64-windows/lib", "type": "PATH" }, { "name": "Boost_REGEX_LIBRARY_RELEASE", "value": "C:/vcpkg/installed/x64-windows/lib/boost_regex-vc140-mt.lib", "type": "FILEPATH" }, { "name": "HOUDINI_CMAKE_LOCATION", "value": "C:/Program Files/Houdini17.5.425/toolkit/cmake", "type": "PATH" }, { "name": "OPENVDB_CORE_SHARED", "value": "True", "type": "BOOL" }, { "name": "OPENVDB_CORE_STATIC", "value": "True", "type": "BOOL" }, { "name": "ZLIB_INCLUDE_DIR", "value": "C:/Program Files/Houdini17.5.425/toolkit/include", "type": "PATH" }, { "name": "USE_SYSTEM_LIBRARY_PATHS", "value": "True", "type": "BOOL" }, { "name": "OPENVDB_DISABLE_BOOST_IMPLICIT_LINKING", "value": "True", "type": "BOOL" }, { "name": "OPENVDB_BUILD_BINARIES", "value": "True", "type": "BOOL" }, { "name": "OPENVDB_BUILD_CORE", "value": "True", "type": "BOOL" }, { "name": "OPENVDB_BUILD_HOUDINI_PLUGIN", "value": "True", "type": "BOOL" }, { "name": "CMAKE_LINKER", "value": "C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe", "type": "FILEPATH" }, { "name": "CMAKE_CXX_STANDARD_LIBRARIES", "value": "kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib", "type": "STRING" }, { "name": "OPENVDB_BUILD_PYTHON_MODULE", "value": "True", "type": "BOOL" }, { "name": "OPENVDB_INSTALL_HOUDINI_PYTHONRC", "value": "False", "type": "BOOL" }, { "name": "OPENVDB_PYTHON_WRAP_ALL_GRID_TYPES", "value": "False", "type": "BOOL" }, { "name": "OPENVDB_HOUDINI_PYTHONRC_DIR", "value": "C:/Users/Owner/Documents/houdini17.5/python2.7libs", "type": "PATH" }, { "name": "Python_EXECUTABLE", "value": "C:/Program Files/Houdini17.5.425/python27/python.exe", "type": "FILEPATH" }, { "name": "Python_INCLUDE_DIR", "value": "C:/Program Files/Houdini17.5.425/python27/include", "type": "PATH" }, { "name": "Python_LIBRARY_RELEASE", "value": "C:/Program Files/Houdini17.5.425/python27/libs/python27.lib", "type": "FILEPATH" }, { "name": "OPENVDB_BUILD_VDB_PRINT", "value": "False", "type": "BOOL" }, { "name": "OPENVDB_ENABLE_3_ABI_COMPATIBLE", "value": "False", "type": "BOOL" } ], "intelliSenseMode": "windows-msvc-x64" } ] } ```

e4lam commented 4 years ago

@ianww The toolset naming scheme is kinda confusing. Houdini build for "x86_64" basically just means Intel 64-bit (as opposed to ARM 64-bit, etc.). When you pick the toolset in cmake it should ideally be msvc_x64_x64 (use 64-bit compiler to build 64-bit binaries) instead of msvc_x86_x64 (use 32-bit compiler to build 64-bit binaries). It works but for large files, the 32-bit compiler might have problems.

As for not being able to load the DLL's, did you try my suggestion of modifying PATH before starting Houdini?

ianww commented 4 years ago

@e4lam I had previously tried msvc_x64_x64, with the same problem. I'll give it another go though, in case other settings have changed since then. Should I pick the matching CMAKE_LINKER

C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\Hostx64\x64\cl.exe

or stick with

C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe?

Yes, my Windows PATH is:

FYI also, in my Houdini.env file, my HOUDINI_DSO_PATH is:

HOUDINI_DSO_PATH = "C:/Users/Owner/Documents/houdini17.5/dso;C:\Users\Owner\Documents\houdini17.5\lib;C:\Users\Owner\Documents\houdini17.5\include;C:\Users\Owner\Documents\houdini17.5\scripts;&"

e4lam commented 4 years ago

@ianww For HOUDINI_DSO_PATH, you should only have directories which contain DLL's which you want to load into Houdini. So the include and scripts paths in there do nothing.

I would expect that adding c:\vcpkg\installed\x64-windows\bin to your PATH allow Houdini to load the dependencies that the openvdb plugins were built against. After you modified he PATH, did you start Houdini from the Start menu? Note that if you start Houdini from a shell, that won't work until it is restarted. If you're starting Houdini from the Start menu and it doesn't work despite the modified PATH, then I think the next step is to load houdini.exe in http://www.dependencywalker.com/.

Download the 64-bit version of Dependency Walker from http://www.dependencywalker.com/ .. Make sure you download the 64-bit version.
Run Dependency Walker from Windows Explorer and do File > Open > path to houdini.exe
It might have some initial errors about system dll's, you can just ignore these.
From the main menu, choose Profile > Start Profiling > use default settings > OK
This should launch Houdini and when the DLL loading error dialog shows up, look at the log in Dependency Walker to see what type of errors occurred when it tried to load it.

ianww commented 4 years ago

openvdb.dll is copied in (with its lib).

Hmm, I don't seem to have any .dll.manifest files. Where should they be?

e4lam commented 4 years ago

@ianww Sorry, I deleted my other comments, the manifest files should already be embedded. I was looking in vcpkg/buildtrees/openvdb/x64-windows-rel/openvdb_houdini

e4lam commented 4 years ago

Hmm, for some reason when I try this, I need to manually copy vcpkg/installed/x64-windows/bin/Half-2_3.dll into somewhere in the PATH for Houdini to be able to load the plugins. eg. into the same directory where houdini.exe exists.

e4lam commented 4 years ago

Of course, I built everything with VS 2019 and Houdini currently refuses to load any DLL built with VS 2019. But at least I got to the stage where it explicitly said "Incompatible compiler versions".

e4lam commented 4 years ago

PS. Also try copying openvdb.dll (from vcpkg/installed/x64-windows/bin) to where houdini.exe exists

ianww commented 4 years ago

Here's the log for the first dso load attempt error (SOP_OpenVDB_Write):

(BTW - SOP_OpenVDB_Write.dll is definitely inside C:\Users\Owner\Documents\houdini17.5\dso)

I'm not sure if its relevant but not-found errors are also occurring in some CHOPs, which aren't part of my build, eg:

Re your .dll.manifest comment, in my buildtrees for openvdb I don't actually have an openvdb_houdini subdirectory, only an openvdb:

I copied vcpkg/installed/x64-windows/bin/Half-2_3.dll in as you suggested but no success.

I don't have a vcpkg/installed/x64-windows/bin/openvdb.dll existing - this would be due to the vcpkg CMake error on the last of 97 packages - see https://github.com/AcademySoftwareFoundation/openvdb/issues/603#issuecomment-579129840 :

Now install openvdb: ./vcpkg --triplet x64-windows openvdb

vcpkg installed 96 out of 97 packages OK.

It then gave a CMake error on package 97. I think this was because vcpkg was actually trying to run CMake as its final step. I've been using Visual Studio's CMake engine instead, so I don't think a vcpkg CMake error matters to me.

However, it may be of note that when I later tried as an experiment .\vcpkg remove --triplet x64-windows --recurse openvdb, it said Package openvdb:x64-windows is not installed.

Maybe we need to go back to resolve that vcpkg error?

Here's the config-x64-windows-out.log for that error

``` [1/2] cmd /c "cd .. && "C:/Program Files/CMake/bin/cmake.exe" "C:/vcpkg/buildtrees/openvdb/src/a0d51d97d8-5a94ba1850" "-DOPENVDB_BUILD_UNITTESTS=OFF" "-DOPENVDB_BUILD_PYTHON_MODULE=ON" "-DOPENVDB_ENABLE_3_ABI_COMPATIBLE=OFF" "-DUSE_GLFW3=ON" "-DOPENVDB_CORE_STATIC=OFF" "-DOPENVDB_CORE_SHARED=ON" "-DOPENVDB_BUILD_VDB_PRINT=" "-DOPENVDB_BUILD_VDB_VIEW=" "-DOPENVDB_BUILD_VDB_LOD=" "-DCMAKE_MAKE_PROGRAM=C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/Ninja/ninja.exe" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=C:/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v142" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-G" "Ninja" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_INSTALL_PREFIX=C:/vcpkg/packages/openvdb_x64-windows"" FAILED: ../CMakeCache.txt cmd /c "cd .. && "C:/Program Files/CMake/bin/cmake.exe" "C:/vcpkg/buildtrees/openvdb/src/a0d51d97d8-5a94ba1850" "-DOPENVDB_BUILD_UNITTESTS=OFF" "-DOPENVDB_BUILD_PYTHON_MODULE=ON" "-DOPENVDB_ENABLE_3_ABI_COMPATIBLE=OFF" "-DUSE_GLFW3=ON" "-DOPENVDB_CORE_STATIC=OFF" "-DOPENVDB_CORE_SHARED=ON" "-DOPENVDB_BUILD_VDB_PRINT=" "-DOPENVDB_BUILD_VDB_VIEW=" "-DOPENVDB_BUILD_VDB_LOD=" "-DCMAKE_MAKE_PROGRAM=C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/Ninja/ninja.exe" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=C:/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v142" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-G" "Ninja" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_INSTALL_PREFIX=C:/vcpkg/packages/openvdb_x64-windows"" -- The C compiler identification is MSVC 19.24.28315.0 -- The CXX compiler identification is MSVC 19.24.28315.0 -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.24.28314/bin/Hostx64/x64/cl.exe -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.24.28314/bin/Hostx64/x64/cl.exe -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.24.28314/bin/Hostx64/x64/cl.exe -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.24.28314/bin/Hostx64/x64/cl.exe -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- CMake Build Type: Release -- Suppressing some noisy MSVC CXX warnings, set OPENVDB_CXX_STRICT=ON to re-enable them. -- Configuring for OpenVDB Version 7.0.1 -- Configuring for OpenVDB ABI Version 7 -- ---------------------------------------------------- -- ------------- Configuring OpenVDBCore -------------- -- ---------------------------------------------------- -- Found ILMBase: C:/vcpkg/installed/x64-windows -- Found ZLIB: optimized;C:/vcpkg/installed/x64-windows/lib/zlib.lib;debug;C:/vcpkg/installed/x64-windows/debug/lib/zlibd.lib (found suitable version "1.2.11", minimum required is "1.2.7") -- Found Blosc: C:/vcpkg/installed/x64-windows (Required is at least version "1.5") -- Found Boost: C:/vcpkg/installed/x64-windows/include (found suitable version "1.71.0", minimum required is "1.61") found components: iostreams system regex -- ---------------------------------------------------- -- ------------ Configuring OpenVDBPython ------------- -- ---------------------------------------------------- CMake Error at openvdb/python/CMakeLists.txt:174 (message): Unable to find boost_python38, boost_python3 or boost_python. -- Configuring incomplete, errors occurred! See also "C:/vcpkg/buildtrees/openvdb/x64-windows-rel/CMakeFiles/CMakeOutput.log". [2/2] cmd /c "cd ../../x64-windows-dbg && "C:/Program Files/CMake/bin/cmake.exe" "C:/vcpkg/buildtrees/openvdb/src/a0d51d97d8-5a94ba1850" "-DOPENVDB_BUILD_UNITTESTS=OFF" "-DOPENVDB_BUILD_PYTHON_MODULE=ON" "-DOPENVDB_ENABLE_3_ABI_COMPATIBLE=OFF" "-DUSE_GLFW3=ON" "-DOPENVDB_CORE_STATIC=OFF" "-DOPENVDB_CORE_SHARED=ON" "-DOPENVDB_BUILD_VDB_PRINT=" "-DOPENVDB_BUILD_VDB_VIEW=" "-DOPENVDB_BUILD_VDB_LOD=" "-DCMAKE_MAKE_PROGRAM=C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/Ninja/ninja.exe" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=C:/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v142" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-G" "Ninja" "-DCMAKE_BUILD_TYPE=Debug" "-DCMAKE_INSTALL_PREFIX=C:/vcpkg/packages/openvdb_x64-windows/debug"" FAILED: ../../x64-windows-dbg/CMakeCache.txt cmd /c "cd ../../x64-windows-dbg && "C:/Program Files/CMake/bin/cmake.exe" "C:/vcpkg/buildtrees/openvdb/src/a0d51d97d8-5a94ba1850" "-DOPENVDB_BUILD_UNITTESTS=OFF" "-DOPENVDB_BUILD_PYTHON_MODULE=ON" "-DOPENVDB_ENABLE_3_ABI_COMPATIBLE=OFF" "-DUSE_GLFW3=ON" "-DOPENVDB_CORE_STATIC=OFF" "-DOPENVDB_CORE_SHARED=ON" "-DOPENVDB_BUILD_VDB_PRINT=" "-DOPENVDB_BUILD_VDB_VIEW=" "-DOPENVDB_BUILD_VDB_LOD=" "-DCMAKE_MAKE_PROGRAM=C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/Ninja/ninja.exe" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=C:/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v142" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-G" "Ninja" "-DCMAKE_BUILD_TYPE=Debug" "-DCMAKE_INSTALL_PREFIX=C:/vcpkg/packages/openvdb_x64-windows/debug"" -- The C compiler identification is MSVC 19.24.28315.0 -- The CXX compiler identification is MSVC 19.24.28315.0 -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.24.28314/bin/Hostx64/x64/cl.exe -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.24.28314/bin/Hostx64/x64/cl.exe -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.24.28314/bin/Hostx64/x64/cl.exe -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.24.28314/bin/Hostx64/x64/cl.exe -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- CMake Build Type: Debug -- Suppressing some noisy MSVC CXX warnings, set OPENVDB_CXX_STRICT=ON to re-enable them. -- Configuring for OpenVDB Version 7.0.1 -- Configuring for OpenVDB ABI Version 7 -- ---------------------------------------------------- -- ------------- Configuring OpenVDBCore -------------- -- ---------------------------------------------------- -- Found ILMBase: C:/vcpkg/installed/x64-windows -- Found ZLIB: optimized;C:/vcpkg/installed/x64-windows/lib/zlib.lib;debug;C:/vcpkg/installed/x64-windows/debug/lib/zlibd.lib (found suitable version "1.2.11", minimum required is "1.2.7") -- Found Blosc: C:/vcpkg/installed/x64-windows (Required is at least version "1.5") -- Found Boost: C:/vcpkg/installed/x64-windows/include (found suitable version "1.71.0", minimum required is "1.61") found components: iostreams system regex -- ---------------------------------------------------- -- ------------ Configuring OpenVDBPython ------------- -- ---------------------------------------------------- CMake Error at openvdb/python/CMakeLists.txt:174 (message): Unable to find boost_python38, boost_python3 or boost_python. -- Configuring incomplete, errors occurred! See also "C:/vcpkg/buildtrees/openvdb/x64-windows-dbg/CMakeFiles/CMakeOutput.log". ninja: build stopped: subcommand failed. ```

BTW - I noticed in this error log that it was being configured for OpenVDB ABI Version 7, but my configuration is for ABI 5, which is what Houdini 17.5 requires https://www.openvdb.org/documentation/doxygen/build.html

e4lam commented 4 years ago

@ianww When I find some time again, I'll try to come up with a more complete vcpkg patch that builds both the python and houdini plugins. From your errors, you're not actually getting to the part where it is building anything of openvdb. Offhand, this is because it's not properly finding vcpkg's python but some other python 3.8 installation that you have. So two things:

You should ONLY build the release openvdb_houdini package because debug won't work. To do this, I added set(VCPKG_BUILD_TYPE release) to my vcpkg/triplets/x64-windows.cmake (or I guess you can probably create a new one called x64-windows-rel.cmake with this line added and use that triplet instead).
Try to force the python detection by adding this to the cmake command in ports/openvdb/portfile.cmake : -DPython_ROOT_DIR=... where ... is some appropriate directory in vcpkg.

ianww commented 4 years ago

Should there be a python.exe in my vcpkg directory? I can't see one.

e4lam commented 4 years ago

@ianww No, my suspicion is that it's picking it up from your PATH when we do NOT want it to.

ianww commented 4 years ago

I'm confused about:

is some appropriate directory in vcpkg

How do I determine what directory in vcpkg Python_ROOT_DIR should point to?

BTW - I've realised that I have python 3.6, 3.7 and 3.8 installed. I'll uninstall 3.7 and 3.8 - I note from the pybin11 portfile.cmake that it looks for python3 and expects 3.6, so the presence of 3.7 and 3.8 might confuse it.

BTW also - I've just realised that when I ran vcpkg install with your openvdb patches, I forgot to then run .\vcpkg integrate install. That would mean the build was looking at an old CMAKE_TOOLCHAIN_FILE - could this be a problem?

e4lam commented 4 years ago

@Idclip FYI, an update on some changes for tomorrow's Houdini 18.0.365 for building openvdb_houdini on Windows:

Fix missing SYS_FORCE_INLINE for GEO_PrimVDB::getGridPtr()
One issue I found when trying to build openvdb_houdini on Windows is that the /Zc:externC- compiler option needs to be AFTER the ones configured by the HDK's HoudiniConfig.cmake. This is a good option to add to HoudiniConfig.cmake when using VS 2019 anyways because of boost so we now check and add this option inside HoudiniConfig.cmake itself.

e4lam commented 4 years ago

@ianww I don't know which appropriate directory either, some experimentation will be needed. From https://cmake.org/cmake/help/v3.15/module/FindPython.html it just says:

Python_ROOT_DIR
Define the root directory of a Python installation.

You don't need to run vcpkg integrate, I've haven't done that at all in my testing yet.

e4lam commented 4 years ago

As of Houdni 18.0.365, it will also accept plugins on Windows compiled with VS 2019.

ianww commented 4 years ago

@e4lam For when you're revising your vcpkg patch -

Looking at error logs, vcpkg's openvdb is trying to build for:

-- Configuring for OpenVDB Version 7.0.1
-- Configuring for OpenVDB ABI Version 7

but for my Houdini 17.5 I'm needing it to be:

-- Configuring for OpenVDB Version 6.2
-- Configuring for OpenVDB ABI Version 5

The way I understand vcpkg to work is that it always uses the latest version of a package, which would be 7.0.1 for openvdb, and a git command would be required force it to use an earlier version (the 6.2 I need) https://github.com/microsoft/vcpkg/issues/1681.

And then, once I had the 6.2 package, its portfile.cmake might be different to that for 7.0.1, which might mean your patches wouldn't work correctly?

As of Houdni 18.0.365, it will also accept plugins on Windows compiled with VS 2019.

I've got a bit lost in versions now and what will and won't work with what!

Does this mean that I could now forget about Houdini 17.5, and start a build from scratch with 18.0.365, without having to remake all the edits to Houdini's files that I've had to do for 17.5?

danrbailey commented 4 years ago

@ianww - as I mentioned earlier, for Houdini 17.5 it would need to be:

-- Configuring for OpenVDB Version 7.0.1
-- Configuring for OpenVDB ABI Version 5

and for Houdini 18.0 it would need to be:

-- Configuring for OpenVDB Version 7.0.1
-- Configuring for OpenVDB ABI Version 6

For Houdini, the ABI is important, not the version. You could build any version from 5.0.0 => 7.0.1 for Houdini 17.5 provided the ABI was set to 5.

When building the OpenVDB core library yourself but with OPENVDB_BUILD_HOUDINI_PLUGIN=ON, the ABI configuration is handled here:

https://github.com/AcademySoftwareFoundation/openvdb/blob/master/cmake/OpenVDBHoudiniSetup.cmake#L323

e4lam commented 4 years ago

The way I understand vcpkg to work is that it always uses the latest version of a package, which would be 7.0.1 for openvdb, and a git command would be required force it to use an earlier version (the 6.2 I need) https://github.com/microsoft/vcpkg/issues/1681.

And then, once I had the 6.2 package, its portfile.cmake might be different to that for 7.0.1, which might mean your patches wouldn't work correctly?

@ianww Not quite. vcpkg will use whichever commit is specified in portfile.cmake. In my patches above, it makes portfile.cmake use latest code in OpenVDB. But yes, the commit used here determines the patching that is required. In other words, the patches depend on the commit that is specified in portfile.cmake.

Does this mean that I could now forget about Houdini 17.5, and start a build from scratch with 18.0.365, without having to remake all the edits to Houdini's files that I've had to do for 17.5?

That is correct. Perhaps I missed this in the discussion above, but what is the reason for requiring Houdini 17.5 instead of 18.0?

AcademySoftwareFoundation / openvdb

Houdini in Windows 10 #603

Find Python. Can't use find_package here, but we already know where everything is