search

AcademySoftwareFoundation / openvdb

OpenVDB - Sparse volume data structure and tools
http://www.openvdb.org/
Mozilla Public License 2.0
2.64k stars 653 forks source link

blosc error with points vdb #256

Closed samhodge closed 5 years ago

samhodge commented 6 years ago

Getting error about buffer exceeds BLOSC_MAX_BUFFERSIZE see:

https://github.com/dreamworksanimation/openvdb/blob/a7a1abde7c955d5a83acac2c82341497d73576c2/openvdb/points/StreamCompression.cc#L69

I am behind a firewall or I would upload the vdb in question.

It is a points vdb, which we are rendering with our own neat little Arnold Points VDB Procedural in HtoA in Arnold.

Houdini gives the same error:

Blosc decompress failed due to exceeding maximum buffer size.

Is this a known issue, what is the work around?

Renders are failing, people are wailing.

danrbailey commented 6 years ago

Hi Sam,

This sounds like a bug worth investigating. There is an easy work-around - to disable blosc, with the obvious drawback that files will be larger. You can either re-compile OpenVDB without OPENVDB_USE_BLOSC or if you're writing these files from Houdini to set the environment variable HOUDINI13_VOLUME_COMPATIBILITY to 1.

Now a few quick questions:

Which version of OpenVDB are you using (or a Git SHAid if you're between versions)? Which version of Blosc are you compiling OpenVDB against? (note that Houdini native uses 1.5.0 http://www.sidefx.com/docs/houdini/licenses/index) Where (and how) are you authoring these VDBs? Is this generated from a Houdini Geometry ROP? Are all the unit tests running and passing? We have a fairly comprehensive set of unit tests to verify file I/O.

If it's possible to get my hands on a reproducible test case, I'd like to have a look, however note that it may only be possible to track this down on writing the data not on read which can complicate things. I'll mention that this I/O has been used extensively and hasn't changed in well over a year now, so I'd be a little suspicious that isn't a bug if this was happening for every single cache instead of for a few particular use cases. For example, the last reported issue was a very specific number of points in a leaf that caused the buffer size for that leaf to fall within a very narrow range that exposed a bug in the I/O.

I hope I can help though.

samhodge commented 6 years ago

00:00:00 69MB | Arnold 5.1.0.1 [8698598b] linux clang-5.0.0 oiio-1.7.17 osl-1.9.0 vdb-4.0.0 clm-1.0.3.513 rlm-12.2.2 2018/04/17 17:37:03

that give you the OpenVDB version.

LDD tells you a little more

/rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.0.1/lib/rsp_vdbPointsArnold.so:
        linux-vdso.so.1 =>  (0x00007fffe1912000)
        /asset/common/software/thirdparty/gcc-runtime/4.8.5-build4/arch/linux-centos6/x86_64/lib64/libstdc++.so (0x00007f036c905000)
        libImath-2_2.so.12 => /asset/common/software/thirdparty/ilmbase/2.2.0-build4/arch/linux-centos6_2/x86_64/gcc4.8/lib/libImath-2_2.so.12 (0x00007f036c6e7000)
        libHalf.so.12 => /asset/common/software/thirdparty/houdini/16.5.439-build2/arch/linux-any/x86_64/dsolib/libHalf.so.12 (0x00007f036c4a5000)
        libIex-2_2.so.12 => /asset/common/software/thirdparty/ilmbase/2.2.0-build4/arch/linux-centos6_2/x86_64/gcc4.8/lib/libIex-2_2.so.12 (0x00007f036c27b000)
        libIexMath-2_2.so.12 => /asset/common/software/thirdparty/ilmbase/2.2.0-build4/arch/linux-centos6_2/x86_64/gcc4.8/lib/libIexMath-2_2.so.12 (0x00007f036c077000)
        libIlmThread-2_2.so.12 => /asset/common/software/thirdparty/ilmbase/2.2.0-build4/arch/linux-centos6_2/x86_64/gcc4.8/lib/libIlmThread-2_2.so.12 (0x00007f036be6d000)
        libz.so.1 => /asset/common/software/thirdparty/houdini/16.5.439-build2/arch/linux-any/x86_64/python/lib/libz.so.1 (0x00007f036bc52000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f036ba4a000)
        libai.so => /asset/common/software/thirdparty/htoa/3.0.1-build2/arch/linux-centos6_2/x86_64/scripts/bin/libai.so (0x00007f03680ba000)
        libopenvdb.so.4.0 => /asset/common/software/thirdparty/openvdb/4.0.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/ndebug/lib/libopenvdb.so.4.0 (0x00007f0367b7e000)
        libblosc.so.1 => /asset/common/software/thirdparty/houdini/16.5.439-build2/arch/linux-any/x86_64/dsolib/libblosc.so.1 (0x00007f036796c000)
        libtbb.so.2 => /asset/common/software/thirdparty/tbb/3.0.20100915-build1/arch/linux-centos5_2/x86_64/lib/libtbb.so.2 (0x00007f0367741000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f03674bd000)
        libgomp.so.1 => /asset/common/software/thirdparty/gcc-runtime/4.8.5-build4/arch/linux-centos6/x86_64/lib64/libgomp.so.1 (0x00007f03672af000)
        libgcc_s.so.1 => /asset/common/software/thirdparty/gcc-runtime/4.8.5-build4/arch/linux-centos6/x86_64/lib64/libgcc_s.so.1 (0x00007f0367099000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f0366e7c000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f0366ae8000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f036ceb6000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f03668e4000)
        liboptix.so.1 => /asset/common/software/thirdparty/htoa/3.0.1-build2/arch/linux-centos6_2/x86_64/scripts/bin/liboptix.so.1 (0x00007f0363be6000)
        libAdClmHub.so => /asset/common/software/thirdparty/htoa/3.0.1-build2/arch/linux-centos6_2/x86_64/scripts/bin/libAdClmHub.so (0x00007f03638c9000)
        libboost_iostreams-mt.so.1.61.0 => /asset/common/software/thirdparty/boost/1.61.0-build3/arch/linux-centos6_2/x86_64/gcc4.8/python2.7/ucs4/ndebug/lib/libboost_iostreams-mt.so.1.61.0 (0x00007f03636b2000)
        libboost_system-mt.so.1.61.0 => /asset/common/software/thirdparty/boost/1.61.0-build3/arch/linux-centos6_2/x86_64/gcc4.8/python2.7/ucs4/ndebug/lib/libboost_system-mt.so.1.61.0 (0x00007f03634af000)
        libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f036329e000)

or a slightly different version

 samh@bladerunner ~/dev/vdbPointsArnold/ run in htoa/latest with vdbPointsArnold/0.18.0-WORKING : kick error.ass --bubbles-command /usr/bin/ldd /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.1/lib/rsp_vdbPointsArnold.so
[bubbles] WARNING: Using unpublished vdbPointsArnold/0.18.0-WORKING
error.ass:
ldd: warning: you do not have execution permission for `./error.ass'
        not a dynamic executable
/rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.1/lib/rsp_vdbPointsArnold.so:
        linux-vdso.so.1 =>  (0x00007fa6dc471000)
        /asset/common/software/thirdparty/gcc-runtime/4.8.5-build4/arch/linux-centos6/x86_64/lib64/libstdc++.so (0x00007fa6dbca2000)
        libImath-2_2.so.12 => /asset/common/software/thirdparty/ilmbase/2.2.0-build4/arch/linux-centos6_2/x86_64/gcc4.8/lib/libImath-2_2.so.12 (0x00007fa6dba84000)
        libHalf.so.12 => /asset/common/software/thirdparty/houdini/16.5.439-build2/arch/linux-any/x86_64/dsolib/libHalf.so.12 (0x00007fa6db842000)
        libIex-2_2.so.12 => /asset/common/software/thirdparty/ilmbase/2.2.0-build4/arch/linux-centos6_2/x86_64/gcc4.8/lib/libIex-2_2.so.12 (0x00007fa6db618000)
        libIexMath-2_2.so.12 => /asset/common/software/thirdparty/ilmbase/2.2.0-build4/arch/linux-centos6_2/x86_64/gcc4.8/lib/libIexMath-2_2.so.12 (0x00007fa6db414000)
        libIlmThread-2_2.so.12 => /asset/common/software/thirdparty/ilmbase/2.2.0-build4/arch/linux-centos6_2/x86_64/gcc4.8/lib/libIlmThread-2_2.so.12 (0x00007fa6db20a000)
        libz.so.1 => /asset/common/software/thirdparty/houdini/16.5.439-build2/arch/linux-any/x86_64/python/lib/libz.so.1 (0x00007fa6dafef000)
        librt.so.1 => /lib64/librt.so.1 (0x00007fa6dade7000)
        libai.so => /asset/common/software/thirdparty/htoa/3.0.3-build1/arch/linux-centos6_2/x86_64/scripts/bin/libai.so (0x00007fa6d743a000)
        libopenvdb.so.4.0 => /asset/common/software/thirdparty/openvdb/4.0.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/ndebug/lib/libopenvdb.so.4.0 (0x00007fa6d6efe000)
        libblosc.so.1 => /asset/common/software/thirdparty/houdini/16.5.439-build2/arch/linux-any/x86_64/dsolib/libblosc.so.1 (0x00007fa6d6cec000)
        libtbb.so.2 => /asset/common/software/thirdparty/tbb/3.0.20100915-build1/arch/linux-centos5_2/x86_64/lib/libtbb.so.2 (0x00007fa6d6ac1000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fa6d683d000)
        libgomp.so.1 => /asset/common/software/thirdparty/gcc-runtime/4.8.5-build4/arch/linux-centos6/x86_64/lib64/libgomp.so.1 (0x00007fa6d662f000)
        libgcc_s.so.1 => /asset/common/software/thirdparty/gcc-runtime/4.8.5-build4/arch/linux-centos6/x86_64/lib64/libgcc_s.so.1 (0x00007fa6d6419000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa6d61fc000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fa6d5e68000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fa6dc253000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fa6d5c64000)
        libAdClmHub.so => /asset/common/software/thirdparty/htoa/3.0.3-build1/arch/linux-centos6_2/x86_64/scripts/bin/libAdClmHub.so (0x00007fa6d5947000)
        libboost_iostreams-mt.so.1.61.0 => /asset/common/software/thirdparty/boost/1.61.0-build3/arch/linux-centos6_2/x86_64/gcc4.8/python2.7/ucs4/ndebug/lib/libboost_iostreams-mt.so.1.61.0 (0x00007fa6d5730000)
        libboost_system-mt.so.1.61.0 => /asset/common/software/thirdparty/boost/1.61.0-build3/arch/linux-centos6_2/x86_64/gcc4.8/python2.7/ucs4/ndebug/lib/libboost_system-mt.so.1.61.0 (0x00007fa6d552d000)
        libbz2.so.1 => /lib64/libbz2.so.1 (0x00007fa6d531c000)
samhodge commented 6 years ago

I will see if i can get the data to you now.

samhodge commented 6 years ago

angry.vdb.zip Here is a zip of the angry file

samhodge commented 6 years ago

strings make me think that the blosc versions is 1.0.3 but SideFX would know better.

samhodge commented 6 years ago

The point VDBs were authored from a SOP ROP output driver in Houdini a bunch of points are converted from points using convert vdb points in Houdini 16.5.439.

They wont read back into Houdini at this point. They also SIGSEGV the procedural which I have LDD'ed above.

samhodge commented 6 years ago

here is a stacktrace from gdb

{noformat} (gdb) bt

0 0x00007fffdbe4cac8 in blosc_d.isra.3 () from /asset/common/software/thirdparty/houdini/16.5.439-build2/arch/linux-any/x86_64/dsolib/libblosc.so.1

1 0x00007fffdbe4e3b9 in do_job () from /asset/common/software/thirdparty/houdini/16.5.439-build2/arch/linux-any/x86_64/dsolib/libblosc.so.1

2 0x00007fffdbe4e6a1 in blosc_run_decompression_with_context () from /asset/common/software/thirdparty/houdini/16.5.439-build2/arch/linux-any/x86_64/dsolib/libblosc.so.1

3 0x00007fffdbe4e7f1 in blosc_decompress_ctx () from /asset/common/software/thirdparty/houdini/16.5.439-build2/arch/linux-any/x86_64/dsolib/libblosc.so.1

4 0x00007fffdc314220 in openvdb::v4_0_0::compression::bloscDecompress(char, unsigned long, unsigned long, char const) () from /asset/common/software/thirdparty/openvdb/4.0.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/ndebug/lib/libopenvdb.so.4.0

5 0x00007fffdc315888 in openvdb::v4_0_0::compression::Page::doLoad() const () from /asset/common/software/thirdparty/openvdb/4.0.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/ndebug/lib/libopenvdb.so.4.0

6 0x00007fffdc315a3c in openvdb::v4_0_0::compression::Page::buffer(int) const () from /asset/common/software/thirdparty/openvdb/4.0.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/ndebug/lib/libopenvdb.so.4.0

7 0x00007fffdc315a76 in openvdb::v4_0_0::compression::PageHandle::read() () from /asset/common/software/thirdparty/openvdb/4.0.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/ndebug/lib/libopenvdb.so.4.0

8 0x00007fffdc237b8f in openvdb::v4_0_0::points::TypedAttributeArray<openvdb::v4_0_0::math::Vec3, openvdb::v4_0_0::points::NullCodec>::loadData() const () from /asset/common/software/thirdparty/openvdb/4.0.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/ndebug/lib/libopenvdb.so.4.0

9 0x00007fffdd28bf73 in openvdb::v4_0_0::points::AttributeHandle<openvdb::v4_0_0::math::Vec3, openvdb::v4_0_0::points::UnknownCodec>::AttributeHandle(openvdb::v4_0_0::points::AttributeArray const&, bool) () from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.0/lib/rsp_vdbPointsArnold.so

10 0x00007fffdd27afbe in (anonymous namespace)::writePoints(ProcArgs&)::{lambda(openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u> const&, unsigned long)#5}::operator()(openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u> const&, unsigned long) const () from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.0/lib/rsp_vdbPointsArnold.so

11 0x00007fffdd27b745 in openvdb::v4_0_0::tree::LeafManager<openvdb::v4_0_0::tree::Tree<openvdb::v4_0_0::tree::RootNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafTransformer<(anonymous namespace)::writePoints(ProcArgs&)::{lambda(openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u> const&, unsigned long)#5}>::operator()(openvdb::v4_0_0::tree::LeafManager<openvdb::v4_0_0::tree::Tree<openvdb::v4_0_0::tree::RootNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafRange const&) const () from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.0/lib/rsp_vdbPointsArnold.so

12 0x00007fffdd27ba8d in void tbb::interface9::internal::balancing_partition_type<tbb::interface9::internal::adaptive_mode >::work_balance<tbb::interface9::internal::start_for<openvdb::v4_0_0::tree::LeafManager<openvdb::v4_0_0::tree::Tree<openvdb::v4_0_0::tree::RootNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafRange, openvdb::v4_0_0::tree::LeafManager<openvdb::v4_0_0::tree::Tree<openvdb::v4_0_0::tree::RootNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafTransformer<(anonymous namespace)::writePoints(ProcArgs&)::{lambda(openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u> const&, unsigned long)#5}>, tbb::auto_partitioner const>, openvdb::v4_0_0::tree::LeafManager<openvdb::v4_0_0::tree::Tree<openvdb::v4_0_0::tree::RootNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafRange>(tbb::interface9::internal::start_for<openvdb::v4_0_0::tree::LeafManager<openvdb::v4_0_0::tree::Tree<openvdb::v4_0_0::tree::RootNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafRange, openvdb::v4_0_0::tree::LeafManager<openvdb::v4_0_0::tree::Tree<openvdb::v4_0_0::tree::RootNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafTransformer<(anonymous namespace)::writePoints(ProcArgs&)::{lambda(openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u> const&, unsigned long)#5}>, tbb::auto_partitioner const>&, openvdb::v4_0_0::tree::LeafManager<openvdb::v4_0_0::tree::Tree<openvdb::v4_0_0::tree::RootNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafRange&) () from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.0/lib/rsp_vdbPointsArnold.so

13 0x00007fffdd27bccf in tbb::interface9::internal::start_for<openvdb::v4_0_0::tree::LeafManager<openvdb::v4_0_0::tree::Tree<openvdb::v4_0_0::tree::RootNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafRange, openvdb::v4_0_0::tree::LeafManager<openvdb::v4_0_0::tree::Tree<openvdb::v4_0_0::tree::RootNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::tree::InternalNode<openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafTransformer<(anonymous namespace)::writePoints(ProcArgs&)::{lambda(openvdb::v4_0_0::points::PointDataLeafNode<openvdb::v4_0_0::PointIndex<unsigned int, 1u>, 3u> const&, unsigned long)#5}>, tbb::auto_partitioner const>::execute() () from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.0/lib/rsp_vdbPointsArnold.so

14 0x00007fffdbc3be01 in tbb::internal::custom_scheduler::local_wait_for_all(tbb::task&, tbb::task*) () from /asset/common/software/thirdparty/tbb/3.0.20100915-build1/arch/linux-centos5_2/x86_64/lib/libtbb.so.2

15 0x00007fffdbc3a869 in tbb::internal::generic_scheduler::local_spawn_root_and_wait(tbb::task&, tbb::task*&) () from /asset/common/software/thirdparty/tbb/3.0.20100915-build1/arch/linux-centos5_2/x86_64/lib/libtbb.so.2

16 0x00007fffdd27ec82 in (anonymous namespace)::writePoints(ProcArgs&) () from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.0/lib/rsp_vdbPointsArnold.so

17 0x00007fffdd28054b in ProceduralInit(AtNode*, void**) () from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.0/lib/rsp_vdbPointsArnold.so

18 0x00007ffff4af4f15 in ?? () from /asset/common/software/thirdparty/htoa/3.0.1-build2/arch/linux-centos6_2/x86_64/scripts/bin/libai.so

19 0x00007ffff4b5beff in ?? () from /asset/common/software/thirdparty/htoa/3.0.1-build2/arch/linux-centos6_2/x86_64/scripts/bin/libai.so

20 0x00007ffff4bf123e in ?? () from /asset/common/software/thirdparty/htoa/3.0.1-build2/arch/linux-centos6_2/x86_64/scripts/bin/libai.so

21 0x0000003033207aa1 in start_thread () from /lib64/libpthread.so.0

22 0x0000003032ae8bcd in clone () from /lib64/libc.so.6

{noformat}

jmlait commented 6 years ago

From Houdini 16.5 Blosc's header:

/ Version numbers /

define BLOSC_VERSION_MAJOR 1 /* for major interface/format changes

*/

define BLOSC_VERSION_MINOR 5 /* for minor interface/format changes

*/

define BLOSC_VERSION_RELEASE 0 /* for tweaks, bug-fixes, or

development */

define BLOSC_VERSION_STRING "1.5.0" /* string version. Sync with

above! */

define BLOSC_VERSION_REVISION "$Rev$" / revision version /

define BLOSC_VERSION_DATE "$Date:: 2014-11-07 #$" / date version /

define BLOSCLZ_VERSION_STRING "1.0.3" /* the internal compressor version

*/

So I presume the 1.0.3 you see from strings is the internal compressor version.

The linked to line:

void bloscCompress(char compressedBuffer, size_t& compressedBytes, const size_t bufferBytes, const char uncompressedBuffer, const size_t uncompressedBytes) { if (bufferBytes > BLOSC_MAX_BUFFERSIZE) {

is a bit worrying. BLOSC_MAX_BUFFERSIZE is INT_MAX - BLOSC_MAX_OVERHEAD)

So unless the bufferBytes is around 2 gigabytes, it shouldn't trigger. The other likely cause is if bufferBytes somehow computed negative, so the size_t cast sent it well above 2Gb. Both these seem unlikely as the paged array is supposed to keep this around 1mb, provided I read the code properly.

In any case, the error seems to be on writing? The offending geometry that is triggering it is likely needed. It sounds like there may be a simple .hip + .bgeo that could generate the angry.vdb?

On Mon, Aug 6, 2018 at 8:14 PM Sam Hodge notifications@github.com wrote:

strings make me think that the blosc versions is 1.0.3 but SideFX would know better.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/dreamworksanimation/openvdb/issues/256#issuecomment-410892263, or mute the thread https://github.com/notifications/unsubscribe-auth/ABd4L6W010x4mIWf8jcTA01F_h3iDB3-ks5uONvygaJpZM4Vv-bJ .

samhodge commented 6 years ago

I tried compiling my procedural against blosc 1.5.0 but that made no difference.

As for the .hip + .bgeo that made angry.vdb I will see if I can get that for you.

samhodge commented 6 years ago

angry_hip_bgeo_vdb.zip As suggested there are reproduction steps if you repay from /mint/scratch/tmp to where you unzipped to you should see the pattern with the two geometry sop networks under /obj

samhodge commented 6 years ago

b8ae38f3-8eb1-4298-acb2-c2663bd6fde0 82528142-af95-43e7-be1c-754c3751f7b4 Screen shot from the attached hip for Houdini 16.5.439 on Linux

jmlait commented 6 years ago

Thank you,

I managed to reproduce the malformed .vdb using 16.5. Interestingly, using 17.0 it produced a valid .vdb (which also loaded in 16.5).

The big difference is that 17.0 is using VDB 5.1.0, while 16.5 is still at VDB 4.0.0. This means fixes from a year ago may not be in 16.5.

Dan, do you know of any patches we should check are present in our 4.0.0 fork to deal with any known bugs?

Thanks,

On Mon, Aug 6, 2018 at 10:06 PM Sam Hodge notifications@github.com wrote:

[image: b8ae38f3-8eb1-4298-acb2-c2663bd6fde0] https://user-images.githubusercontent.com/86946/43750107-fc5f1bec-9a35-11e8-808b-85ddd701acbf.png [image: 82528142-af95-43e7-be1c-754c3751f7b4] https://user-images.githubusercontent.com/86946/43750108-fca02cea-9a35-11e8-88fb-136d3dd84d49.png Screen shot from the attached hip for Houdini 16.5.439 on Linux

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dreamworksanimation/openvdb/issues/256#issuecomment-410909136, or mute the thread https://github.com/notifications/unsubscribe-auth/ABd4L_MZepEoCMtPr_ROue-ErLVsO0TGks5uOPYhgaJpZM4Vv-bJ .

samhodge commented 6 years ago

Sweet thanks for being generous with your time @jmlait

Hopefully we can patch our way out of this issue.

It seems that it only occurs on some machines so we can play roulette and requeue on the rare frames, but it would be nice to have a fix.

danrbailey commented 6 years ago

Thanks for testing that @jmlait - I believe you might be missing the fix from this PR - https://github.com/dreamworksanimation/openvdb/pull/127

That was merged in-between 4.0.0 and 4.0.1 so would appear to line up with this issue. However, it looks like Edward picked this fix up for SESI's branch of 3.3.0, so (without testing this), it may even be that this issue only occurs in H16.5 and not H16.0 or H17.0.

@samhodge - from the stacktrace and the version of the OpenVDB library you're picking up, it appears that you're building the same version of OpenVDB that's shipped with Houdini (which I assume is intentional). If that is the case, just to mention that you can actually build a different version of OpenVDB from the one Houdini is shipped with because the two libraries are namespaced differently. So without needing to pick up a fix from SideFX, you could also just build the latest version of OpenVDB (5.1.0) and use ABI=4 and you'll also get this fix as the native output driver in Houdini will then pick up this installed version.

samhodge commented 6 years ago

@danrbailey

It seems that Solid Angle Arnold also has OpenVDB 4.0.0 scribed into it.

see:

Arnold 5.1.0.1 [8698598b] linux clang-5.0.0 oiio-1.7.17 osl-1.9.0 vdb-4.0.0 clm-1.0.3.513 rlm-12.2.2 2018/04/17 17:37:03

At once stage I did build OpenVDB 4.0.1 but the results were not good, maybe my build was awry, but if you say the SESI symbols are namespaced, then maybe the Solid Angle ones are too and I should have a go at getting OpenVDB 5.1.0 built and see if I can render that angry.vdb regardless.

More later after some testing.

samhodge commented 6 years ago

OK I had some good and bad luck.

The build systems at our workplace seem to ignore -DCMAKE_INSTALL_PATH for unknown reasons.

That being said OpenVDB 5.1.0 ABI=4 built just fine.

But I took the vdb_view tool from the less than accurate install location and I was able to open angry.vdb without issue.

So for my next trick is to get the install to put it in the right place and statically or dynamically link our procedural to it and see if I can render against the angry.vdb file.

If this is the case and I can get the runtime environment to supply the correct shared object(s) to the procedural we should be out of hot water.

jmlait commented 6 years ago

Yes, I can confirm that PR - 127 does not appear to be in 16.5. I can also confirm that fix is in our 3.3 branch. Sadly it isn't just a header change so it will take a bit more work to verify this fixes the problem. But it certainly looks promising! On Mon, Aug 6, 2018 at 11:55 PM Dan Bailey notifications@github.com wrote:

Thanks for testing that @jmlait - I believe you might be missing the fix from this PR - #127

That was merged in-between 4.0.0 and 4.0.1 so would appear to line up with this issue. However, it looks like Edward picked this fix up for SESI's branch of 3.3.0, so (without testing this), it may even be that this issue only occurs in H16.5 and not H16.0 or H17.0.

@samhodge - from the stacktrace and the version of the OpenVDB library you're picking up, it appears that you're building the same version of OpenVDB that's shipped with Houdini (which I assume is intentional). If that is the case, just to mention that you can actually build a different version of OpenVDB from the one Houdini is shipped with because the two libraries are namespaced differently. So without needing to pick up a fix from SideFX, you could also just build the latest version of OpenVDB (5.1.0) and use ABI=4 and you'll also get this fix as the native output driver in Houdini will then pick up this installed version.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

danrbailey commented 6 years ago

Ok, good to know, thanks! @samhodge - from what Jeff has said, you should be able to just use H16.0 to write out these caches as a temporary work around as well (although I appreciate that may be complicated if you're relying on features introduced in H16.5).

jmlait commented 6 years ago

I applied the patch to our dev16.5 branch of OpenVDB and the file then saved successfully from my 16.5 build.

I've applied this patch, so the next successful nightly build should work for you: 16.5.557 or greater.

On Tue, Aug 7, 2018 at 1:31 PM Dan Bailey notifications@github.com wrote:

Ok, good to know, thanks! @samhodge https://github.com/samhodge - from what Jeff has said, you should be able to just use H16.0 to write out these caches as a temporary work around as well (although I appreciate that may be complicated if you're relying on features introduced in H16.5).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dreamworksanimation/openvdb/issues/256#issuecomment-411138103, or mute the thread https://github.com/notifications/unsubscribe-auth/ABd4L0kBE0evSTbMgry8bO2ldB2JLRDaks5uOc7rgaJpZM4Vv-bJ .

danrbailey commented 6 years ago

Thanks Jeff!

dokipen3d commented 6 years ago

Good to hear jeff. We were having some blosc issues here at Weta the other day (not myself but a colleague in my office). We'll test when we get 557 installed.

samhodge commented 6 years ago

@danrbailey I tried your idea.

Compiled OpenVDB 5.1.0 from the tar ball

Compile my Arnold procedural against that with the -DOPENVDB_ABI_VERSION_NUMBER=4 as g++ flags.

blosc is statically linked from 1.7.2

here is the backtrace from GDB

(gdb) bt
#0  0x00007fffdc4051a2 in blosc_d.isra.4 () from /asset/common/software/thirdparty/openvdb/5.1.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/debug/lib/libopenvdb.so.5.1
#1  0x00007fffdc40689c in do_job () from /asset/common/software/thirdparty/openvdb/5.1.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/debug/lib/libopenvdb.so.5.1
#2  0x00007fffdc406ef1 in blosc_run_decompression_with_context () from /asset/common/software/thirdparty/openvdb/5.1.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/debug/lib/libopenvdb.so.5.1
#3  0x00007fffdc4073b1 in blosc_decompress_ctx () from /asset/common/software/thirdparty/openvdb/5.1.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/debug/lib/libopenvdb.so.5.1
#4  0x00007fffdc2228de in openvdb::v5_1abi4::compression::bloscDecompress(char*, unsigned long, unsigned long, char const*) ()
   from /asset/common/software/thirdparty/openvdb/5.1.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/debug/lib/libopenvdb.so.5.1
#5  0x00007fffdc2233d0 in openvdb::v5_1abi4::compression::Page::decompress(std::unique_ptr<char [], std::default_delete<char []> > const&) ()
   from /asset/common/software/thirdparty/openvdb/5.1.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/debug/lib/libopenvdb.so.5.1
#6  0x00007fffdc22363a in openvdb::v5_1abi4::compression::Page::doLoad() const () from /asset/common/software/thirdparty/openvdb/5.1.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/debug/lib/libopenvdb.so.5.1
#7  0x00007fffdc222dd4 in openvdb::v5_1abi4::compression::Page::load() const () from /asset/common/software/thirdparty/openvdb/5.1.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/debug/lib/libopenvdb.so.5.1
#8  0x00007fffdc222e4f in openvdb::v5_1abi4::compression::Page::buffer(int) const () from /asset/common/software/thirdparty/openvdb/5.1.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/debug/lib/libopenvdb.so.5.1
#9  0x00007fffdc223839 in openvdb::v5_1abi4::compression::PageHandle::read() () from /asset/common/software/thirdparty/openvdb/5.1.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/debug/lib/libopenvdb.so.5.1
#10 0x00007fffdc27da0f in openvdb::v5_1abi4::points::TypedAttributeArray<openvdb::v5_1abi4::math::Vec3<float>, openvdb::v5_1abi4::points::NullCodec>::doLoadUnsafe(bool) const ()
   from /asset/common/software/thirdparty/openvdb/5.1.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/debug/lib/libopenvdb.so.5.1
#11 0x00007fffdc27dc3a in openvdb::v5_1abi4::points::TypedAttributeArray<openvdb::v5_1abi4::math::Vec3<float>, openvdb::v5_1abi4::points::NullCodec>::doLoad() const ()
   from /asset/common/software/thirdparty/openvdb/5.1.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/debug/lib/libopenvdb.so.5.1
#12 0x00007fffdc261054 in openvdb::v5_1abi4::points::TypedAttributeArray<openvdb::v5_1abi4::math::Vec3<float>, openvdb::v5_1abi4::points::NullCodec>::loadData() const ()
   from /asset/common/software/thirdparty/openvdb/5.1.0-build1/arch/linux-centos6_2/x86_64/gcc4.8/ucs4/debug/lib/libopenvdb.so.5.1
#13 0x00007fffdd292c03 in openvdb::v5_1abi4::points::AttributeHandle<openvdb::v5_1abi4::math::Vec3<float>, openvdb::v5_1abi4::points::UnknownCodec>::AttributeHandle(openvdb::v5_1abi4::points::AttributeArray const&, bool) ()
   from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.1/lib/rsp_vdbPointsArnold.so
#14 0x00007fffdd286106 in (anonymous namespace)::writePoints(ProcArgs&)::{lambda(openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u> const&, unsigned long)#5}::operator()(openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u> const&, unsigned long) const ()
   from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.1/lib/rsp_vdbPointsArnold.so
#15 0x00007fffdd286605 in openvdb::v5_1abi4::tree::LeafManager<openvdb::v5_1abi4::tree::Tree<openvdb::v5_1abi4::tree::RootNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafTransformer<(anonymous namespace)::writePoints(ProcArgs&)::{lambda(openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u> const&, unsigned long)#5}>::operator()(openvdb::v5_1abi4::tree::LeafManager<openvdb::v5_1abi4::tree::Tree<openvdb::v5_1abi4::tree::RootNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafRange const&) const ()
   from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.1/lib/rsp_vdbPointsArnold.so
#16 0x00007fffdd28694d in void tbb::interface9::internal::balancing_partition_type<tbb::interface9::internal::adaptive_mode<tbb::interface9::internal::auto_partition_type> >::work_balance<tbb::interface9::internal::start_for<openvdb::v5_1abi4::tree::LeafManager<openvdb::v5_1abi4::tree::Tree<openvdb::v5_1abi4::tree::RootNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafRange, openvdb::v5_1abi4::tree::LeafManager<openvdb::v5_1abi4::tree::Tree<openvdb::v5_1abi4::tree::RootNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafTransformer<(anonymous namespace)::writePoints(ProcArgs&)::{lambda(openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u> const&, unsigned long)#5}>, tbb::auto_partitioner const>, openvdb::v5_1abi4::tree::LeafManager<openvdb::v5_1abi4::tree::Tree<openvdb::v5_1abi4::tree::RootNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafRange>(tbb::interface9::internal::start_for<openvdb::v5_1abi4::tree::LeafManager<openvdb::v5_1abi4::tree::Tree<openvdb::v5_1abi4::tree::RootNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafRange, openvdb::v5_1abi4::tree::LeafManager<openvdb::v5_1abi4::tree::Tree<openvdb::v5_1abi4::tree::RootNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafTransformer<(anonymous namespace)::writePoints(ProcArgs&)::{lambda(openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u> const&, unsigned long)#5}>, tbb::auto_partitioner const>&, openvdb::v5_1abi4::tree::LeafManager<openvdb::v5_1abi4::tree::Tree<openvdb::v5_1abi4::tree::RootNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafRange&) ()
   from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.1/lib/rsp_vdbPointsArnold.so
#17 0x00007fffdd286b8f in tbb::interface9::internal::start_for<openvdb::v5_1abi4::tree::LeafManager<openvdb::v5_1abi4::tree::Tree<openvdb::v5_1abi4::tree::RootNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafRange, openvdb::v5_1abi4::tree::LeafManager<openvdb::v5_1abi4::tree::Tree<openvdb::v5_1abi4::tree::RootNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::tree::InternalNode<openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u>, 4u>, 5u> > > const>::LeafTransformer<(anonymous namespace)::writePoints(ProcArgs&)::{lambda(openvdb::v5_1abi4::points::PointDataLeafNode<openvdb::v5_1abi4::PointIndex<unsigned int, 1u>, 3u> const&, unsigned long)#5}>, tbb::auto_partitioner const>::execute() ()
   from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.1/lib/rsp_vdbPointsArnold.so
#18 0x00007fffdb8d899a in tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::local_wait_for_all(tbb::task&, tbb::task*) ()
    at /rsp/tmp/weekly/kevinc/octobuild/tbb/source-4.4.20160526/tbb44_20160526oss/src/tbb/custom_scheduler.h:469
#19 0x00007fffdb8d67b0 in tbb::internal::generic_scheduler::local_spawn_root_and_wait(tbb::task&, tbb::task*&) () at /rsp/tmp/weekly/kevinc/octobuild/tbb/source-4.4.20160526/tbb44_20160526oss/src/tbb/scheduler.cpp:649
#20 0x00007fffdd288cdc in (anonymous namespace)::writePoints(ProcArgs&) () from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.1/lib/rsp_vdbPointsArnold.so
#21 0x00007fffdd28a4c1 in ProceduralInit(AtNode*, void**) () from /rsp/tmp/weekly/samh/projectDeploy/packages/vdbPointsArnold/0.18.0-WORKING/arch/linux-centos6/x86_64/gcc4.8/arnold5.1.1/lib/rsp_vdbPointsArnold.so
#22 0x00007ffff4e29235 in ?? () from /asset/common/software/thirdparty/arnoldSDK/5.1.1.0-build1/arch/linux-any/x86_64/bin/libai.so
#23 0x00007ffff4e8faef in ?? () from /asset/common/software/thirdparty/arnoldSDK/5.1.1.0-build1/arch/linux-any/x86_64/bin/libai.so
#24 0x00007ffff4f24fee in ?? () from /asset/common/software/thirdparty/arnoldSDK/5.1.1.0-build1/arch/linux-any/x86_64/bin/libai.so
#25 0x0000003033207aa1 in start_thread () from /lib64/libpthread.so.0
#26 0x0000003032ae8bcd in clone () from /lib64/libc.so.6

I will have a look at Jeff's overnight build next.

samhodge commented 6 years ago

the vdb tools can interrogate it just fine

 samh@bladerunner ~/dev/vdbPointsArnold/ run -x in openvdb/latest : vdb_print /mnt/scratch/tmp/angry.vdb -l
[bubbles] WARNING: Using unpublished glfw/3.2.0-build3
VDB version: 4.0/224
creator: Houdini 16.5.439/GEO_VDBTranslator

Name: points
Information about Tree:
  Type: Tree_ptdataidx32_5_4_3
  Configuration:
    Root(1 x 2), Internal(2 x 32^3), Internal(4 x 16^3), Leaf(28 x 8^3)
  Background value: 0
  Min value: 13
  Max value: 91
  Number of active voxels:       91
  Number of active tiles:        0
  Bounding box of active voxels: [27, -12, -139] -> [94, 5, -77]
  Dimensions of active voxels:   68 x 18 x 63
  Percentage of active voxels:   0.118%
  Average leaf node fill ratio:  0.635%
  Number of unallocated nodes:   0 (0%)
Memory footprint:
  Actual:                1.364 MB
  Active leaf voxels:      364 Bytes
  Dense equivalent:    301.219 KB
  Actual footprint is 464% of an equivalent dense volume
  Leaf voxel footprint is 0.0255% of actual footprint
Additional metadata:
  class: unknown
  file_bbox_max: [94, 5, -77]
  file_bbox_min: [27, -12, -139]
  file_compression: blosc + active values
  file_mem_bytes: 1505780
  file_voxel_count: 91
  is_local_space: false
  is_saved_as_half_float: false
  name: points
  value_type: ptdataidx32
  vector_type: invariant
Transform:
  voxel size: 16.7
  index to world:
     [16.7, 0, 0, 0] 
     [0, 16.7, 0, 0] 
     [0, 0, 16.7, 0] 
     [0, 0, 0, 1]
samhodge commented 6 years ago

@danrbailey

I can see the issue

https://github.com/dreamworksanimation/openvdb/pull/127

Related to compression which will fix the issue on the conversion of a .bgeo file to a vdb points file.

but it doesnt help out with the decompression crash which is still present with the files that have already been cached out with the error.

So are we left with the only possibility of picking up the fresh build from SESI and caching out the sims again and leaving the Arnold points procedural untouched?

samhodge commented 6 years ago

Last daily build I can see is https://www.sidefx.com/download/download-houdini/54386/ which is 554 so I guess we wait for 557.

But I can see the note here:

https://www.sidefx.com/changelog/

samhodge commented 6 years ago

Cool pulling down Houdni 16.5.557 now I will see if that is a good solution.

samhodge commented 6 years ago

Houdini 16.5.557 solved the issue yesterday. But we will need to recache.

jmlait commented 6 years ago

Excellent news that it solves the problem!

But very sad news you have to re-cache :<

I'm sorry we missed this patch in 16.5. Thank you for raising this issue so we could rectify the oversight.

On Thu, Aug 9, 2018 at 4:43 PM Sam Hodge notifications@github.com wrote:

Houdini 16.5.557 solved the issue yesterday. But we will need to recache.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dreamworksanimation/openvdb/issues/256#issuecomment-411890663, or mute the thread https://github.com/notifications/unsubscribe-auth/ABd4Lx6vBYCxBVUldok7FzcJhWojXUNaks5uPJ8MgaJpZM4Vv-bJ .

jmlait commented 5 years ago

I believe this is now closed because the missing #127 pull request was added to 16.5.