Maya AbcImport Plugin crashes: memory corruption

[GoogleCodeExporter]

What steps will reproduce the problem?
The problem seems to occur somewhat randomly, but happens when using AbcImport; 
I've tried to find a series of steps that will reproduce it, but so far I 
haven't found a pattern

What is the expected output? What do you see instead?
In the shell I see:
 *** glibc detected *** /usr/autodesk/maya2011.04-x64/bin/maya.bin: malloc(): memory corruption: 0x000000000ed93920 ***
Maya seems to hang until I kill it.

What version of the product are you using? On what operating system?
Maya2011 on Fedora 14

Please provide any additional information below.

Original issue reported on code.google.com by jspatr...@gmail.com on 24 May 2011 at 8:37

[GoogleCodeExporter]

Found a way to reproduce the crash.
Execute the following in Maya:

def _test(geo=None, range=None, newFile=False, export=True, reImport=True, 
          makeDeform=True, makeDups=True):

    import maya.cmds as mc
    import os
    mc.loadPlugin('AbcImport', 'AbcExport')

    if newFile:
        mc.file(new=1, force=1)

    if not geo:
        s1= mc.polySphere(name='s1')[0]
        s2=mc.polySphere(name='s2')[0]
        g = mc.group(em=True)
        mc.parent(s1, g)
        mc.parent(s2, g)

        if makeDeform:
            cn, ch = mc.cluster(s1)
            mc.setKeyframe(ch, attribute='sx', t=[1], v=.1)
            mc.setKeyframe(ch, attribute='sx', t=[10], v=1.5)

        #make a test attribtue
        sShape= mc.listRelatives(s1,shapes=1)[0]
        mc.addAttr(sShape, longName='LUMA_testAttr')
        mc.setKeyframe(sShape, attribute='LUMA_testAttr', t=[1], v=10)
        mc.setKeyframe(sShape, attribute='LUMA_testAttr', t=[10], v=18.125)

        mc.currentTime(1, e=1)
        mc.setKeyframe(s1)
        mc.setKeyframe(s2)
        mc.setKeyframe(g)

        mc.currentTime(10, e=1)
        mc.xform(s1, t=[5,6,7], ro=[20,40,60])
        mc.xform(s2, t=[-5,6,7], ro=[-20,-40,-60])
        mc.xform(g, t=[-5,-6,-7], ro=[-20,-40,-60])
        mc.setKeyframe(g)
        mc.setKeyframe(s1)
        mc.setKeyframe(s2)

        if makeDups:
            #these are named the same as the first spheres, but parented to world
            ds1= mc.polySphere(name='s1')
            ds2= mc.polySphere(name='s2')

        geo = "%s|%s %s|%s" % (g, s1, g, s2)
        print "exporting %s" % geo
    if not range:
        range = "1 10"
    path = os.path.join(os.environ.get('HOME'), 'abcTest.abc')

    if export:
        mc.AbcExport('range %s worldSpace uv attrPrefix LUMA_ %s %s' % (range, geo, path), d=1)    
        print "Finished exporting"

    if reImport:
        mc.AbcImport(path, d=1)
        print "Finished importing"

_test(newFile=1)
_test(geo='null1', newFile=0)

I also see some warnings about UV

Original comment by jspatr...@gmail.com on 25 May 2011 at 11:46

[GoogleCodeExporter]

Do you know if Maya would multi-thread this read? We're seeing some thread safe 
related issues on our end.

Original comment by asuter on 25 May 2011 at 11:54

[GoogleCodeExporter]

I haven't seen any thread safety reading issues yet.
You might want to double check how HDF5 was configured, you can do this via:

h5cc -showconfig

Under features you should see:
Threadsafety:  yes

Original comment by miller.lucas on 26 May 2011 at 12:03

[GoogleCodeExporter]

Looks like hdf is configured with threadsafety enabled.  This is built against 
1.8.7, but I originally built against 1.8.5-patch1.  Here is the result from 
h5cc -showconfig:

            SUMMARY OF THE HDF5 CONFIGURATION
           =================================

General Information:
-------------------
                  HDF5 Version: 1.8.7
                 Configured on: Mon May 16 18:46:52 PDT 2011
                 Configured by: johnp@ws-018.luma-pictures.com
                Configure mode: production
                   Host system: x86_64-unknown-linux-gnu
             Uname information: Linux ws-018.luma-pictures.com 2.6.35.11-83.fc14.x86_64 #1 SMP Mon Feb 7 07:06:44 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
                      Byte sex: little-endian
                     Libraries:
            Installation point: /Volumes/newhome/johnp/alembic-deps/hdf5-1.8.7

Compiling Options:
------------------
              Compilation Mode: production
                    C Compiler: /usr/bin/gcc ( gcc (GCC) 4.5.1 20100924 )
                        CFLAGS: -m64 -fPIC
                     H5_CFLAGS: -std=c99 -pedantic -Wall -Wextra -Wundef -Wshadow -Wpointer-arith -Wbad-function-cast -Wcast-qual -Wcast-align -Wwrite-strings -Wconversion -Waggregate-return -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -Wnested-externs -Winline -Wno-long-long -Wfloat-equal -Wmissing-format-attribute -Wmissing-noreturn -Wpacked -Wdisabled-optimization -Wformat=2 -Wendif-labels -Wdeclaration-after-statement -Wold-style-definition -Winvalid-pch -Wvariadic-macros -Wnonnull -Winit-self -Wmissing-include-dirs -Wswitch-default -Wswitch-enum -Wunused-macros -Wunsafe-loop-optimizations -Wc++-compat -Wstrict-overflow -Wlogical-op -Wlarger-than=2048 -Wvla -Wsync-nand -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wstrict-aliasing -Wstrict-overflow=5 -Wjump-misses-init -Wunsuffixed-float-constants -O3 -fomit-frame-pointer -finline-functions
                     AM_CFLAGS:
                      CPPFLAGS:
                   H5_CPPFLAGS: -D_POSIX_SOURCE   -DNDEBUG -UH5_DEBUG_API
                   AM_CPPFLAGS: -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_BSD_SOURCE
              Shared C Library: no
              Static C Library: yes
 Statically Linked Executables: no
                       LDFLAGS: -fPIC
                    H5_LDFLAGS:
                    AM_LDFLAGS:
               Extra libraries:  -lpthread -lz -lm
                      Archiver: ar
                        Ranlib: ranlib
             Debugged Packages:
                   API Tracing: no

Languages:
----------
                       Fortran: no

                           C++: no

Features:
---------
                 Parallel HDF5: no
            High Level library: yes
                  Threadsafety: yes
           Default API Mapping: v18
 With Deprecated Public Symbols: yes
        I/O filters (external): deflate(zlib)
        I/O filters (internal): shuffle,fletcher32,nbit,scaleoffset
                           MPE: no
                    Direct VFD: no
                       dmalloc: no
Clear file buffers before write: yes
          Using memory checker: no
        Function Stack Tracing: no
                          GPFS: no
     Strict File Format Checks: no
  Optimization Instrumentation: no
      Large File Support (LFS): yes
            H5dump Packed Bits: yes

Original comment by jspatr...@gmail.com on 26 May 2011 at 12:58

[GoogleCodeExporter]

Your configuration looks fine.

I'm also getting the crash, one huge hint is that the file generated can't 
looked at via h5ls or h5dump.

This means the file has been corrupted somehow.

Original comment by miller.lucas on 26 May 2011 at 1:06

• Changed state: Accepted

[GoogleCodeExporter]

Looking at the script a little closer, you are writing the file, reading it in 
your scene, then writing over it again.

Since the file has animated stuff in it a custom Maya AlembicNode is in your 
scene and is keeping the file open.

Writing over an HDF5 file while it is open is very bad, more info about it can 
be found here:

http://www.hdfgroup.org/HDF5/doc/RM/RM_H5F.html#File-Create

This is what is creating the corrupted file.

However, there still needs to be better handling in AbcImport for trying to 
read non-HDF files (like the corrupted one).

Reading an invalid HDF file should not cause Maya to crash.

Original comment by miller.lucas on 26 May 2011 at 6:56

[GoogleCodeExporter]

Makes sense - this is good to know.  I hadn't realized writing over the file 
while it was in use would corrupt it.

Thanks!

Original comment by jspatr...@gmail.com on 26 May 2011 at 9:35

[GoogleCodeExporter]

Fix for the crash when reading an invalid HDF file is here:  (AbcImport 
"/dev/null")

http://code.google.com/r/millerlucas-dev/source/detail?r=6d2ac34368147ef793e2adf
7d0cc793f799b343c

Original comment by miller.lucas on 26 May 2011 at 9:38

• Changed state: PleaseVerify

[GoogleCodeExporter]

Original comment by miller.lucas on 2 Jun 2011 at 5:44

[GoogleCodeExporter]

Sorry for the delay.  Confirmed this prevents an invalid file from being 
loaded.  Thanks!

Original comment by jspatr...@gmail.com on 13 Jun 2011 at 11:28

[GoogleCodeExporter]

Original comment by ard...@gmail.com on 23 Jun 2011 at 9:37

• Changed state: Verified