P/Invoke: Calling GetRotationalLimitMotor of 6DoF constraint causes crash in .NET2.0

[GoogleCodeExporter]

Hello,

I've been using the P/Invoke version in Unity and it's been working great so 
far. Yesterday I found out that creating a 6DoF constraint and calling 
GetRotationalLimitMotor causes the Unity process to crash about a second after 
running the scene. Debugging the process in Visual Studio gives "Unhandled 
exception at ntdll.dll in Unity.exe: 0xC0000374: A heap has been corrupted" 
error.

I continued to investigate if I could reproduce the crash in a demo program by 
modifying MotorDemo to contain a 6DoF constraint and calling 
GetRotationalLimitMotor, and found out the crash happens only when the program 
is built for .NET2.0 or 3.5.

Additionally I found out the modified demo program crashes only when "Enable 
Native Code Debugging" is unchecked in the project properties and the demo is 
ran in Visual Studio debugger (i.e. not running the .exe directly). Weird thing 
is that the motor seems to be working correctly for the ~1 second before the 
crash, and Visual Studio never triggers any breakpoints when the crash happens.

I've attached the modified VS solution here, which uses
-Bullet (2.8.3) compiled in VS2008
-libbulletc (r643) compiled in VS2008 targeting .NET2.0
-Bulletsharp P/Invoke branch (r643) targeting .NET2.0
-SharpDX .NET2.0 binaries

Other versions of bullet (2.8.2) or unmodified libbulletc (targeting .NET4.0) 
should work similarly. The solution also contains a picture of the crash. 
Running MotorDemo in Visual Studio should reproduce the issue.

I kind of hate myself for reporting this as a bug when I know the wrapper is 
built for .NET4.0 and doesn't offically support Unity, but since I'm running 
out of ideas and other functionality is working so well I thought maybe there's 
something that could be done.

What steps will reproduce the problem?
1. Create a 6DoF constraint in .NET 2.0 or 3.5 environment
2. Call GetRotationalLimitMotor of the constraint with any index
3. Host application crashes

What is the expected output? What do you see instead?
-Program should not crash.

What version of the product are you using? On what operating system?

Tested with various Bullet/Bulletsharp PInvoke library combinations on Windows 
8.1, built in VS2012:

-libbulletc.dll (0.4) + bulletsharp.dll (0.4) from the front page link

-Built bullet 2.8.3 from GitHub (28f19f1b...) + built libbulletc.dll (r643) + 
built bulletsharp.dll (r643, tried targeting both .NET2.0 and 3.5); I had to do 
slight modifications to make libbulletc and bulletsharp compile, mostly 
commenting some parts related to serialization/importing.

-Built bullet 2.8.2 (r2727) + built libbulletc.dll (r643) + built 
bulletsharp.dll (r643+.NET2.0/3.5)

-Built bullet 2.8.3 from GitHub (28f19f1b...) in VS2008, built libbulletc 
(r643) in VS2008 targeting .NET2.0, built bulletsharp.dll targeting .NET2.0.

And probably few other combinations I can't remember. I also tried specifying 
both "x86" and "Any CPU" in Bulletsharp project with no success.

Sorry for the wall of text.

Original issue reported on code.google.com by TheZ...@gmail.com on 23 Jun 2014 at 3:31

Attachments:

• BulletsharpPInvoke.NET2.0.zip

[GoogleCodeExporter]

Okay, so I managed to get VS to trigger a AccessViolationException exception by 
compiling bullet and libbulletc in debug configuration and found out the crash 
is caused by the Dispose method of RotationalLimitMotor in 
Generic6DofConstraint.cs on line btRotationalLimitMotor_delete(_native);.

It would seem like the garbage collector in .NET2.0/3.5 tries to dispose the 
rotational limit motor around a second after running the program, which causes 
a crash for some reason. The crash can be postponed by explicitly saving a 
reference to the RotationalLimitMotor object, but in this case the crash 
happens when quitting the program.

For now I just commented out the dispose methods of RotationalLimitMotor to 
suppress the crashes, which will cause memory leak on the long run if I'm not 
completely mistaken. I wonder if there's a better solution?

Original comment by TheZ...@gmail.com on 24 Jun 2014 at 10:59

[GoogleCodeExporter]

btGeneric6DofConstraint allocates memory for the btRotationalLimitMotors by 
itself, but the wrapper class RotationalLimitMotor thinks that it is 
responsible for freeing that memory.

The solution is to add a preventDelete flag to RotationalLimitMotor so that it 
woouldn't delete the object that really belongs to btGeneric6DofConstraint. 
I'll fix it today.

Don't worry about the wall of text. I like detailed bug reports very much. :)

Original comment by andres.traks on 24 Jun 2014 at 3:00

• Changed state: Started

[GoogleCodeExporter]

Fixed in r644.

Original comment by andres.traks on 24 Jun 2014 at 10:50

• Changed state: Fixed