Newton Dynamics Pros/Cons

[nuzrub]

I am looking for a physics library to create some testing benchmarks for collision detection and I am currently interested in trying Newton for it. However, I can't find any page/article/faq focused on telling me what Newton has that might be interesting to me. The sole mention of a distinguishing feature is the solver. So, my questions are: 1) is Newton fully or partially threaded? 2) Does it support a huge amount of spheres/boxes (100k - 10kk)? By support, I ask if it could potentially solve such a simulation without blowing up, even if it would take minutes per frame. 3) Which Broad Phase algorithms does Newton Support? If there is only one algorithm, does the library provides support for changing it? Such as Bullet's "btBroadphaseInterface"?

Thanks and sorry if this is not the proper place to ask. Cheers!

[JulioJerez]

I am not focused on comparison, for your needs maybe, some other libraries are a better fix for you.

On Thu, Sep 13, 2018, 06:36 Cyvasse notifications@github.com wrote:

I am looking for a physics library to create some testing benchmarks for collision detection and I am currently interested in trying Newton for it. However, I can't find any page/article/faq focused on telling me what Newton has that might be interesting to me. The sole mention of a distinguishing feature is the solver. So, my questions are:

1. is Newton fully or partially threaded?
2. Does it support a huge amount of spheres/boxes (100k - 10kk)? By support, I ask if it could potentially solve such a simulation without blowing up, even if it would take minutes per frame.
3. Which Broad Phase algorithms does Newton Support? If there is only one algorithm, does the library provides support for changing it? Such as Bullet's "btBroadphaseInterface"?

Thanks and sorry if this is not the proper place to ask. Cheers!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/MADEAPPS/newton-dynamics/issues/114, or mute the thread https://github.com/notifications/unsubscribe-auth/AHxHpMFeP__rA2KGh6zaaQlIgw6uOKfAks5ual9ugaJpZM4WnRT9 .

[nuzrub]

I see, that's ok. However, could you please answer whether the library is threaded and which broadphase algorithm is used? Thanks

[JulioJerez]

the library is threaded yes. the broad phase is a bound box hierarchic and I believe it is good enough that people can do thinks like this: https://www.youtube.com/watch?v=jjDZKQiAU-M

you can just download, and run it the sandbox demos, there is a option menu that let people test most of the basics functionality. It takes few minute to test it and if you feel it does not meet your expectations, you just simply delete the download folder.

[nuzrub]

Thanks man, I will have a look at the samples. Feel free to close this issue. Thanks!

[JulioJerez]

here is a more recent stress test made with the version that is currently in github https://www.youtube.com/watch?v=XqiYLl6JNc0&t=1772s

This can place over 50k rigid bodies, and can runs at interactive time on and 8 core 16 thread ryzen cpu there is not cheating of any kind as many of those demos made in blender and some other demo where bodies are frozen. the video is long because test many options, but if you see it by the end is test pile of 52k bodies.

[nuzrub]

That's pretty nice Right now, I have been using PhysX and around 52k it is pretty much slower than what you are showing me. Besides, I have to run it with really small timesteps to have a stable sim. Seeing this video, I will probably try Newton and see where I can get with it.

Currently, with PhysX, I am stuck at 128k objects for a "stack" scene. To make it work, I have to run it in timesteps of 1/30 seconds with 16 subsamples, about 512 solver iterations and a lot of patience. A 1000 frames sim is taking about 5 hours. Later I can send you a video of the simulation runnining for 4k objects, so that you can see how it looks.

About Newton, I am interested in the fact that it is threaded and that it seems pretty accurate, so I guess it can take me to the 1kk objects mark I want, even if it's going to take a long time to compute.

[JulioJerez]

there is a lot of misinformation about the Newton engine out there. look forward to see what you get.

[nuzrub]

Hey man, I have some feedback. First of all, the kind of simulations I am making are these: https://www.youtube.com/watch?v=1eJVuaKm6Mo In special, the Free Fall scenario is really problematic, as the "stacking" of objects is highly irregular. The ones in the video have 4k objects, the current limit I could reach with PhysX is 128k for the FreeFall Cubes, 1kk for the Brownian Cubes and 256k for Gravity Cubes. For the Assorted ones, it is about 32k, 128k and 32k.

I have used the NewtonUnityPlugin to use Newton instead of the PhysX and managed to integrate it within a day. However, the library didn't help me to go further into the number of objects. In fact, I keep having crashes around 32k cubes on the Free Fall scenario. The speed is pretty ok, but it crashes when a significant amount of the objects is at the bottom of the floor.

[JulioJerez]

Maybe the unity plug in is not up today. where does it crashes? It should be quite essay to recreate that in unity, but I am not the person who maintain that plug in. If you post over the forum or over the github unity plug in, I am sure Sweenie, the owner, can tell you what could be wrong.

[nuzrub]

About the plugin, I have compiled it myself with the lastest newton out of the repository. IMO, I don't think it is plugin related, maybe it is a race condition or memory issue. I will probably play with it a bit more tomorrow, se if changing some parameters can hold the sim up. I will also run it with the Brownian sim to pinpoint if it is number of objects related or number of collisions related.

[JulioJerez]

is that video the newton plugin or the unity physics engine? But in any case contact Sweeny he is very smart and can point out anything,
there are some tricks that has to be made to get the glue between cpp and ccharp

[nuzrub]

This is with the newton plugin

On Wed, Sep 19, 2018 at 8:03 PM NewtonDynamics notifications@github.com wrote:

is that video wit the newton plugin of with the unity physics engine? But in any case contact Sweeni he is very smart and can point out anything, there are some trick that has to be made to get the glue between cpp and ccharp

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MADEAPPS/newton-dynamics/issues/114#issuecomment-422986575, or mute the thread https://github.com/notifications/unsubscribe-auth/AD8ujiEW-G2LeF7wuFYIQVZSMO19FEEuks5ucs0mgaJpZM4WnRT9 .

[JulioJerez]

also I am not sure if he integrated the the AVX plugin it which give you some extra.
he would need to add those option to the selection menu so that you know what option you are selecting.

[JulioJerez]

what do you mean by this "In special, the Free Fall scenario is really problematic, as the "stacking" of objects is highly irregular." can you rephrase that?

[JulioJerez]

Ok I synced to the current unity plugin and was not enabling the parallel solver, so your code will not run a full speed. I quickly commited the changes to enable the default parallel on large island, that will make it better for those large island but I could not present the option in the editor since I have crashes doing that in Unity. Anyway if you sync both NewtonSDK and Unity plugin and buidl your simulation should be much better.

Also I suggest you post in the newton Forum so that you can make your case, there and the people who knows more about Unity and Newton combined gives you better advise than I can.

I will ask Sweenie to recreate a scene wit a large number of bodies in a transparent box, maybe that can recreate the crash you are having.

[nuzrub]

I will definitively try syncing both and seeing if I can get it to work with the parallel solver. Later I will post it on the newton forums. Thanks :)

On Thu, Sep 20, 2018 at 12:02 PM NewtonDynamics notifications@github.com wrote:

Ok I synced to the current unity plugin and was not enabling the parallel solver, so your code will not run a full speed. I quickly commited the changes to enable the default parallel on large island, that will make it better for those large island but I could not present the option in the editor since I have crashes doing that in Unity. Anyway if you sync both NewtonSDK and Unity plugin and buidl your simulation should be much better.

Also I suggest you post in the newton Forum so that you can make your case, there and the people who knows more about Unity and Newton combined gives you better advise than I can.

I will ask Sweenie to recrate a scene wit a large number of bodies in a transparent box, maybe that can recreate the crash you had.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MADEAPPS/newton-dynamics/issues/114#issuecomment-423216831, or mute the thread https://github.com/notifications/unsubscribe-auth/AD8ujlYxua9K589Z-bqfs1d45OskQoDFks5uc64AgaJpZM4WnRT9 .

[JulioJerez]

alright now I added the ability of loading the plugin. now in the world editor, you can check the option "Solve large island in parallel" and when the game start it will lode all the available parallel plugins and select the best according to your CPU. when running multithreaded the really make a big difference specially when running multithreaded. you need to sync both newton SDK and Unity plugin.

Good luck.

[nuzrub]

About the "In special, the Free Fall scenario is really problematic, as the "stacking" of objects is highly irregular.". Consider the following two images:

The first one is from the Free Fall simulation and is highly irregular. By irregular, I mean that the objects are not tightly packed/stacked on top of each other. On the second image, we can see that the "triangles" are highly "regular" in the sense that the objects that compose it are nicely packed/stacked on top of each other. In the rest of the video, the triangles are blown-up and generate a lot of irregularity.

The additional difficulty with the FreeFall scenario is that the space is rather limited. On the second image, the objects have plenty of room to disperse and rest whereas on the Free Fall they are bounded within the "cage".

[JulioJerez]

that does not make difference for the parallel solver. in fact the pyramid stacks are much harder to simulate because the stack is more neatly organized and organizing things in some specific configuration takes more effort than keeping them random. the random pile just goes to a point of lower potential energy state, while the tall stack has a much higher potential energy state. that's the metric that is used to determine which configuration is more complex.

anyway, the committed a version now that let you run the parallel solver, if you try it you will see that if you let it go for awhile it even manage to bring the whole pile to equilbrium, which is the equivalent of the pyramid stacks.

I am working the edit property so that you can select the solver from the list down.

[nuzrub]

I am using the lastest version of the sdk and plugin. It now prints that is using the avx2 plugin. I have also set m_useParallelSolver to true. It has improved the speed significantly, but did not solve the crashing issue. I will pack a minimal working example for you soon.

On Fri, Sep 21, 2018 at 10:54 AM NewtonDynamics notifications@github.com wrote:

that does not make difference for the parallel solver. in fact the pyramid stacks are much harder to simulate because the stack is more neatly organized and organizing things in some specific configuration takes more effort than keeping them random. the random pile just goes to a point of lower potential energy state, while the tall stack has a much higher potential energy state. that's the metric that is used to determine which configuration is more complex.

anyway, the committed a version now that let you run the parallel solver, if you try it you will see that if you let it go for awhile it even manage to bring the whole pile to equilbrium, which is the equivalent of the pyramid stacks.

I am working the edit property so that you can select the solver from the list down.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MADEAPPS/newton-dynamics/issues/114#issuecomment-423538905, or mute the thread https://github.com/notifications/unsubscribe-auth/AD8uji_S5tExZVKM4LLgw5t3bqZyL6Q_ks5udO-sgaJpZM4WnRT9 .

[JulioJerez]

ah ok if you can make a repro test, that will be great. I was thinking to ask Sweenie to recreate a mass rigid body demo to see what is wrong, but if you make a real repro that will be even better.

anyway I now added the last update that let you select from a drop down from the available plugins.

julio

[JulioJerez]

you may wnat to sync to latest I did find a bug that might case a crash and is fixed now.

[nuzrub]

Sorry for taking long to answer. Here is a minimal version of my project that is enough to reproduce the crash. I have added a readme file to guide you in using it. newton_crashscene.zip

I did sync to the lastest versions of both newton and its plugin, but the crash persists. I believe it will be interesting to compile newton and the plugin (or just the plugin) in debug mode to see if there are any prints that might indicate the reason for the crash.

[JulioJerez]

I try to open but I get a bunch of errors and I do not know how to make it work

Unexpected compatilibitylevel UnityEditorInternal.InternalEditorUtility:GetMonoIslands() System.Reflection.MethodBase:Invoke(Object, Object[]) SyntaxTree.VisualStudio.Unity.Bridge.CompilationUnit:CompilationUnits(String) SyntaxTree.VisualStudio.Unity.Bridge.CompilationUnit:CompilationUnits() SyntaxTree.VisualStudio.Unity.Bridge.ProjectSystem.UnitySolutionBuilder:.ctor() SyntaxTree.VisualStudio.Unity.Bridge.ProjectSystem.UnitySolutionBuilder:CreateSolutionFromAssetDatabase() SyntaxTree.VisualStudio.Unity.Bridge.ProjectFilesGenerator:GenerateProject() SyntaxTree.VisualStudio.Unity.Bridge.ProjectFilePostprocessor:OnPreGeneratingCSProjectFiles() UnityEditor.SyncVS:SyncVisualStudioProjectIfItAlreadyExists()

you need post over the forum so that the people who know more about Unity have a say on hwo to recret the crash. I know very litlly about Unity,

[JulioJerez]

You are using a version of Unity that I am no familiar with. I do not find those functions GetMonoIslands. In any case I was trying to make a demo with lot of objects and I believe I did find the cause for the crash, I am fixing it now.

[JulioJerez]

Ok I think I have it fixed now, or a t least that was one insidious crash bug that will happen only when the number of contact joint found in one frame was larger than the active joint buffer. the problem was that the buffer was no resized to be a least the size of the new number of contacts. This can happen to people doing the things you are doing. if you Sync the newton SDK, you can give another try.

BTW I am experimenting with the a generic GPU support, that should make stuff like you are doing much faster, maybe possible to do it in real time with a good GPU while keeping the simulation quality.
that's the hope.

[nuzrub]

When I get home I will try it out. I will also make a build of the project for you, so that you can run it without having to worry with Unity. Are you on Windows, right?

I am definitively interested in trying out the GPU support, speedups are always nice :) I have a 1060GT GPU, which is pretty good, but will probably run out of memory on really large sims, I guess.

Ygor

On Mon, Sep 24, 2018 at 3:29 PM NewtonDynamics notifications@github.com wrote:

Ok I think I have that foxed now, or a t least that was one insidious crash bug that will happen only when the the number of joint found in one frame was larger than the active joint buffer. This can happen to people doing the thing you are doing. if you Sync the newton SDK, you can give another try.

BTW I am experimenting with the a generic GPU support, that should make stuff like you are doing mush faster, maybe possible to do it in real time with a good GPU while keeping the simulation quality. that's the hope.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MADEAPPS/newton-dynamics/issues/114#issuecomment-424077368, or mute the thread https://github.com/notifications/unsubscribe-auth/AD8ujtmx8JsinxI0uv0kExpVA0bcGSy-ks5ueSSNgaJpZM4WnRT9 .

[nuzrub]

Sorry for taking long to answer. I am currently having some issues with the DLL generation processes. For some reason I still couldn't understand, the UnityPlugin isn't recognizing the Newton DLL any more. My guess so far is that the recent API changes have impacted the DLL's ABI in some way and the SWIG Bindings used by the plugin are not working properly anymore. For this reason, I could not test the changes you did to the constraints array.

On Mon, Sep 24, 2018 at 3:50 PM Ygor Rebouças ygor.reboucas@gmail.com wrote:

When I get home I will try it out. I will also make a build of the project for you, so that you can run it without having to worry with Unity. Are you on Windows, right?

I am definitively interested in trying out the GPU support, speedups are always nice :) I have a 1060GT GPU, which is pretty good, but will probably run out of memory on really large sims, I guess.

Ygor

On Mon, Sep 24, 2018 at 3:29 PM NewtonDynamics notifications@github.com wrote:

Ok I think I have that foxed now, or a t least that was one insidious crash bug that will happen only when the the number of joint found in one frame was larger than the active joint buffer. This can happen to people doing the thing you are doing. if you Sync the newton SDK, you can give another try.

BTW I am experimenting with the a generic GPU support, that should make stuff like you are doing mush faster, maybe possible to do it in real time with a good GPU while keeping the simulation quality. that's the hope.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MADEAPPS/newton-dynamics/issues/114#issuecomment-424077368, or mute the thread https://github.com/notifications/unsubscribe-auth/AD8ujtmx8JsinxI0uv0kExpVA0bcGSy-ks5ueSSNgaJpZM4WnRT9 .

[JulioJerez]

no nothing has change on the interphase, I just compiled the plugin and launched in Unity and runs fine.

[nuzrub]

Hey! I could get the NewtonUnityPlugin back running, it was just a minor mistake by me. However, the crashes are still happening, even though I am using the latest version of both repositories. I am sending you my testing framework as a build, so it should not need any unity knowledge to use. The application is a wizard to generate the sims, it should not be hard to figure out how to use it. If you leave all params as they are and just click on "Setar Parâmetros" and then on "Iniciar", a simulation will start and crash after a couple os frames.

Gerador Newton.zip

[JulioJerez]

Ok the demo that you provide does crashes after few frames and freezes window. When I build the newton DLL and replace NewtonWrapper.dll with the new build in does I run all the way until the end but crashes from Unity garbage collector on exit trying to delete some collision shapes.

I run in debug mode and simulate 2 frame, and it seems some how unity try to delete a shape that either was already deleted or is just a garbaged pointer. the problem is not deterministic the box that cause the crash is random from run to run. but I do not know who to debug unity code. Maybe if you post the demo in the forum sweney can tell you what is causing the problem

NewtonWrapper.dll!NewtonCollisionSetUserData(const NewtonCollision const collision, void const userData) Line 3971 C++ NewtonWrapper.dll!dNewtonCollision::DeleteShape() Line 77 C++ NewtonWrapper.dll!dNewtonCollision::~dNewtonCollision() Line 47 C++ [External Code] NewtonWrapper.dll!CSharp_delete_dNewtonCollisionBox(void * jarg1) Line 5110 C++ [External Code] mono-2.0-bdwgc.dll!00007fff5ff2c01b() Unknown mono-2.0-bdwgc.dll!00007fff5feb1db2() Unknown mono-2.0-bdwgc.dll!00007fff5febadbf() Unknown UnityPlayer.dll!00007fff51276d54() Unknown UnityPlayer.dll!00007fff512745c0() Unknown UnityPlayer.dll!00007fff5127467e() Unknown UnityPlayer.dll!00007fff51261226() Unknown UnityPlayer.dll!00007fff5126643e() Unknown UnityPlayer.dll!00007fff50fb839d() Unknown

I notice few things. 1-I do not undertand what the demo does, I see teh gravity value been different base of teh body count. 2-no matter hwo many body I set I see lot on teh few frame of tow and then they all move away and onel a few hundred are visible. 3-I see yopu are uint liek a milimiter scale. The engien is calobrate to work on Meters, I sugget you use a myuer scale, this is something that could be chnge but unfortunally ther are many hard code constant and tolreance in the engine that will cause drastic behavior. when using such huge scale. 4- I see a lot of random jitter not sure is that's part of your simulation, but is may be intruced by the collision system malfuntioning because on the large scale. 5-some of the bodies are gaing an insane angula velocity, not sure where that torque is coming from.

[nuzrub]

Could you send me the DLLs you built? If the crash happens only at the end, I'm pretty fine with it. About your comments: 1) All values scale with the number of objects to get the same object density. This includes the gravity and the scenario size. This helps with comparing a 1k objects scene with a 1kk objects scene. 2) I am not exactly sure what you meant. 3) I haven't paid much attention to this. Do you recommend scalling everything by 1 thousand? 4 and 5) Maybe these issues are due to scale.

[JulioJerez]

can't you just build it with visual studio? how did you get it in the first place?

[nuzrub]

Well, I am using the project that comes with the github repository. The one called "visualStudio_2015_static_mt". I am asking you for the DLLs you generated simply because for some reason yours isn't crashing during the simulation and mine is. I will try again with the CMake build.

On Sat, Sep 29, 2018 at 10:25 PM NewtonDynamics notifications@github.com wrote:

can't you just build it with visual studio? how did you get it in the first place?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MADEAPPS/newton-dynamics/issues/114#issuecomment-425686301, or mute the thread https://github.com/notifications/unsubscribe-auth/AD8ujuRT0fpcGB3g8C2E79DOQYHK_bCAks5ugB2JgaJpZM4WnRT9 .

[JulioJerez]

I guess thsi was resolved

[nuzrub]

I more or less gave up on it and stuck with PhysX (which is built-in into Unity) I kept having crash issues with Newton :c

[JulioJerez]

ah ok.

[nuzrub]

You can leave the issue closed though. Thanks for the help