Safe memory allocation, fix parallel violation overwrite variables.

[OmidGhotbi]

Safe memory allocation, fix parallel violation overwrite variables.

[OmidGhotbi]

i will make the change you asked me and then I have many changes in multithreaded parts that I will push in a few days, Also I modified the simulation equation to get much better results, in my test is absolutely stable now. i tested it with 4 beta testers with about 2 million particles and 70 substeps for 50 frames (3500 simulation steps) and it took about 57 minutes, no crash at all. in 4 different hardware configurations 4 different blender versions 3.6, 4.02, 4.1

[OmidGhotbi]

More than 2 Million particles, using all, link, relink, break, randomization 6 collisions. RAM 3840 MB CPU usage 16%-22% Cache on disk Binary So far no crash no instability.

[PavelBlend]

and then I have many changes in multithreaded parts that I will push in a few days

Will it be possible to achieve greater computing speed? So that the CPU is loaded not at 12%, but at 70-100%? Or does the simulation algorithm not allow it to be parallelized well?

[OmidGhotbi]

i may be able to go up to 40% not more, but there is a few problems, first, the nature of the connection to Blender with Python takes too much time and It has a huge overhead, I probably you already noticed that if you want to just load the cache and update particles you need to wait for more than normal, especially with a huge number of particles. Second the algorithm that has been used in the addon is not the best and it relies on a huge amount of unnecessary data transfer reallocation and calculation. you can achieve better results with MLS-MPM as well. (Actually, i have a question about the Taichi addon that you wrote please) Third is the C type of multithreading that I'm not perfect in, i may convert it to C++ to take advantage of using TBB so i can make nested loops and non-loop parts of the code multithreaded as well. That will help me to get even to 65% or 70% usage in CPU. If you can help me to somehow bypass Blender and load the cache just every 1 Frame instead of every single subframe we can get even more performance as ti will reduce data transfer between Blender slow Python and simulation. in the best case with existing code and my code ability i can reach about 70% and it is about 4 times faster in reality because not all of CPU usage will be actual simulation.

[PavelBlend]

Second the algorithm that has been used in the addon is not the best

what is the name of the simulation method used in blender-molecular? Did Pyroevil come up with it himself or is it some kind of known method? I want to search the Internet about the simulation method in this addon, but I don’t know what words to use to do this.

Actually, i have a question about the Taichi addon that you wrote please

What question do you have? I only wrote integration with blender in the taichi-elements addon. I did not take part in the development of the MPM engine.

If you can help me to somehow bypass Blender and load the cache just every 1 Frame instead of every single subframe we can get even more performance as ti will reduce data transfer between Blender slow Python and simulation.

I would like to port the particle systems and collision code from blender source code into the C core molecular addon. This will avoid having to access python/blender at all. But I don't know how to do it.

[OmidGhotbi]

1- He does not mention the exact name of the solver but by looking at the code it seems that is based on a simple version of Iterative-relaxation that works with connection and relation of a point. the original idea is from 1962, developed in 1980-83-86-2002 was the last time I have seen progress in it. MPM can do much more and far better. in some cases, Iterative-relaxation is a better choice for example if you want to shred bread or cloth. 2- I can not make it work at all, even I install it and it is simulated with no errors but the cash files are empty. can you help me to make it thank you in advance. 3- I will be more than happy to help you. the existing cache system has a problem in animation render so it needs to be fixed before continuing to better the cache system.

[PavelBlend]

MPM can do much more and far better.

What is the performance and computational speed of the MPM method? It seems to me that MPM is too demanding on CPU performance. To create a simulation with a large number of particles, you need to set a huge number of substeps (several hundred). This is necessary for the stability of the simulation. At least this is the behavior of MPM from taichi-elements. In my opinion and experience, molecular-addon works faster, at least on my CPU.

I can not make it work at all, even I install it and it is simulated with no errors but the cash files are empty. can you help me to make it thank you in advance.

I'll now try to run taichi-elements. I haven't done this for a long time. I'll write about the results later.

[OmidGhotbi]

About MLS-MPM it is quite fast if we sacrifice the overheads and non useful data for visual. i can make it faster than molecular with the same number of particles. the old MPM is slower than SPH, Flip or PIC but the last modified version from 2018 is much faster than all of them and quite more accurate. To be sure I want to have all of them in one place, I want a multiphysics simulation for blender ones for all. i had great progress to be honest. in some parts use the code like molecular and make it more stable and faster in other parts write completely new solvers.

Thank you for trying Taichi-Ellements which can help me a lot.

[PavelBlend]

I checked taichi elements. I installed taichi 1.7.0 in blender 3.6. Then I installed the addon. After running the simulation, warnings were printed in the console, but there were no critical errors. As a result, particles were created:

https://github.com/PavelBlend/blender-molecular/assets/7983249/05d08001-2402-4cb2-8866-9341b82bf443

Nodes:

Most likely you placed the emitter outside the domain: https://taichi-elements.readthedocs.io/en/latest/blender_nodes.html#mpm-solver

Change the MPM Solver Size parameter or move the Emitter.

[PavelBlend]

Try installing taichi 1.2.0 instead of 1.7.0 as it is more stable.

pip install taichi==1.2.0

[OmidGhotbi]

taichi 1.2.0 is not available anymore i tried to install Taichi 1.6.0 and at least now it starts to simulate and I can see particles in the viewport but it works just on Cuda and it will be stuck after a few frames. if I use a small resolution it will work but at the end I see this error at the end:

With the CPU, it will crash right away. Is taichi unstable like that or it is my PC? do I need to install anything to make it more stable? maybe I'm doing something wrong!!!

I've seen this error for render too.

[OmidGhotbi]

can you test and confirm CPU backend working for you or not, I tried on two windows systems both of them blender crashed at the simulation start. i wasn't able to install Taichi 1.2 so I was forced to use 1.6.0 pip install taichi==1.2.0 ERROR: Could not find a version that satisfies the requirement taichi==1.2.0 (from versions: 1.6.0, 1.7.0) ERROR: No matching distribution found for taichi==1.2.0

[PavelBlend]

What version of blender/python are you using? I'm using blender 3.6 and python 3.10. Installation 1.2.0 works for me. If I try to install a version that doesn't exist, I get this error:

pip install taichi==9.9.9
ERROR: Could not find a version that satisfies the requirement taichi==9.9.9 (from versions: 0.9.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.1.0, 1.1.2, 1.1.3, 1.2.0, 1.2.1, 1.2.2, 1.3.0, 1.4.0, 1.4.1, 1.5.0, 1.6.0, 1.7.0)
ERROR: No matching distribution found for taichi==9.9.9

Update the taichi_elements addon. Yesterday I added a new commit.

[OmidGhotbi]

Let me completely uninstall and reinstall Python, blender, GCC, etc. i think something is not working with my Python. as it can not even find taichi less than 1.6.0 I try to re-install everything and let you know.

[OmidGhotbi]

i reinstalled everything, and now I'm using Taichi 1.2.0 on Python 3.10.11, With Cuda even the demos going to stop after a few frames, around 28.

With the CPU is stable for Demos and working with no issues, The Taichi element on CUDA is not stable and will freeze after a few frames, The Cpu is working the first time with no issues but if I do simulate again it will crash,

[PavelBlend]

Try using version 0.8.1. Or older. Or newer. I am not a taichi developer. I don't know why the errors occur. A long time ago, when I used it, the version was about 0.8.1 and there were no errors on CPU. I haven't tested it on CUDA. I don't have a CUDA video card.

[OmidGhotbi]

you help a lot already, thank you. I fixed all the changes that you requested if you want to merge the commit. i will add a suggestion for fixing the cache and making it independent from blender, if you have time I'll be glad to work on it together. By the way do you have any information about how mesh collision works in Taichi-Element or maybe in old repo I can find anything about it?

[PavelBlend]

Also I modified the simulation equation to get much better results

What are the changes? Looking at the code, I can't figure out what changed in the equations. I'm not good at math or physics engines.

[PavelBlend]

@OmidGhotbi can you rewrite these loops to run in parallel? https://github.com/PavelBlend/blender-molecular/blob/902db6a8e52a7fa36e4617d62aa0ab665c09c6b8/core/sim.c#L190

[OmidGhotbi]

I already did i will push the new code after approving the last pushed code. but do not expect so much improvement I managed to get about 20% of CPU at the moment.

[OmidGhotbi]

great i will do a cleanup and push new code.

[OmidGhotbi]

in meanwhile can you tell me how can I render the Taichi Element simulation?

[PavelBlend]

You need to use a geometry node. Here is an example blend file: test_ti_render.zip

[OmidGhotbi]

thank you.