A dual-GPU DEM solver with complex grain geometry support
DEM-Engine, nicknamed DEME, does Discrete Element Method simulations:
Currently DEME is a C++ package with an API design similar to Chrono's, and should be easy to learn for existing Chrono users.
You can find the movies of some of DEME's demos here.
You are welcome to discuss DEME on Project Chrono's forum.
DEME is now available as a Python package, pyDEME. It is quick to install and pick up its usage by trying this Python version. If you want to maximize the performance and use the cutting-edge features, you can instead install the C++ version of DEME from source.
To install pyDEME, use a Linux machine, install CUDA if you do not already have it. Useful installation instructions may be found here.
Some additional troubleshooting tips for getting CUDA ready:
Once CUDA is ready, you can pip
install pyDEME. In your conda environement, do
conda create -n pyDEME python=3.11
conda activate pyDEME
conda install cmake
pip3 install DEME
You can also install pyDEME via (Please don't use conda install
:conda install
for now, it is not yet behaving correctly)
conda create -n pyDEME python=3.11
conda activate pyDEME
conda install -c projectchrono pydeme
pyDEME
can be replaced with an environement name of your choice. Other Python versions other than 3.11 should work as well.
Then Python scripts can be executed in this environment. To understand the content of each Python demo, refer to the explanations of the C++ demos with the same names in Examples section.
You can also build C++ DEME from source. It allows for potentially more performance and more tailoring.
On a Linux machine, install CUDA. The newest release is recommended.
Once CUDA is ready, clone this project and then:
git submodule init
git submodule update
This will pull the submodule NVIDIA/jitify so that we can do runtime compilation.
Then, one typical choice is to make a build directory in it. Then in the build directory, use cmake
to configure the compilation. An example:
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
You may want to use this information if you need to update cmake to the newest.
We suggest that you install a cmake
GUI such as ccmake
, and ninja_build
generator, to better help you configure the project. In this case, the example above can be done like this alternatively:
mkdir build
cd build
ccmake -G Ninja ..
You generally do not have to change the build options in the GUI, but preferably you can change CMAKE_BUILD_TYPE
to Release
, and if you need to install this package as a library you can specify a CMAKE_INSTALL_PREFIX
.
Some additional troubleshooting tips for generating the project:
$PATH
and $LD_LIBRARY_PATH
. An example is given below for a specific version of CUDA, note it may be different on your machine or cluster. You should also inspect if nvidia-smi
and nvcc --version
give correct returns.
export CPATH=/usr/local/cuda-12.5/targets/x86_64-linux/include${CPATH:+:${CPATH}}
export PATH=/usr/local/cuda-12.5/bin${PATH:+:${PATH}}
export PATH=/usr/local/cuda-12.5/lib64/cmake${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-12.5/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export CUDA_HOME=/usr/local/cuda-12.5
Finally, build the project.
ninja
Some additional troubleshooting tips for building the project:
filesystem
not being a member of std
or arguments not expanded with ...
, then manually setting the flag TargetCXXStandard
to STD_CXX17
might help.ccmake
GUI as /usr/local/cuda/lib64/cmake/cub
. It may be a slightly different path on your machine or cluster.libcudacxx
is not found, then you may manually set it in the ccmake
GUI as /usr/local/cuda-12.0/targets/x86_64-linux/lib/cmake/libcudacxx
. Depending on your CUDA version it may be a slightly different path on your machine or cluster. You may also try to find these packages using find
.After the build process is done, you can start trying out the demos.
./bin/DEMdemo_SingleSphereCollide
can be used to test a correct installation. If it runs outputting a lot of texts (those are debug messages; the user do not have to worry about the content) and stops without an error in the end, the installation is probably good../bin/DEMdemo_Mixer
../bin/DEMdemo_Centrifuge
and ./bin/DEMdemo_Sieve
./bin/DEMdemo_BallDrop
, ./bin/DEMdemo_ConePenetration
, /bin/DEMdemo_RotatingDrum
, ./bin/DEMdemo_Repose
, ./bin/DEMdemo_Plow
../bin/DEMdemo_WheelDP
shows how to load a checkpointed configuration file to instantly generate a settled granular terrain, then run a drawbar-pull test on it. This demo therefore requires you to first finish the two GRCPrep demos to obtain the terrain checkpoint file. The granular terrain in these demos features DEM particles with a variety of sizes and shapes../bin/DEMdemo_WheelDPSimplified
is a simplified version of the previous drawbar-pull test which has no prerequisite. The terrain is simpilified to be made of only one type of irregular-shaped particles. It serves as a quick starting point for people who want to create similar experiments../bin/DEMdemo_Indentation
is a more advanced examples showing the usage of the custom additional properties (called wildcards) that you can associate with the simulation entities, and use them in the force model and/or change them in simulation then deposit them into the output files. Wildcards have more use cases especially if coupled together with a custom force model, as shown in some of the follwing demos../bin/DEMdemo_Electrostatic
simulates a pile of complex-shaped and charged granular particles interacting with a mesh that is also charged. Its purpose is to show how to define a non-local force (electrostatic force) which takes effect even when the bodies are not in contact, using a custom force model file. This idea can be extended to modeling a custom cohesion force etc../bin/DEMdemo_FlexibleMesh
simulates a deforming mesh interacting with DEM particles. The intention is to show that the user can extract the force pairs acting on a mesh, then update the mesh with deformation information. DEME does not care how this deformation is calculated. Presumably the user can feed the forces to their own solid mechanics solver to get the deformation. DEME does not come with a built-in linear solver so for simplicity, in this demo the mesh deformation is instead prescribed../bin/DEMdemo_GameOfLife
is a fun game-of-life simulator built with the package, showing the flexibility in terms of how you can use this tool../bin/DEMdemo_Fracture_Box
simulates a concrete bar breaking using a custom force model that creates inter-particle bonds and lets them break under certain conditions. This is a showcase for advanced usage of custom models that involves per-contact wildcard variables.The documentations for DEME are hosted on Chrono website (work in progress).
Some additional troubleshooting tips for running the demos:
CUDA_ERROR_UNSUPPORTED_PTX_VERSION
are encountered while you run the demos, or (rarely) the simulations proceed without detecting any contacts, then please make sure the CUDA installation is the same version as when the code is compiled.SetForceCalcThreadsPerBlock
prior to the start of simulation with the argument being 256 or even smaller choices like 128.expression must have pointer-to-object type but it has type "float"
, or unknown variable "delta_time"
? Check out what we did in demo DEMdemo_Electrostatic
. You may need to manually specify what material properties are pairwise and what contact wildcards you have using SetMustPairwiseMatProp
and SetPerContactWildcards
.DEMCubContactDetection.cu
)? Then did you pull a new version and just re-built in-place? A new update may modify the force model, and the force model in DEME are given as text files so might not be automatically copied over when the project is re-built. I am sorry for the trouble it might cause, but you can do a clean re-build from an empty directory and it should fix the problem. Do not forget to first commit your own branches' changes and relocate the data you generated in the build directory. Another solution is to copy everything in src/DEM
to the DEM
directory in the build directory, then everything in src/kernel
to the kernel
directory in the build directory, then try again.DEME is designed to simulate the interaction among clump-represented particles, the interaction between particles and mesh-represented bodies, as well as the interaction between particles and analytical boundaries. DEME does not resolve mesh–mesh or mesh–analytical contacts.
Set the CMAKE_INSTALL_PREFIX
flag in cmake
GUI to your desired installation path and then
ninja install
We provide examples of linking against both Chrono and DEME for co-simulations in chrono-projects. You need to checkout the feature/DEME
branch after cloning the code.
You need to build chrono-projects
linking against a Chrono installation (Chrono installation guide is here; note you have to make install
to install Chrono, not just build it), then link against DEME. The steps for building chrono-projects
:
Chrono_DIR
. It should be in <your_Chrono_install_dir>/lib/cmake
. Then configure the project;ENABLE_PROJECTS
to ON
and configure the project;ENABLE_DEME_TESTS
to ON
. Then configure the project;DEME_DIR
when prompted. It should be in <your_DEME_install_dir>/lib64/cmake/DEME
. Then configure the project.ChPF_DIR
option being prompted. If you did not build DEME with USE_CHPF being ON, you can ignore this option. Otherwise CMake will refuse to generate, and you need to set this option to be <your_DEME_install_dir>/lib64/cmake/ChPF
.Then build the project and you should be able to run the demo scripts that demonstrate the co-simulation between DEME and Chrono.
More documentations on using this package for co-simulations are being added.
This project should be treated as the collective intellectual property of the Author(s) and the University of Wisconsin - Madison. The following copyright statement should be included in any new or modified source files
Copyright (c) 2021, Simulation-Based Engineering Laboratory
Copyright (c) 2021, University of Wisconsin - Madison
SPDX-License-Identifier: BSD-3-Clause
New authors should add their name to the file CONTRIBUTORS.md
rather than to individual copyright headers.
This project exists independently of Chrono so developers should be sure to include the appropriate BSD license header on any code which is sourced from Chrono::GPU(DEM) or other parts of Chrono.
SAMPLE header for files sourced from Chrono
Copyright (c) 2021, SBEL GPU Development Team Copyright (c) 2021, University of Wisconsin - Madison SPDX-License-Identifier: BSD-3-Clause This file contains modifications of the code authored by the Project Chrono Development Team. The original license can be found below: Copyright (c) 2016, Project Chrono Development Team All rights reserved. Use of this source code is governed by a BSD-style license that can be found in the LICENSE file at the top level of the distribution and at http://projectchrono.org/license-chrono.txt. A copy of the license is below. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: - Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. - Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. - Neither the name of the nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
See the paper that explains the design and usage of DEME and cite
@article{zhang_2024_deme,
title = {Chrono {DEM-Engine}: A Discrete Element Method dual-{GPU} simulator with customizable contact forces and element shape},
journal = {Computer Physics Communications},
volume = {300},
pages = {109196},
year = {2024},
issn = {0010-4655},
doi = {https://doi.org/10.1016/j.cpc.2024.109196},
author = {Ruochun Zhang and Bonaventura Tagliafierro and Colin {Vanden Heuvel} and Shlok Sabarwal and Luning Bakke and Yulong Yue and Xin Wei and Radu Serban and Dan Negruţ},
keywords = {Discrete Element Method, GPU computing, Physics-based simulation, Scientific package, BSD3 open-source},
}
See the paper on using DEME for simulating rover dynamics and cite
@article{ruochunGRC-DEM2023,
title={A {GPU}-accelerated simulator for the {DEM} analysis of granular systems composed of clump-shaped elements},
author={Ruochun Zhang and Colin {Vanden Heuvel} and Alexander Schepelmann and Arno Rogg and Dimitrios Apostolopoulos and Samuel Chandler and Radu Serban and Dan Negrut},
year={2024},
journal={Engineering with Computers},
doi={https://doi.org/10.1007/s00366-023-01921-9}
}