Open ctian282 opened 3 years ago
to force SAMRAI to use host allocation for GriddingAlgorithm you can run something like this on program startup (or sometime before the AllocatorDatabase is initialized)
auto& rm = umpire::ResourceManager::getInstance();
{ // initialize samrai internals allocator to HOST
assert(!rm.isAllocator("samrai::data_allocator"));
[[maybe_unused]] auto samrai_allocator = rm.makeAllocator<umpire::strategy::DynamicPool>(
"samrai::data_allocator", rm.getAllocator(umpire::resource::Host));
assert(samrai_allocator.getPlatform() == umpire::Platform::host);
}
edit: this will only work after #180 is merged
The recent changes in #180 should fix the issue with the allocator you receive from getTagAllocator, as that will be a host allocator by default, and the kernels for CellData will stay on the host for the tag variables. In general you can reassign the resource for the allocators provided in tbox::AllocatorDatabase
by using the syntax suggested here by @PhilipDeegan .
When compiling with CUDA, RAJA and umpire, in my environment with gcc-7, CUDA 10, I noticed that the usage of
getTagAllocator()
inGriddingAlgorithm.C
to initializeCellVariables
will cause CUDA illegal memory access when filling those tag variables withGriddingAlgorithm::fillTags()
. This causes the test sets failure as I mentioned in a previous issue. https://github.com/LLNL/SAMRAI/issues/175The issue is caused by initialing tag data with tagAllocator, but filling it as a shared memory variable. Replacing
getTagAllocator()
withgetDefaultAllocator()
will temporary resolve this issue and pass all the tests, but I am not sure if this will bring some potential problems.