StanfordLegion / legion

The Legion Parallel Programming System
https://legion.stanford.edu
Apache License 2.0
678 stars 145 forks source link

regent: Leak in deferred buffer created for CUDA formal parameters #1029

Closed elliottslaughter closed 3 years ago

elliottslaughter commented 3 years ago

I'm tracking down a leak with @syamajala in S3D, which we believe is coming from the code added to address #638. Valgrind is showing us backtraces that look like:

==2702930== 2,304 bytes in 96 blocks are definitely lost in loss record 265 of 288
==2702930==    at 0x483BEC3: operator new(unsigned long) (in /home/seshu/pkg/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2702930==    by 0x6E819CC: legion_deferred_buffer_char_1d_create (legion_c.cc:3178)
==2702930==    by 0x4FEDCAB: $__regent_task_10060_cuda (in /scratch2/seshu/legion_s3d/build/h2/libregent_tasks.so)
==2702930==    by 0x9A4238E: Realm::LocalTaskProcessor::execute_task(unsigned int, Realm::ByteArrayRef const&) (proc_impl.cc:1122)
==2702930==    by 0x9AAFA7E: Realm::Task::execute_on_processor(Realm::Processor) (tasks.cc:306)
==2702930==    by 0x9AB3C71: Realm::KernelThreadTaskScheduler::execute_task(Realm::Task*) (tasks.cc:1380)
==2702930==    by 0x9B0CA47: Realm::Cuda::GPUTaskScheduler<Realm::KernelThreadTaskScheduler>::execute_task(Realm::Task*) (cuda_module.cc:1855)
==2702930==    by 0x9AB2B9E: Realm::ThreadedTaskScheduler::scheduler_loop() (tasks.cc:1127)
==2702930==    by 0x9AB3006: Realm::ThreadedTaskScheduler::scheduler_loop_wlock() (tasks.cc:1231)
==2702930==    by 0x9ABA653: void Realm::Thread::thread_entry_wrapper<Realm::ThreadedTaskScheduler, &Realm::ThreadedTaskScheduler::scheduler_loop_wlock>(void*) (threads.inl:97)
==2702930==    by 0x9AC6FD1: Realm::KernelThread::pthread_entry(void*) (threads.cc:774)
==2702930==    by 0xAA78608: start_thread (pthread_create.c:477)

Based on initial discussion with @magnatelee we believe this is coming from the cudahelper.t code that creates deferred buffers to hold the formal parameters:

https://github.com/StanfordLegion/legion/blob/master/language/src/regent/cudahelper.t#L565-L594

magnatelee commented 3 years ago

This should be fixed by e17e1d1b. @elliottslaughter can you pull and try again?

syamajala commented 3 years ago

The fix works. Thanks!