Memory leak in SMS EPI and future use under XA

[ManfredKloebl]

Dear All,

we noticed a considerable memory leak in the R016a release for VE11C of the SMS EPI we run on our Prisma scanner. The extent of the memory leak depends on the sequence settings (resolution and TR) and builds up when repeatedly running the sequence. I read in the change logs that a memory leak was previously fixed, so the issue might have survived in this version or it is something different. Restarting the MrIrisContainer frees the memory but we can't do this between repeated acquisitions since the reconstruction is still running for quite some time after recording was finished.

I also wanted to ask whether you already know if a version of the DWI and BOLD SMS sequences for the new XA operating system will be available.

Many thanks in advance and best regards, Manfred

[eauerbach]

I am not aware of any significant memory leaks in R016, it has been in heavy use by a large number of users for many years. Are you really seeing MaRS crashes due to out of memory errors?

ICE has a centrally managed memory pool, with a specific size limit set based on the system (varies depending on the MaRS hardware and the software version). ICE programs request memory from this centrally managed pool. This memory is not all preallocated at startup, so you will see the MrIrisContainer process consume more and more memory over time, this is normal. Eventually it should plateau. It appears that this pool memory is never released to the system, only reallocated internally as needed.

Yes, an XA version is coming (see pinned issue #258).

[ManfredKloebl]

Thanks for the quick response (and sorry for overlooking the existing XA issue)!

Colleagues previously helped me to backtrack the issue with the SMS to the lines in the error log attached (PRISMA_HCP_errorlog_SID_164_fromMemAllocFailed.txt). In the user interface the error is just displayed as "Image reconstruction failed. MeasUid 42. Domain: MRI_PRC", which was quite uninformative to us. As the suspicious messages in the log file start with "Memory allocation failed.Tried to allocate memory bloc of 398721024 bytes (380 Mb)" and due to the observed buildup of reserved memory which we haven't seen for other acquisitions, we concluded that a memory leak is the most probable reason. That releasing the memory resolved the problem also substantiated our assumption. However, I agree that we cannot be absolutely sure about this being a memory leak especially since you mentioned that this memory is not automatically released. As for the memory use reaching a plateau, we also observed this, but when starting a new acquisition with the SMS sequence without restarting the MrIris component, the buildup continues from the previous plateau, even though at a slower speed. Acquisition is usually aborted at around 80% memory use.

[eauerbach]

I agree that doesn't look good. If it happens again please collect a savelog and send that to me, also a copy of the protocol you are running so I can try to replicate it.

These sequences are really heavily used by a lot of sites and I am not hearing of this problem elsewhere, so I'm curious what must be different in your case. Are you running any other unusual/custom sequences or reconstructions that might be using a lot of memory? It is conceivable that another ICE program is leaking memory, or "stealing" memory the framework expects to be available, in which case the error during the MB acquisition could be exposing a preexisting problem. Or it is also possible you have found an uncommon combination of parameters that is exposing a bug in the MB ICE I haven't seen before, I don't know.