Open 90600 opened 10 months ago
We have many customers use PyTorch engine, and we didn't observe memory leak issue. PyTorch use memory pool to manage native memory. It won't release back to system. As long as it stable at the peak, it should be fine.
We have many customers use PyTorch engine, and we didn't observe memory leak issue. PyTorch use memory pool to manage native memory. It won't release back to system. As long as it stable at the peak, it should be fine.
那你用过OnnxRuntime引擎去跑么,它是不会回收的
We have many customers use PyTorch engine, and we didn't observe memory leak issue. PyTorch use memory pool to manage native memory. It won't release back to system. As long as it stable at the peak, it should be fine.
你可以执行一下我上面的代码,你就会发现问题,用OnnxRuntime引擎
We have many customers use PyTorch engine, and we didn't observe memory leak issue. PyTorch use memory pool to manage native memory. It won't release back to system. As long as it stable at the peak, it should be fine.
PyTorch引擎确实会把内存稳定在一个区间里,但是OnnxRuntime引擎会一直递增,同一份代码
我也存在这个问题。
@kkangert
We are running onnxruntime benchmark nightly, and we never obverse memory leak. I created a project similar to above code, and it dose not have memory leak: https://github.com/frankfliu/djl/tree/yolo5/yolo
Here is a few common mistakes that causing memory issues:
Model
or Predictor
object, and you didn't close them, this will cause native memory leak and eventually will cause OOM. Make sure you load model only once, and all the threads use the same Predictor
ThreadLocal
for each Predictor
, but Predictor
is not closed when thread terminated. This will cause memory leak.Translator
implementation has memory leak. In my example, I'm using our built-in Translator
Description
本来是把djl推理yolov5模型整合到springboot项目中,压测发现内存无法回收,随着压测的增加内存也增加。 注意:和这个issues并不是一个问题https://github.com/deepjavalibrary/djl/issues/2800 ,它这个在main线程中执行确实不会有问题,但是异步执行就会有内存泄漏问题。 在spring项目中内存并不会回收 项目中没有用到opencv
Expected Behavior
100个线程预测就会出现内存变大并且并不会回收![image](https://github.com/deepjavalibrary/djl/assets/147969415/2ff01279-01fd-4388-82c8-9a4a97ef7cd4)
How to Reproduce?
Environment Info
希望您们抽出宝贵的时间帮忙看下,因为这个对我很重要
今天我在换成PyTorch引擎加载后,内存会回收,但是压测完后还是会有3G内存在,这让我怀疑是不是djl的问题。