Using the gem get_process_mem, I noticed that the memory usage by my process was increasing dramatically if I tried to do a large number of predictions (where each prediction creates an input tensor, and calls session.run)
I've tracked this down to line 59 of tensor.rb, which calls the TF_NewTensor_wrapper function. Looking at this, I noticed that in the tensorflow API documentation, it mentions that you must call TF_DeleteTensor whenever you call TF_NewTensor, and then pass in a deallocator function that will clean up any further data after the tensor is deleted. This appears to never occur, so although ruby is deallocating the tensor objects in ruby, the C++ memory is never freed as tensors are not deleted and the memory thats allocated in TF_NewTensor_wrapper is also never freed.
It seems like we need some sort of destructor in tensor.rb that then will call a TF_DeleteTensor_wrapper function that calls TF_DeleteTensor. In addition you'll need to pass in a deallocator function that calls free(data) when you create the tensors. I've experimented with this, but I'm not sure how to do this as ruby doesn't have an easy way to destruct objects.
arafatkatze
commented
6 years ago
Using the gem get_process_mem, I noticed that the memory usage by my process was increasing dramatically if I tried to do a large number of predictions (where each prediction creates an input tensor, and calls session.run)
I've tracked this down to line 59 of
tensor.rb, which calls theTF_NewTensor_wrapper function. Looking at this, I noticed that in the tensorflow API documentation, it mentions that you must callTF_DeleteTensorwhenever you callTF_NewTensor, and then pass in a deallocator function that will clean up any further data after the tensor is deleted. This appears to never occur, so although ruby is deallocating the tensor objects in ruby, the C++ memory is never freed as tensors are not deleted and the memory thats allocated inTF_NewTensor_wrapperis also never freed.It seems like we need some sort of destructor in
tensor.rbthat then will call aTF_DeleteTensor_wrapperfunction that calls TF_DeleteTensor. In addition you'll need to pass in a deallocator function that callsfree(data)when you create the tensors. I've experimented with this, but I'm not sure how to do this as ruby doesn't have an easy way to destruct objects.