[Python] max_mem_size limits which memory

[5james]

Hello, I tried using thundersvm and it was super fast on gpu. Unfortunately after some time my python script gets killed due to lack of RAM memory. I thought about using max_mem_size. Unfortunately my script got killed again :(. When I set max_mem_size=6000 I can see in htop about 7/16 GB ram usage at first. That would check out with other memory allocated. After couple minutes usage rises to ~11GB. And it doesn't drop. When predict was happening script got killed again. Beside usual kernel parameters I'm using: probability=True, verbose=True, random_state=np.random.RandomState(0), max_mem_size=6000.

Now my question is: max_mem_size restricts RAM memory or VRAM (GPU) memory?

Extra question: Is there any way of utilizing my gpu more efficiently? Now I'm using like 2% of its vram.

[QinbinLi]

The max_mem_size restricts the maximum memory size of RAM memory and GPU memory. If you set max_mem_size to 6000, then both RAM memory and GPU memory will not exceed 6GB. To avoid the lack of RAM memory, I think you can set max_mem_size smaller.

[5james]

Well then either I'm doing something wrong or it doesn't work:

svm_classifier = thundersvm.SVC(kernel=params['kernel'],
                                degree=params['degree'],
                                coef0=params['coef0'],
                                probability=True,
                                verbose=True,
                                max_mem_size=6000,
                                random_state=np.random.RandomState(0))

When I'm watching it in htop it starts ~7.5GB of total RAM usage (1.6GB other processes and loaded data). After couple minutes it rises to ~10GB. When it's doing fit function it rises above 12-13GB.

Could it be somehow due to wrong thundersvm (through pip) installation?

[QinbinLi]

Hi, @5james

Thundersvm doesn't use 6GB memory at the beginning of the training. The usage of memory will increase in the training process. In your case, I think the other processes already occupied 7.5GB at the beginning. Thundersvm only uses about (13-7.5)=5.5 GB. We have tried max_mem_size and it works. If you still find the parameter not working, please provide the datasets so that we may reproduce it. Thanks.

[zeyiwen]

This issue is not active now, so we like to close it.