search

intel / scikit-learn-intelex

Intel(R) Extension for Scikit-learn is a seamless way to speed up your Scikit-learn application
https://intel.github.io/scikit-learn-intelex/
Apache License 2.0
1.23k stars 175 forks source link

memory leak using SVR #1869

Open montagne5641 opened 5 months ago

montagne5641 commented 5 months ago

When using SVR with patch, a memory leak occurs every time an assignment is made repeatedly with for. This does not occur if patch is not used. Similar issues have been reported previously with SVC, and daal4py was suggested as an alternative at the time. However, SVR does not support daal4py. Is it possible to address the memory leak?

-----------------------------------------------------

use_intelex =1

if use_intelex==1: from sklearnex import patch_sklearn patch_sklearn()

from sklearn.svm import SVR import numpy as np

X = np.random.randint(10, size=(1000,500)) y = np.random.randint(2, size=(1000))

for i in range(10000): results = SVR().fit(X,y)

-----------------------------------------------------

montagne5641 commented 4 months ago

This issue was not resolved by using del and go.collect().... I think this problem becomes even more pronounced when used in combination with gridsearchcv etc.

import gc

use_intelex =1

if use_intelex==1: from sklearnex import patch_sklearn patch_sklearn()

import os from sklearn.svm import SVR import numpy as np

X = np.random.randint(10, size=(1000,500)) y = np.random.randint(2, size=(1000))

for i in range(10000): results = SVR().fit(X,y) del results gc.collect()

samir-nasibli commented 4 months ago

Thank you @montagne5641 for the report! In order to reproduce the issues, please share the version of daal4py and scikit-learn-intelex and more details about your env

montagne5641 commented 4 months ago

dear samir-nasibli Thank you. This is my environment. If you need any other information, I'll be happy to help.

daal4py==2024.4.0 scikit-learn-intelex==2024.4.0 scikit-learn==1.5.0 Windows 10 Pro

Note: The same issue occurred even when I lowered the version as shown below. daal4py-2023.1.1 scikit-learn-intelex-2023.1.1

montagne5641 commented 4 months ago

Any update on this?

jamescporter commented 1 month ago

Same here, any updates?

jamescporter commented 1 month ago

This issue was not resolved by using del and go.collect().... I think this problem becomes even more pronounced when used in combination with gridsearchcv etc.

import gc

use_intelex =1

if use_intelex==1: from sklearnex import patch_sklearn patch_sklearn()

import os from sklearn.svm import SVR import numpy as np

X = np.random.randint(10, size=(1000,500)) y = np.random.randint(2, size=(1000))

for i in range(10000):

results = SVR().fit(X,y) del results gc.collect()

did you find that you would always run out of memory doing a grid search using this?