CNTK 2.2 memory leak problem during evalution of a model

[MaximumProgrammer]

Hello, I just faced a memory leak problem, during the evaluation of my model into a c++ program. The classificator was trained with Python (2.7) on a linux machine and afterwards exported and reloaded in a c++ program. Mainly i programmed a server (thrift) because i want to use this model into a 32 bit program where those needed data are just exchanged. Are there any workarounds ?, my program has to classificate multiple images so it is not possible to stop the program, but reloading the same model doesn't work after each evaluation the memory usage just increases.

Are there any workarounds ?

-setting parameters in my python script (Keras) ? -in my c++ program, reloading does not work !!!

[ebarsoumMS]

Can you share your eval code?

[MaximumProgrammer]

Thx, this is more or less the code of the evaluation, maybe there are some conflicts between other libraries. If i comment this line modelFunc->Evaluate(inputDataMap, outputDataMap, device), than there is no memory leak.

int SingleSampleEvaluationUsingDense ( cv::Mat inputimage, const DeviceDescriptor& device, std::vector &labels, std::vector &confidence_intervals ) {

ifdef DEBUG

std::cout << "===== Gaining input image." << std::endl;

endif

//Model definition:
const wchar_t* modelFile = L"./files_cntk/model_cntk.dnn";

//Parameter defintion:
const wchar_t* inputparameter = L"./files_keras/parameters.json";

wstring ws(inputparameter);
// your new String
string parameters(ws.begin(), ws.end());
// Show String

ifdef DEBUG

cout << "Getting parameters from " << parameters << endl;

endif

// read a JSON file
//http://www.parresianz.com/c++/json/json-input/
std::ifstream input_json(parameters);
json json_file;
input_json >> json_file;
input_json.close();

//std::cout << json_file.dump(4) << std::endl;
// converting the image to the necessary dim and for normalization
std::string str_imgsize = json_file["image size"];
std::string str_imgequalize = json_file["image normalize"];

int inputheight = stoi(str_imgsize);
int inputwidth = stoi(str_imgsize);
bool equalize = false;
if (str_imgequalize == "True")
{
    equalize = true;
}

std::string str_mean = json_file["mean"];
std::string str_std = json_file["standard deviation"];
std::string str_classes = json_file["classes"];
std::string str_interpolation = json_file["cv_resize_interpolation_method"];

double mean = stod(str_mean);
double std = stod(str_std);

ifdef DEBUG

std::cout << "Image size: " << inputheight << " x " << inputwidth << "" << " mean: " << mean << " standard deviation: " << std << " output classes: " << str_classes << std::endl;
std::cout << "===== Pre-processing image" << std::endl;
std::cout << "Resizing\n";

endif // DEBUG

if (equalize == true)
{
    equalizeHist(inputimage, inputimage);
}

cv::Mat image;
cv::resize(inputimage, image, cv::Size(inputwidth, inputheight), 0, 0, stoi(str_interpolation)); //cv2.INTER_AREA == 3

ifdef DEBUG

std::cout << "Success resizing \n";

endif

cv::Mat image2;
image.convertTo(image2, CV_32FC1);
image = image2;

image = image / (255.0 * 255.0);
image = image - mean;

ifdef DEBUG

std::cout << "Width : " << inputimage.cols << std::endl;
std::cout << "Height: " << inputimage.rows << std::endl;

std::cout << std::endl;

std::cout << "Width : " << image.cols << std::endl;
std::cout << "Height: " << image.rows << std::endl;
//std::cout << image << std::endl;

endif

/*Pre-processing images*/
/*Pre processing possibilty 1 (normalization)*/
if (std >= -1)
{
    image = Image_Normalize(image, std);
}
/*Pre processing possibilty 2 (PCA - whitening)*/
else if (std == -2)
{
    image = Image_PCA_Whiten(image, 1E-9);
}
/*Pre processing possibilty 3 (PCA - whitening)*/
else if (std == -3)
{
    image = Image_PCA_Whiten(image, 1E-9);
}
/*Pre processing possibilty 4 (ZCA whitening)*/
else if (std == -4)
{
    image = Image_ZCA_Whiten(image, 1E-9);
}
/*Pre processing possibilty 5 (SVD whitening)*/
else if (std == -5)
{
    image = Image_SVD_Whiten(image);
}

//image = Image_PCA_Whiten(image, 1E-9);    //works, but not preferable weak preprocessing
//image = Image_ZCA_Whiten(image, 1E-9);    //works, preferable, same outputs from c++ and python
//image = Image_SVD_Whiten(image);          //works, same outputs from c++ and python
//image = Image_Normalize(image, std);      //works, preferable, same outputs from c++ and python

//#pca #not working //1 works, different outputs c++ and python (black box pca)
//#pca #not working //2 works, helps but weak improvements especially for testing
//#zca #works       //3 works, best but expensivest possibility
//#svd #works       //4 works, cheap but useless, no improvements
//#normalize        //5 works, cheap but good, good improvements
//#                 //>5 only subract mean, no improvements 

ifdef DEBUG

std::cout << "\n===== Evaluate single sample using dense format." << std::endl;

endif

// Load the model.
// The model is trained by <CNTK>/Examples/Image/Classification/ResNet/Python/TrainResNet_CIFAR10.py
// Please see README.md in <CNTK>/Examples/Image/Classification/ResNet about how to train the model.

FunctionPtr modelFunc = Function::Load(modelFile, CNTK::DeviceDescriptor::CPUDevice());

// Get input variable. The model has only one single input.
Variable inputVar = modelFunc->Arguments()[0];

// The model has only one output.
// If the model has more than one output, use modelFunc->Outputs to get the list of output variables.
Variable outputVar = modelFunc->Output();

// Prepare input data.
// For evaluating an image, you first need to perform some image preprocessing to make sure that the input image has the correct size and layout
// that match the model inputs.
// Please note that the model used by this example expects the CHW image layout.
// inputVar.Shape[0] is image width, inputVar.Shape[1] is image height, and inputVar.Shape[2] is channels.
// For simplicity and avoiding external dependencies, we skip the preprocessing step here, and just use some artificially created data as input.
//std::vector<float> inputData(inputVar.Shape().TotalSize());
/*std::cout << inputVar.Shape().TotalSize() << std::endl;
for (size_t i = 0; i < inputData.size(); ++i)
{
inputData[i] = static_cast<float>(i % 255);
}
*/

std::vector<float> inputData;
if (image.isContinuous()) {
    inputData.assign((float*)image.datastart, (float*)image.dataend);
}
else {
    for (int i = 0; i < image.rows; ++i) {
        inputData.insert(inputData.end(), image.ptr<float>(i), image.ptr<float>(i) + image.cols);
    }
}

// Create input value and input data map
ValuePtr inputVal = Value::CreateBatch(inputVar.Shape(), inputData, device);
std::unordered_map<Variable, ValuePtr> inputDataMap = { { inputVar, inputVal } };

// Create output data map. Using null as Value to indicate using system allocated memory.
// Alternatively, create a Value object and add it to the data map.
std::unordered_map<Variable, ValuePtr> outputDataMap = { { outputVar, nullptr } };

ifdef DEBUG

std::cout << "Input size (channels x height x width) " << inputVar.Shape().TotalSize() << std::endl;
std::cout << "Number of outputs (channels x height x width) " << outputVar.Shape().TotalSize() << std::endl;

endif

// Start evaluation on the device
modelFunc->Evaluate(inputDataMap, outputDataMap, device);

// Get evaluate result as dense output
ValuePtr outputVal = outputDataMap[outputVar];
std::vector<std::vector<float>> outputData;
outputVal->CopyVariableValueTo(outputVar, outputData);

ifdef DEBUG

std::cout << "===== Evaluate." << std::endl;

endif

//print output to console

ifdef DEBUG

PrintOutput<float>(outputVar.Shape().TotalSize(), outputData);

endif*/

//getting confidence intervalls
std::vector<double> results_confidence = GetResults_Confidence<float>(outputVar.Shape().TotalSize(), outputData);

//getting labels
std::vector<int> results_labels = GetResults_Labels<float>(outputVar.Shape().TotalSize(), outputData);

//inserting into intervals
confidence_intervals.insert(confidence_intervals.end(), results_confidence.begin(), results_confidence.end());

//inserting into labels
labels.insert(labels.end(), results_labels.begin(), results_labels.end());

/*
#ifdef HANDLE_MEMORY_LEAK
//there is an memory leak in CNTK thats why reloading the model after each evaluation
//https://github.com/Microsoft/CNTK/issues/1838
m_CNTK_Loader->re_load_model();
#endif HANDLE_MEMORY_LEAK

#ifdef DEBUG
std::cout << "===== Done." << std::endl;
#endif
*/

return 1;

}

[MaximumProgrammer]

This thread can be closed, but the error still exists on my PC not on my colloge, perhabs same software and hardware.

[MaximumProgrammer]

This thread can be closes.