European Distributed Deep Learning (EDDL) library. A general-purpose library initially developed to cover deep learning needs in healthcare use cases within the DeepHealth project.
(All of these "issues" are not issues per se, but a set of suggestions to improve the usability of the EDDL)
Use case:
I'm trying to import an onnx model from scratch to perform a single prediction but I have faced some usability issues that the average user (non-developer) won't be able to easily solve.
Issues: Marked in the code below
Issue 1: Why is device needed? Nothing. We should remove this parameter.
Issue 6: build(...) should not care about the initialization of the net: If you use import_net_from_onnx_file, cool! Pre-trained weights are loaded. And if you use Model(...), cool too! Weights are initialized. But the build function (aka. compile) should only copy tensors from the CPU to the corresponding device.
Issue 7: On the build(...), the optimizer, losses, and metrics are only needed for training/testing but not for inference: This is a minor issue as we can ignore them, but it will be cool to simply write build(net,CS_GPU({1}))` when we only care about inference.
Issues fixed:
~Issue 2: net->summary() not working before build. How do I know the name of input/output layers?~
~Issue 3: net->summary() exists but net->plot() doesn't? For the sake of consistency.~
~Issue 4: Which extension/s does eddl::plot(net, "file") support/expect? They are not mentioned in the documentation. Also, should we leave a default output name for "speed" and so that the end user can know the extension that is expected? (.jpg, .svg, .eps, .pdf,...)~
~Issue 5: How do I know the input/output names of the model if I cannot print its layers? I've tried Netron and I have still no idea about how to get the layer names~
~Issue 8: New function to predict top-k classes with probabilities and class names: We could add a method to load the class names from a vector of strings or a text file, and another method to print top-k classes with their probabilities.~
~Issue 9: Missing Tensor broadcasting. It is usually needed (here) when we need to apply a specific preprocessing for a given network~
~Issue 10: Add example to the documentation with pre-trained models~
// Set default vars
string image_fname = "../../examples/data/elephant.jpg";
string class_names_file = "../../examples/data/imagenet_class_names.txt";
string model_path = "resnet34-v1-7.onnx";
// Specific
int in_channels = 3;
int in_height = 224;
int in_width = 224;
************************* ISSUE #1 *************************
std::cout << "Importing ONNX..." << std::endl;
Net *net = import_net_from_onnx_file(model_path, {in_channels, in_height, in_width}, DEV_CPU); // Why is the device needed?
***************************************************************
************************* ISSUE #2 (fixed) *************************
net->summary(); // Not working before the build. How do I know the layers' name?
***************************************************************
********************** ISSUE #3 (fixed) and #4 (fixed( *********************
net->plot("default.pdf", "LR"); // Not intuitive. Which extension? What is "LR"?
***************************************************************
************************* ISSUE #5 (fixed) *************************
// Optional: This model does not include a softmax layer, so we need to add it to get the final probabilities
// Get input/output + Add softmax
layer in = net->lin[0]; // getLayer(net,"input_layer_name");
layer l = net->lout[0]; // getLayer(net,"output_layer_name");
layer out = Softmax(l);
// Create model
net = Model({in},{out});
// **********************************************************************************************
******************* ISSUE #6 and #7 **********************
// Build model
build(net,
adam(0.001), // Optimizer
{"softmax_cross_entropy"}, // Losses
{"categorical_accuracy"}, // Metrics
CS_GPU({1}), // one GPU
false // Parameter that indicates that the weights of the net must not be initialized to random values.
);
***************************************************************
// Load test image
Tensor *image = Tensor::load(image_fname);
// Preprocess input. (Depends on the model)
Tensor* image_preprocessed = preprocess_input_resnet34(image, {in_height, in_width});
// Predict image
vector<Tensor*> outputs = net->predict({image_preprocessed});
************************* ISSUE #8 (fixed) *************************
// Read class names from txt file
std::cout << "Reading class names..." << std::endl;
vector<string> class_names = eddl::read_txt_file(class_names_file);
// Show top K predictions
int top_k = 5;
std::cout << "Top " << top_k << " predictions:" << std::endl;
std::cout << eddl::get_topk_predictions(outputs[0], class_names, top_k) << std::endl;
***************************************************************
Preprocessing function:
Tensor* preprocess_input_resnet34(Tensor* input, const vector<int> &target_size){
// Preprocess image (depends on the model) ***************************
// Values from: https://github.com/onnx/models/tree/master/vision/classification/resnet
auto* mean_vec = new Tensor( {0.485, 0.456, 0.406}, {3, 1}, input->device);
auto* std_vec = new Tensor( {0.229, 0.224, 0.225}, {3, 1}, input->device);
// Check dimension. Input must be a 3D or 4D tensor
if(!(input->ndim == 3 || input->ndim == 4)){
throw std::runtime_error("A 3D or 4D tensor is expected. " + std::to_string(input->ndim) + "D tensor received.");
}
// Convert from 3D to 4D (if needed)
if(input->ndim == 3){
input->unsqueeze_(0);
}
// Resize tensor (creates a new instance)
Tensor* new_input = input->scale(target_size); // (height, width)
// Scale to range [0..1]
new_input->mult_(1/255.0f);
// Normalize: (X-mean)/std
Tensor* mean = Tensor::broadcast(mean_vec, new_input);
Tensor* std = Tensor::broadcast(std_vec, new_input);
new_input->sub_(mean);
new_input->div_(std);
delete mean_vec;
delete std_vec;
delete mean;
delete std;
return new_input;
}
Problem description
(All of these "issues" are not issues per se, but a set of suggestions to improve the usability of the EDDL)
Use case: I'm trying to import an onnx model from scratch to perform a single prediction but I have faced some usability issues that the average user (non-developer) won't be able to easily solve.
Issues: Marked in the code below
device
needed? Nothing. We should remove this parameter.build(...)
should not care about the initialization of the net: If you useimport_net_from_onnx_file
, cool! Pre-trained weights are loaded. And if you useModel(...)
, cool too! Weights are initialized. But thebuild
function (aka.compile
) should only copy tensors from the CPU to the corresponding device.build(...)
, the optimizer, losses, and metrics are only needed for training/testing but not for inference: This is a minor issue as we can ignore them, but it will be cool to simply writebuild(net,
CS_GPU({1}))` when we only care about inference.Issues fixed:
net->summary()
not working before build. How do I know the name of input/output layers?~eddl::plot(net, "file")
support/expect? They are not mentioned in the documentation. Also, should we leave a default output name for "speed" and so that the end user can know the extension that is expected? (.jpg, .svg, .eps, .pdf,...)~Example code
Preprocessing function:
Preprocessing function: