Usability issues

[salvacarrion]

Problem description

(All of these "issues" are not issues per se, but a set of suggestions to improve the usability of the EDDL)

Use case: I'm trying to import an onnx model from scratch to perform a single prediction but I have faced some usability issues that the average user (non-developer) won't be able to easily solve.

Issues: Marked in the code below

• Issue 1: Why is device needed? Nothing. We should remove this parameter.
• Issue 6: build(...) should not care about the initialization of the net: If you use import_net_from_onnx_file, cool! Pre-trained weights are loaded. And if you use Model(...), cool too! Weights are initialized. But the build function (aka. compile) should only copy tensors from the CPU to the corresponding device.
• Issue 7: On the build(...), the optimizer, losses, and metrics are only needed for training/testing but not for inference: This is a minor issue as we can ignore them, but it will be cool to simply write build(net,CS_GPU({1}))` when we only care about inference.

Issues fixed:

• ~Issue 2: net->summary() not working before build. How do I know the name of input/output layers?~
• ~Issue 3: net->summary() exists but net->plot() doesn't? For the sake of consistency.~
• ~Issue 4: Which extension/s does eddl::plot(net, "file") support/expect? They are not mentioned in the documentation. Also, should we leave a default output name for "speed" and so that the end user can know the extension that is expected? (.jpg, .svg, .eps, .pdf,...)~
• ~Issue 5: How do I know the input/output names of the model if I cannot print its layers? I've tried Netron and I have still no idea about how to get the layer names~
• ~Issue 8: New function to predict top-k classes with probabilities and class names: We could add a method to load the class names from a vector of strings or a text file, and another method to print top-k classes with their probabilities.~
• ~Issue 9: Missing Tensor broadcasting. It is usually needed (here) when we need to apply a specific preprocessing for a given network~
• ~Issue 10: Add example to the documentation with pre-trained models~

Example code

Preprocessing function:

Download the ResNet34 from here

    // Set default vars
    string image_fname = "../../examples/data/elephant.jpg";
    string class_names_file = "../../examples/data/imagenet_class_names.txt";
    string model_path = "resnet34-v1-7.onnx";

    // Specific
    int in_channels = 3;
    int in_height = 224;
    int in_width = 224;

    ************************* ISSUE #1 *************************
    std::cout << "Importing ONNX..." << std::endl;
    Net *net = import_net_from_onnx_file(model_path, {in_channels, in_height, in_width}, DEV_CPU);  // Why is the device needed?
    ***************************************************************

    ************************* ISSUE #2 (fixed) *************************
    net->summary();  // Not working before the build. How do I know the layers' name?
    ***************************************************************

    ********************** ISSUE #3 (fixed)  and #4 (fixed( *********************
     net->plot("default.pdf", "LR");   // Not intuitive. Which extension? What is "LR"?
    ***************************************************************

    ************************* ISSUE #5 (fixed) *************************
    // Optional: This model does not include a softmax layer, so we need to add it to get the final probabilities
    // Get input/output + Add softmax
    layer in = net->lin[0];   // getLayer(net,"input_layer_name");
    layer l = net->lout[0];   // getLayer(net,"output_layer_name");
    layer out = Softmax(l);

    // Create model
    net = Model({in},{out});
    // **********************************************************************************************

    ******************* ISSUE #6 and #7 **********************
    // Build model
    build(net,
          adam(0.001), // Optimizer
          {"softmax_cross_entropy"}, // Losses
          {"categorical_accuracy"}, // Metrics
          CS_GPU({1}), // one GPU
          false       // Parameter that indicates that the weights of the net must not be initialized to random values.
    );
    ***************************************************************

     // Load test image
    Tensor *image = Tensor::load(image_fname);

    // Preprocess input. (Depends on the model)
    Tensor* image_preprocessed = preprocess_input_resnet34(image, {in_height, in_width});

    // Predict image
    vector<Tensor*> outputs = net->predict({image_preprocessed});

    ************************* ISSUE #8 (fixed) *************************
    // Read class names from txt file
    std::cout << "Reading class names..." << std::endl;
    vector<string> class_names = eddl::read_txt_file(class_names_file);

    // Show top K predictions
    int top_k = 5;
    std::cout << "Top " << top_k << " predictions:" << std::endl;
    std::cout << eddl::get_topk_predictions(outputs[0], class_names, top_k)  << std::endl;
    ***************************************************************

Preprocessing function:

Tensor* preprocess_input_resnet34(Tensor* input, const vector<int> &target_size){
    // Preprocess image (depends on the model) ***************************
    // Values from: https://github.com/onnx/models/tree/master/vision/classification/resnet

    auto* mean_vec = new Tensor( {0.485, 0.456, 0.406}, {3, 1}, input->device);
    auto* std_vec = new Tensor( {0.229, 0.224, 0.225}, {3, 1}, input->device);

    // Check dimension. Input must be a 3D or 4D tensor
    if(!(input->ndim == 3 || input->ndim == 4)){
        throw std::runtime_error("A 3D or 4D tensor is expected. " + std::to_string(input->ndim) + "D tensor received.");
    }

    // Convert from 3D to 4D (if needed)
    if(input->ndim == 3){
        input->unsqueeze_(0);
    }

    // Resize tensor (creates a new instance)
    Tensor* new_input = input->scale(target_size);  // (height, width)

    // Scale to range [0..1]
    new_input->mult_(1/255.0f);

    // Normalize: (X-mean)/std
    Tensor* mean = Tensor::broadcast(mean_vec, new_input);
    Tensor* std = Tensor::broadcast(std_vec, new_input);
    new_input->sub_(mean);
    new_input->div_(std);

    delete mean_vec;
    delete std_vec;
    delete mean;
    delete std;

    return new_input;
}