deephealthproject / eddl

European Distributed Deep Learning (EDDL) library. A general-purpose library initially developed to cover deep learning needs in healthcare use cases within the DeepHealth project.
https://deephealthproject.github.io/eddl/
MIT License
34 stars 10 forks source link

Model merging leads to segmentation faults #190

Closed simleo closed 4 years ago

simleo commented 4 years ago

In the current develop branch, the mnist_auto_encoder_merging example breaks with a segmentation fault:

./build/bin/mnist_auto_merging
[...]
Generating Random Table
Building model
Segmentation fault (core dumped)

In eddl 0.6.0 the example runs fine, but only because there are no deletes. Deleting the merged net and one of the components leads to segmentation faults. E.g.:

#include <eddl/apis/eddl.h>

using namespace eddl;

int main() {
    layer in = Input({784});
    layer l = in;
    l = Activation(Dense(l, 256), "relu");
    layer out = Activation(Dense(l, 64), "relu");
    model m1 = Model({in}, {out});

    in = Input({64});
    l = Activation(Dense(in, 128), "relu");
    out = Sigmoid(Dense(l, 784));
    model m2 = Model({in}, {out});

    model merged = Model({m1, m2});

    build(merged,
          adam(0.0001),
          {"mse"},
          {"dice"},
          CS_GPU({1}));

    delete merged;                                                           
    delete m1;                                                               
}

This breaks any pyeddl program that tries to use the model merging function.

Things seem to be even worse in the current develop branch, since the program segfaults even with no deletes at all (as seen above for the original example).

RParedesPalacios commented 4 years ago

Yes I am aware will be fixed soon

RParedesPalacios commented 4 years ago

Fixed, please check to within the new Master version

simleo commented 4 years ago

I still get a segmentation fault after deleting the models. Checked with current master, with the same code as above, only difference is that I'm using a CS_CPU(). Switching the order of delete merged and delete m1 makes no difference, but if I delete only one of them it does not segfault.

The output is:

Generating Random Table
Building model
Building model
Building model
Segmentation fault (core dumped)
RParedesPalacios commented 4 years ago

Simone ok i will recheck that, thanks

RParedesPalacios commented 4 years ago

We have checked together with Simone. Now in develop branch is solved. I close this.