Closed jef5ez closed 6 years ago
@turambar Anything obvious here?
I tried removing the dropouts and changing the dim ordering with similar results It looks like the weights in the last dense layer align but the values in the first convolution layer do not.
https://gist.github.com/jef5ez/f17f3d98fb2c52e8b336b074fe0c55ce I also added a warning I was getting about about Vfs.Dir not being created.
It looks like the weights in the last dense layer align but the values in the first convolution layer do not.
Disregard that, after digging around there are comments saying theano does rotation in the conv net filters. If the comments here are right then the weights do seem to be getting read correctly
Yes,theano rotates the filters. It by default does a correlation. But the python code in the gist says dim ordering is tensor flow?
@jef5ez @eraly yes, be very careful about dim_ordering
(called image_data_format
in Keras 2) and choice of backend. The first thing to check is that the shape of the outputs of each convolutional layer match (across DL4J and Keras -- shape orders might be different, but the sizes should be the same).
Where I've previously encountered problems is "flatten" operations between convolutional and dense layers -- we added specialized preprocessors to handle this, but there may be edge cases we've missed.
yes the weights look to be loading correctly but I'm still getting inconsistent results between python and java
@jef5ez this seems to have been reported in https://github.com/deeplearning4j/deeplearning4j/issues/3659 as well. Let me try to investigate this and come back to you. thanks
@jef5ez this might be interesting for you as well. I ran this with theano backend and the output matches. I assume you created your keras 1 model with TF, which has different dim_ordering
.
https://gist.github.com/maxpumperla/1d3c67f8c787bdb534c462c314839e63
Correct, this was using Tensorflow backend
OK, that's good to hear. while the results for theano don't match 100%, at least they roughly align in this case (for now I assume some numerical deviation due to diff. implementations). For TF they seem completely off.
Hi,
I'm having what seems to be a similar issue. For some reason the output of a model made in Keras only outputs 1. I have been struggling to find the reason but the same model imported into R loading the same images seems to work.
DL4J Code:
restoredCNN = KerasModelImport.importKerasModelAndWeights(modelFile.getAbsolutePath(),false)
Java2DNativeImageLoader loader = new Java2DNativeImageLoader(tileSize, tileSize,3)
INDArray imageArray = loader.asMatrix(img,true);
imageArray.divi(255);
imageArray.subi(0.5);
imageArray.muli(2);
INDArray output = restoredCNN.outputSingle(imageArray)
output = [[1.0000]]
the img
is a RGB BuffuredImage (TYPE_INT_RGB). I have played around with the Image Loader and that one is the only one that seems to get the channels right.
R Code:
model.load2 <- load_model_hdf5("xception-210418.h5")
test.img_raw <- image_load("Tile 1.tif",)
test.img_array <- image_to_array(test.img_raw)
test.img_scale <- xception_preprocess_input(test.img_array)
dim(test.img_scale) <- c(1,224,224,3)
output <- predict(model.load2,test.img_scale)
output = 0.1193942
I have manually looked at many of the weights loaded in DL4J which seem to be correct. The ordering of the channels when the Keras model is loaded into DL4J seems to be channels first and same goes for the INDarray returned by as.Matrix - (3,224,224), whereas in R the weights and image have channels last. I'm not sure this is the issue since it is consistent, but I also am not sure exactly why that's the case. I have started to manually look at the output of the first few layers and they do seem to match up but are very slow to compare since they are stored in different array dimensions in the two programs.
I have tried both inception V3 and xception models.
Keras Version: 2.1.4 (TF backend) DL$J Version 1.0.0 - SNAPSHOT
After looking a little deeper today I notice that discrepancies begin to be noticeable (to the first few decimal points), but are very small after the first couple of conv layers:
block1_conv2 (dl4j): 1.5889398, 2.3035548, 2.6691766, 2.148596 block1_conv2 (Keras R): 1.589807, 2.303508, 2.667844, 2.148716
block1_conv2_act (dl4j): 0.80208445, 1.040886, 1.1630652, 0.9891038 block1_conv2_act (Keras R): 0.8024021 ,1.040883, 1.162625, 0.9891598
But somehow between the block1_conv2 and the block1_sepconv1, the discrepancies jump considerably
block2_sepconv1(dl4j): 1.2818, 3.1922, 2.7721 block2_sepconv1(Keras R): 0.3427208, -1.18535, -0.6530807
@maxpumperla I certainly don't mean to pester/rush you at all but wanted to double check that you had seen this - I wasn't completely sure if it was better to comment here or make a new issue. Let me know if any other clues would help get to the bottom of this.
@ClnSchlssr that's fine, thanks for reporting. One of the reasons I haven't looked into this more closely is that
a) we have unit tests in keras import that check that weights are loaded correctly b) we have gradient checks for networks after import
and right now all the tests pass. Now, I'm definitely not saying everything works, but I don't know how to boil this down to anything I can reproduce. It might just be that a border mode is mapped incorrectly - not sure.
So far all people reporting such issues have seen discrepancies in the millionth layer. It is possible that 5th digit rounding errors or numerical discrepancies add up to a lot.
What I would like to see is a notable discrepancy in one layer. Any chance you can isolate your sepconv layer?
Yes that all sounds reasonable. I will keep probing to see if anything more concrete comes up. Is there any documentation about how sepconv layers are dealt with? One thing I am noticing is that the shape is very flat in dl4j, but I'm not sure if that's a bug or just the way they are implemented. For instance:
block2_sepconv1_bn_layer$weights
[[1]]
Variable(shape=(3, 3, 64, 1), dtype=float32_ref)
[[2]]
Variable(shape=(1, 1, 64, 128), dtype=float32_ref)
dl4j shapeInformation for the parameters in block2_sepconv1
[2,1,8768,0,1,0,1,99]
The handful of weights I've looked through in both layers are identical so given the weird output of this layer, the shape mismatch seems suspicious to me.
Instead of looking at the shapeinformation field, take a look at the output of the shape() method, it may be quite different.
Actually yeah that's my fault. All of the convolution layers seem to be equally flattened when looking at the parameter shape. The output shape of that layer seems to be ok so I will have to compare the parameters more closely. Possibly run the layer manually myself...
@ClnSchlssr if you find configs that differ between keras and dl4j after import, that'd be very helpful to know.
closing this one in favour of #4756
@ClnSchlssr just an effort to clean up and organise issues, I'm still looking into this -- and would still be very much interested in further hints.
Thanks @maxpumperla. I will keep an eye on the other issue and let you know if I make any discoveries on my end.
@ClnSchlssr we've just fixed a keras mapping issue (depth multiplier wasn't picked up properly) for sepconv, might be worth checking this again on your side, using snapshots.
I gave it a shot but I'm still having some issues. I'm noticing now a bunch of the following messages are output when the model is loaded. It looks like mostly activation layers. Is that any useful clue?
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
000: C:\autotest\hdf5110-StdRelease-15:44:01.758 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block1_conv2_act, continuing import.
15:44:01.766 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block2_sepconv2_act, continuing import. 15:44:01.773 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block3_sepconv1_act, continuing import. code-10vs14\build\hdfsrc\src\H5G.c line 472 in H5Gopen2(): unable to open group major: Symbol table minor: Can't open object
001: C:\autotest\hdf5110-StdRelease-code-10vs14\build\hdfsrc\src\H5Gint.c line 287 in H5G__open_name(): group not found
major: Symbol table minor: Object not found
002: C:\autotest\hdf5110-StdRelease-code-10vs14\build\hdfsrc\src\H5Gloc.c line 428 in H5G_loc_find(): can't find object
major: Symbol table minor: Object not found
003: C:\autotest\hdf5110-StdRelease-code-10vs14\build\hdfsrc\src\H5Gtraverse.c line 867 in H5G_traverse(): internal path traversal failed
major: Symbol table minor: Object not found
004: C:\autotest\hdf5110-StdRelease-code-10vs14\build\hdfsrc\src\H5Gtraverse.c line 639 in H5G_traverse_real(): traversal operator failed
major: Symbol table minor: Callback failed
005: C:\autotest\hdf5110-StdRelease-code-10vs14\build\hdfsrc\src\H5Gloc.c line 383 in H5G_loc_find_cb(): object 'block1_conv2_act' doesn't exist
major: Symbol table minor: Object not found
Colin
@maxpumperla when importing xception models made in keras I still get this set of warnings and errors and models are still behaving funny (in the most recent case outputs hover around some arbitrary level output of 0.36. Do these mean anything in particular to you? If it seems like the import code is working on tests I would really appreciate a nudge in the right direction on this. As before the same model can be imported into R and makes reasonable predictions.
17:56:23.293 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing quote for a string value at [Source: java.io.StringReader@343e841e; line: 1, column: 36001] 17:56:23.298 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing '"' for name at [Source: java.io.StringReader@2d835a41; line: 1, column: 40001] 17:56:23.309 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing '"' for name at [Source: java.io.StringReader@41a13d1c; line: 1, column: 44001] 17:56:23.314 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input within/between ARRAY entries at [Source: java.io.StringReader@1ab0f5f; line: 1, column: 48001] 17:56:23.321 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing quote for a string value at [Source: java.io.StringReader@41757500; line: 1, column: 52001] 17:56:23.325 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: expected close marker for ARRAY (from [Source: java.io.StringReader@76137e3d; line: 1, column: 43982]) at [Source: java.io.StringReader@76137e3d; line: 1, column: 56001] 17:56:23.330 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input in FIELD_NAME at [Source: java.io.StringReader@3e92e43e; line: 1, column: 60001] 17:56:23.349 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing '"' for name at [Source: java.io.StringReader@679ad544; line: 1, column: 64001] 17:56:23.359 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: expected close marker for OBJECT (from [Source: java.io.StringReader@dad71c2; line: 1, column: 49929]) at [Source: java.io.StringReader@dad71c2; line: 1, column: 68001] 17:56:23.364 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing '"' for name at [Source: java.io.StringReader@60a214dc; line: 1, column: 72001] 17:56:23.369 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing '"' for name at [Source: java.io.StringReader@4938bfe2; line: 1, column: 76001] 17:56:23.379 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing '"' for name at [Source: java.io.StringReader@78b97374; line: 1, column: 80001] 17:56:23.388 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing '"' for name at [Source: java.io.StringReader@1e7c2ec0; line: 1, column: 84001] 17:56:23.393 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing quote for a string value at [Source: java.io.StringReader@6e08dd27; line: 1, column: 88001] 17:56:23.398 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing quote for a string value at [Source: java.io.StringReader@6243b8ab; line: 1, column: 92001] 17:56:23.406 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing quote for a string value at [Source: java.io.StringReader@51525544; line: 1, column: 96001] 17:56:23.411 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input in FIELD_NAME at [Source: java.io.StringReader@2ea76b80; line: 1, column: 68001] 17:56:23.417 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing quote for a string value at [Source: java.io.StringReader@66e6a96; line: 1, column: 72001] 17:56:23.422 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing '"' for name at [Source: java.io.StringReader@79170c10; line: 1, column: 76001] 17:56:23.444 [JavaFX Application Thread] [INFO ] o.d.nn.modelimport.keras.Hdf5Archive - Unexpected end-of-input: was expecting closing '"' for name at [Source: java.io.StringReader@3aac9e03; line: 1, column: 80001] 17:56:25.319 [JavaFX Application Thread] [WARN ] org.nd4j.linalg.factory.Nd4jBackend - Skipped [JCublasBackend] backend (unavailable): java.lang.UnsatisfiedLinkError: C:\Users\cschlosser.javacpp\cache\cuda-9.0-7.0-1.3-windows-x86_64.jar\org\bytedeco\javacpp\windows-x86_64\jnicuda.dll: Can't find dependent libraries 17:56:25.361 [JavaFX Application Thread] [INFO ] org.nd4j.linalg.factory.Nd4jBackend - Loaded [CpuBackend] backend 17:56:29.878 [JavaFX Application Thread] [INFO ] org.nd4j.nativeblas.NativeOpsHolder - Number of threads used for NativeOps: 4 17:56:31.066 [JavaFX Application Thread] [INFO ] org.nd4j.nativeblas.Nd4jBlas - Number of threads used for BLAS: 4 17:56:31.084 [JavaFX Application Thread] [INFO ] o.n.l.a.o.e.DefaultOpExecutioner - Backend used: [CPU]; OS: [Windows 7] 17:56:31.086 [JavaFX Application Thread] [INFO ] o.n.l.a.o.e.DefaultOpExecutioner - Cores: [4]; Memory: [2.7GB]; 17:56:31.088 [JavaFX Application Thread] [INFO ] o.n.l.a.o.e.DefaultOpExecutioner - Blas vendor: [OPENBLAS] 17:56:31.355 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block10_sepconv1_act, continuing import. 17:56:31.384 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block10_sepconv2_act, continuing import. 17:56:31.412 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block10_sepconv3_act, continuing import. HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
17:56:31.471 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block11_sepconv2_act, continuing import. 17:56:31.496 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block11_sepconv3_act, continuing import. 17:56:31.521 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block12_sepconv1_act, continuing import. tdRelease-code-10vs14\build\hdfsrc\src\H5Gtraverse.c line 867 in H5G_traverse(): internal path traversal failed major: Symbol table minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) t17:56:31.547 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block12_sepconv2_act, continuing import. 17:56:31.571 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block12_sepconv3_act, continuing import. 17:56:31.597 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block13_sepconv1_act, continuing import. hread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
17:56:31.692 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block14_sepconv1_act, continuing import. 17:56:31.801 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block14_sepconv2_act, continuing import. 17:56:31.812 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block1_conv1_act, continuing import. averse.c line 867 in H5G_traverse(): internal path traversal failed major: Symbol table minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
17:56:31.828 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block2_sepconv2_act, continuing import. code-10vs14\build\hdfsrc\src\H5G.c line 472 in H5Gopen2(): unable to open group major: Symbol table minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
17:56:31.847 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block3_sepconv2_act, continuing import. 17:56:31.860 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block4_sepconv1_act, continuing import. 17:56:31.884 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block4_sepconv2_act, continuing import. 17:56:31.908 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block5_sepconv1_act, continuing import. versal failed major: Symbol table minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
17:56:31.956 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block5_sepconv3_act, continuing import. 17:56:31.985 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block6_sepconv1_act, continuing import. 2(): unable to open group major: Symbol table minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Ob17:56:32.009 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block6_sepconv2_act, continuing import.
17:56:32.038 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block6_sepconv3_act, continuing import. 17:56:32.063 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block7_sepconv1_act, continuing import. 17:56:32.089 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block7_sepconv2_act, continuing import. ject not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
17:56:32.117 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block7_sepconv3_act, continuing import. 17:56:32.142 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block8_sepconv1_act, continuing import. 17:56:32.168 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block8_sepconv2_act, continuing import. minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Callback failed
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.1) thread 0:
major: Symbol table
minor: Can't open object
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
major: Symbol table
minor: Object not found
17:56:32.224 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block9_sepconv1_act, continuing import. 17:56:32.265 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block9_sepconv2_act, continuing import. 17:56:32.289 [JavaFX Application Thread] [WARN ] o.d.n.m.keras.utils.KerasModelUtils - No HDF5 group with weights found for layer with name block9_sepconv3_act, continuing import. 17:56:32.491 [JavaFX Application Thread] [INFO ] o.d.nn.graph.ComputationGraph - Starting ComputationGraph with WorkspaceModes set to [training: ENABLED; inference: ENABLED], cacheMode set to [NONE]
@ClnSchlssr Please open another issue including a model that causes these errors. Thanks!
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Keras v. 1.2.2 dl4j 0.8.0
Tried making conv net as in this gist: https://gist.github.com/jef5ez/fbe9e181c7132078d4f3a0855801be80
The dl4j model does not produce the same outputs and required that the input shape be changed to run (channels dimension last vs first).
Saved model attached model.zip