Using lf-net in OpenCV dnn

[gpcbitnik]

Hello, i try to freeze your model in order to use it with opencv dnn module to test it in my application. i frozen the model using this lines :

` input_graph_def = sess.graph.as_graph_def()

output_node_names=ops['kpts'].op.name+","+ops['feats'].op.name+","+ops['scale_maps'].op.name+","+ops['kpts_scale'].op.name+","+ops['degree_maps'].op.name+","+ops['kpts_ori'].op.name
output_graph_def = graph_util.convert_variables_to_constants(
        sess, # The session
        input_graph_def, # input_graph_def is useful for retrieving the nodes 
        output_node_names.split(",")  
)

output_graph="export/frozen.pb"
with tf.gfile.GFile(output_graph, "wb") as f:
    f.write(output_graph_def.SerializeToString())
tf.train.write_graph(output_graph_def, 'export/', 'frozentxt.pbtxt',as_text=True)`

and then i use the thoses lines to optimze the networks

`with tf.gfile.FastGFile(output_graph, "rb") as f:
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(f.read())
    tf.summary.FileWriter('logs', graph_def)

    inp_node = 'Placeholder'
    out_node =  output_node_names.split(",")
    graph_def = optimize_for_inference_lib.optimize_for_inference(graph_def, [inp_node], out_node,
                                                                tf.float32.as_datatype_enum)
    graph_def = TransformGraph(graph_def, [inp_node], out_node, ["sort_by_execution_order"])

    tf.train.write_graph(graph_def, "export/", 'graph_opt.pbtxt', as_text=True)`

but i get some warnings

WARNING:tensorflow:Didn't find expected Conv2D input to 'MSDeepDet/ConvOnlyResNet/block-1/pre-bn/FusedBatchNorm'

WARNING:tensorflow:Didn't find expected Conv2D input to 'MSDeepDet/ConvOnlyResNet/block-1/mid-bn/FusedBatchNorm'

WARNING:tensorflow:Didn't find expected Conv2D input to 'MSDeepDet/ConvOnlyResNet/block-2/pre-bn/FusedBatchNorm'

WARNING:tensorflow:Didn't find expected Conv2D input to 'MSDeepDet/ConvOnlyResNet/block-2/mid-bn/FusedBatchNorm'

WARNING:tensorflow:Didn't find expected Conv2D input to 'MSDeepDet/ConvOnlyResNet/block-3/pre-bn/FusedBatchNorm'

WARNING:tensorflow:Didn't find expected Conv2D input to 'MSDeepDet/ConvOnlyResNet/block-3/mid-bn/FusedBatchNorm'

WARNING:tensorflow:Didn't find expected Conv2D input to 'MSDeepDet/ConvOnlyResNet/fin-bn/FusedBatchNorm'

WARNING:tensorflow:Didn't find expected Conv2D input to 'SimpleDesc/bn1/FusedBatchNorm'

WARNING:tensorflow:Didn't find expected Conv2D input to 'SimpleDesc/bn2/FusedBatchNorm'

WARNING:tensorflow:Didn't find expected Conv2D input to 'SimpleDesc/bn3/FusedBatchNorm'

thoses nodes that generate warnings are the nodes with ConvOnlyRestNet as an input. this bloc have no relation with the placeholder node so how can it get the input blob?

there is something i'm not understanding, could you help me please? kind regards

[kmyi]

Oh, this is so cool. I suspect the main reason would be tensorflow version issue. There's also a docker image available. Have you tried them?

[gpcbitnik]

Hi, Thank you to take time to answer me :) I'm not sur that is a tensorflow issue. I succesfully ran the run_lfnet.py and got the image result (as explain in your readme) and you provide a requirements.txt with the version (thank's a lot for that!) I think it's just because i am a begginner with tensorflow and i'm not understanding your model architectur and all what am i doing. When i display the original graph in tensorboard, i see a subgraph named ConvOnlyResnet with the node Save as input and MSDeepDet as one of his output. In MSDeepDet subgraph, i see an other subgraph call ConvOnlyRestnet and each node of that one take as input the first ConvOnlyResnet. Is it a training trick or something like that? or there is 2 subgraph with that name? the first ConvOnlyRestnet is also not linked to the placeholder node (the input node, am i wrong?) so, how can it be evolve during the training process? [image: originalGraph.png]

When i display de frozen model, they are still the 2 subgraph, but the first one has no input (and it is not linked to the placeholder nodes) Can you explain the aims of the first convonlyresnet? frozen graph image : [image: frozentxt.png] kind regards

Le mer. 4 déc. 2019 à 18:35, Kwang Moo Yi notifications@github.com a écrit :

Oh, this is so cool. I suspect the main reason would be tensorflow version issue. There's also a docker image available. Have you tried them?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/vcg-uvic/lf-net-release/issues/17?email_source=notifications&email_token=AKNUOCLMYOOZC4MUOXHAMIDQW7S75A5CNFSM4JVKOBZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF534VI#issuecomment-561757781, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKNUOCJWALK2DDVM254WK6TQW7S75ANCNFSM4JVKOBZQ .

[kmyi]

Hi, it seems your image decided to go to the void :-( What I suspect is that these could be remnants of our hasty development, but I am not entirely sure. @xyukiono Can you perhaps try to chime in?

[gpcbitnik]

try to add the image again frozen Graph original Graph