Closed pawartur closed 5 years ago
Do you get an error message?
Also see this: http://machinethink.net/blog/help-core-ml-gives-wrong-output/
Thanks again for such a quick response.
Actually, I wasn't specific enough describing what I'd done.
Here it is:
darknet detector train (...)
with my modified tiny-yolo-voc-1c.cfg file and a proper .data file that points to my ~400 pics as training pics and another ~50 pics as test picsdarknet detector test (...)
with the resulting .weights file on my test imagesdarknet detector test (...)
So it seems for me that the problem in my case is in the step that converts darknet to keras. I've gone through the yad2k.py script and didn't find anything that could be specific to the standard tiny-yolo-voc case that you used it for, but maybe there is something...
What do you think?
I'm quite new to ML and Vision, so maybe I messed sth. up. I would appreciate your help a lot :).
The yad2k script isn't mine so I don't really know the ins and outs, but if the test_yolo script already doesn't work then it's likely the conversion to Keras did not go right -- or the input images passed into the model in test_yolo are in a different format than the format you trained darknet on.
Do you know the author of yad2k script?
The link is in the readme.
@pawartur Did you find a solution? I have met the exactly the same issue as yours... My model works on the original darknet but doesn't work after converted to Keras model. Still couldn't find the reason for this.
Hi!
Unfortunately not. And I need to leave this for now, will probably get back to solving this in september.
Best of luck with solving this, I will probably ping you in a few weeks :-).
On 8 Aug 2017, at 02:17, zhenkaiwang notifications@github.com wrote:
@pawartur Did you find a solution? I have met the exactly the same issue as yours... My model works on the original darknet but doesn't work after converted to Keras model. Still couldn't find the reason for this.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
I'm having the exact same issue. The Darknet to Keras converter works fine when using the weights from https://pjreddie.com/darknet/yolo/. But not with our own weights. It breaks down on the exact same spot as with @pawartur (Darknet to Keras converter).
Also noticed that the pb file one can create from darkflow works perfectly on Android and Windows devices in our case. I've been looking at YAD2k to see what needs to change to fix this.
If anybody finds anything pls post a message.
I solve my problem by directly retraining a keras model through a modified retrain_yolo.py.
On 2017年9月11日, at 07:33, Richard Chirino notifications@github.com wrote:
I'm having the exact same issue. The Darknet to Keras converter works fine when using the weights from https://pjreddie.com/darknet/yolo/. But not with our own weights. It breaks down on the exact same spot as with @pawartur (Darknet to Keras converter). Also noticed that the pb file one can create from darkflow works perfectly on Android and Windows devices in our case. I've been looking at YAD2k to see what needs to change to fix this. If anybody finds anything pls post a message.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
That's an interesting solution, @zhenkaiwang :).
I haven't solved the problem yet, needed to leave it for now.
I also trained darknet with tiny-yolo-voc configuration for only one object. It works before conversion and detects the objects as expected but I cannot get it to work after converting to Keras. Did anybody solve how to make this work with a weight file from training a custom object and not the pre-trained weights that is provided?
@zhenkaiwang could you please give some guidelines how to retraining the model
@zhenkaiwang are you able to share your retrain_yolo.py please? Or nay guidance on how to retrain this model on different classes and images would be great.
I was finally able to use my custom darknet model with CoreML by doing the following:
I used darkflow (https://github.com/thtrieu/darkflow) to convert my model to a Tensorflow protobuf. Then I used Apple and Google's new Tensorflow to CoreML converter (https://developers.googleblog.com/2017/12/announcing-core-ml-support.html) to convert the model into an mlmodel. This conversion went well and allowed me to use my custom darknet model (tiny-yolo) in CoreML.
Important to keep in mind that my yolo model used BGR for the pixel order and expected a value from 0-1 so I also had to set the scale and is_bgr to true. My conversion script ended up looking like this:
import tfcoreml as tf_converter
tf_converter.convert(tf_model_path = 'yolo.pb',
mlmodel_path = 'yolo.mlmodel',
output_feature_names = ['output:0'],
is_bgr = True,
input_name_shape_dict = {'input:0' : [1, 416, 416, 3]},
image_input_names = ['input:0'],
image_scale = 1 / 255.0)
@richard-giantrobot great job ! could you let me know how many Fps your tiny yolo coreml is on real device thanks
@oishi89
I'm getting 30 fps on my iPhone X, and around 5 fps on an iPhone 6 Plus. I don't currently have other devices to test against.
@richard-giantrobot this is very promising! having same issues you are. 2 questions:
1) were you able to use Matthjis's ios code in this repo with little modification with the returned mlmodel from TF Convert? (obv class changes, input names etc)
2) in TFConvert (coreml), is that block above that you used ALL that you used? or did you have to do any stripping of ops? I struggled greatly using those TF scripts for my SSD-MobileNets model, made me strip and use Preproccessor/sub as my input name (which CoreML hates).
This is the most promising news ive heard yet! Thank you! b
@bmgdev Did you try apple's new open source Turi Create. Which runs a tiny yolo behind the scenes, and is very easy to retrain and export for iOS.
WOW @helloniklas! this is huge!! thank you so much for the link! now to test this puppy out... have you had successes with it yet?
@helloniklas thank so much for your sharing. have you had a chance to test a model trained by Turi on real devices ? As you mentioned the tiny yolo is being used under the hook so i think the performance is similar to tiny yolo model. It would be around 5 - 6 fps for IPhone 6 / 5se
@bmgdev @oishi89 yes works great. Managed to train my custom model on 1 class with only 30 images. And it’s still very accurate running at 22 FPS. They recommend at least 200 images for better results. But so far works really well. And super easy to use and train with. Lots of tools to visualise the test data to see what’s going on as well.
@helloniklas ill have to try that next, looking forward to diving in. I JUST this second got my model converted by way of Darkflow->TFCoreml and working in-app, so pretty happy. Turi next!
@helloniklas what kind of iphone you were testing ? is it iphone X or earlier ? 22fps is impressive
@helloniklas if you don’t mind could you share a quick video for showing the accuracy and performance. oishi89@gmail.com
@oishi89 it's running on an iPhone X, same fps as when I used a keras converted tiny yolo.
@bmgdev
were you able to use Matthjis's ios code in this repo with little modification with the returned mlmodel from TF Convert? (obv class changes, input names etc)
All I had to change was the number of classes, the labels and the name of the input and output tensors which darkflow automatically changes to input and output (in the predict function, I'm not running it trough the Vision API right now, it works but it looks slower).
in TFConvert (coreml), is that block above that you used ALL that you used? or did you have to do any stripping of ops? I struggled greatly using those TF scripts for my SSD-MobileNets model, made me strip and use Preproccessor/sub as my input name (which CoreML hates).
Yeah that was all I used, after loading the model in my app it ran correctly right away. I have no experience with SSD-MobileNets
so I don't know what difference that makes.
@richard-giantrobot I got my Model Converted to CoreML using your Methods, but now I can't seem to get it to run in the TinyYOLO-CoreML
Any ideas?
@RGravity
Yeah that was what I meant with I needed to change the input and output names. Should be like this (if you used darkflow):
if let output = try? model.prediction(input__0: image) {
return computeBoundingBoxes(features: output.output__0)
} else {
return []
}
@richard-giantrobot Thanks thats it 👍
@bmgdev @richard-giantrobot can you please share steps on how you converted your darknet tiny-yolo model from tensorflow (pb) to coreml?
I was able to convert my tiny-yolo model from darknet to tensorflow but can't seem to convert from tensorflow (.pb) to .mlmodel
I get the following error:
Traceback (most recent call last): File "converttest1.py", line 6, in
image_input_names = ['input:0']) File "/Users/tgwagner/fresh/tf-coreml/tfcoreml/_tf_coreml_converter.py", line 483, in convert predicted_probabilities_output=predicted_probabilities_output) File "/Users/tgwagner/fresh/tf-coreml/tfcoreml/_tf_coreml_converter.py", line 175, in _convert_pb_to_mlmodel ('Output feature cannot be a placeholder') AssertionError: Output feature cannot be a placeholder
when I run:
import tfcoreml as tf_converter
tf_converter.convert(tf_model_path = '/users/fresh/darkflow/built_graph/tiny-yolo-NFPA.pb',
mlmodel_path = 'my_model.mlmodel',
output_feature_names = 'output:0',
input_name_shape_dict = {'input:0':[1, 416, 416, 3]},
image_input_names = ['input:0'])
I also ran summarize on the *.pb model and here's the output:
=/users/fresh/darkflow/built_graph/tiny-yolo-NFPA.pb
Found 1 possible inputs: (name=input, type=float(1), shape=[?,416,416,3])
No variables spotted.
Found 1 possible outputs: (name=output, op=Identity)
Found 15770590 (15.77M) const parameters, 0 (0) variable parameters, and 0 control_edges
Op types used: 59 Const, 16 Mul, 9 Conv2D, 9 BiasAdd, 9 Pad, 8 Sub, 8 Maximum, 8 RealDiv, 6 MaxPool, 1 Placeholder, 1 Identity, 1 NoOp
Thanks!
@tgwagner you need to flow your graph. take your darknet tiny yolo model to darkflow, then you can run those same commands in tf_converter
look under: "Save the built graph to a protobuf file (.pb)"
this should be the only command you need:
flow --model lex-yolo.cfg --load lex-yolo.weights —savepb
then to tf convert with 'import tfcoreml as tf_converter'...
you're very close :) b
@bmgdev thanks for your quick reply! Yes, I've actually already flowed my model, that's where I got my *.pb file (originally weights). My issue is converting the pb file into the mlmodel file.
I used flow --model tiny-yolo-nfpa.cfg --load tiny-yolo-nfpa_6100.weights -savepb
How did you convert pb to mlmodel?
@tgwagner only other step I had in addition to yours is image scale.
import tfcoreml as tf_converter
tf_converter.convert(tf_model_path = 'lex-yolo.pb',
mlmodel_path = 'lex-yolo.mlmodel',
output_feature_names = ['output:0'],
input_name_shape_dict = {'input:0' : [1, 416, 416, 3]},
image_input_names = ['input:0'],
image_scale = 1 / 255.0)
based on your error "Output feature cannot be a placeholder", I assume this is an error in training?
Overview on placeholders here: https://learningtensorflow.com/lesson4/
and in tf convert script where its catching that placeholder:
# run through all placeholders
for op in OPS:
output_names = set([compat.as_bytes(x.name) for x in op.outputs])
if op.type == 'Placeholder':
# Handle placeholders -- all placeholders are inputs
assert not filter(output_names.__contains__, output_feature_names), \
('Output feature cannot be a placeholder')
.
.
.
Hopefully that gives you a little more to poke at.... b
actually @tgwagner, i think I see your error: check this line:
output_feature_names = 'output:0',
should be:
output_feature_names = ['output:0'],
:)
@bmgdev I did a clean install of coremltools and tfcoreml (using python 2.7 vs 3.6) it worked. Thanks for your help investigating. As an aside, I can't believe how fragmented this toolchain is.
@tgwagner I was just about to mention trying py3 instead of py2 - happy you got it. guys like @hollance gave me a pointer or two when I first started out, the more we can help each other the better.
The fragmentation has only begun. Am now trying to slim my model to a set of ops that TF Lite supports so I can start doing cross platform here. If you want more fun, try getting TF Mobile building on iOS... highly dependent on specific versions and quite fragile. Oh boy the fun never stops.
Best of luck on your journey.
@bmgdev cheers buddy, I hear you. I'm sure we'll be chatting a bunch more!
Had similar problems, the solution from @richard-giantrobot worked. Used darkflow to convert my custom yolo model to tf protobuf. then Used tfcoreml to convert to coreml model. One thing I have noticed that the darkflow model is less accurate, though. Darknet to keras using yad2k is much more accurate.
@richard-giantrobot @saikatbsk i have created the .pb file from darknet (it does produce a .meta) file too. when i am using @richard-giantrobot code for converting it into mlmodel , it is indeed creating a mlmodel file, basically a json i quess, but i have read that it should create shrad files too(assuming it is the weights file for the graph) . Can you please guide us that just a .*model file is enough . I will be using tensorflow.js.
@richard-giantrobot @saikatbsk I am trying to get the tiny-yolo-voc to work on my iPhone X. I am using this project: https://github.com/hollance/YOLO-CoreML-MPSNNGraph. It gives me ridiculous bounding box predictions. I tried to convert the Darknet to .pb and then convert to core ml using tfcoreml, I still get similar wrong predictions. Anybody has an idea what might be the reason it dint work for me?
@JiangkunLiu @akuma308 I was successful in converting a darknet YOLOv2 model to .pb and then to .mlmodel. But it didn't work for TinyYOLO. I had followed the instructions mentioned here: https://github.com/thtrieu/darkflow
Hi!
I have a darknet model that is based on tiny-yolo-voc and finds license plates pretty accurately (checked that many times with darknet's detector command).
Your conversion mechanism works pretty well for your model (just like you write in readme).
However, when I apply your conversion process to my darknet model, the resulting coreml model doesn't seem to be able to recognise anything.
Would you be kind enough to help me in any way?