search

xmartlabs / Bender

Easily craft fast Neural Networks on iOS! Use TensorFlow models. Metal under the hood.
https://xmartlabs.github.io/Bender/
MIT License
1.8k stars 90 forks source link

Iphone 6 plus style transfer example not working on iOS 11 #42

Closed AndreJFBico closed 7 years ago

AndreJFBico commented 7 years ago

(SOLVED:) updating to latest xcode beta version solved the issue.

(UPDATE:) I just realised i was testing on iOS 11, the example runs ok with iOS 10 with the latest pull request branch. https://github.com/xmartlabs/Bender/pull/39

However Im still posting this as an issue for iOS 11.

Xcode version: 8.3.2

This is the console output: ALL TEST PASSED "Unknown input: ^moments/sufficient_statistics/mean_ss" "Unknown input: ^moments/sufficient_statistics/var_ss" "Unknown input: ^moments_1/sufficient_statistics/mean_ss" "Unknown input: ^moments_1/sufficient_statistics/var_ss" "Unknown input: ^moments_2/sufficient_statistics/mean_ss" "Unknown input: ^moments_2/sufficient_statistics/var_ss" "Unknown input: ^moments_3/sufficient_statistics/mean_ss" "Unknown input: ^moments_3/sufficient_statistics/var_ss" "Unknown input: ^moments_4/sufficient_statistics/mean_ss" "Unknown input: ^moments_4/sufficient_statistics/var_ss" "Unknown input: ^moments_5/sufficient_statistics/mean_ss" "Unknown input: ^moments_5/sufficient_statistics/var_ss" "Unknown input: ^moments_6/sufficient_statistics/mean_ss" "Unknown input: ^moments_6/sufficient_statistics/var_ss" "Unknown input: ^moments_7/sufficient_statistics/mean_ss" "Unknown input: ^moments_7/sufficient_statistics/var_ss" "Unknown input: ^moments_8/sufficient_statistics/mean_ss" "Unknown input: ^moments_8/sufficient_statistics/var_ss" "Unknown input: ^moments_9/sufficient_statistics/mean_ss" "Unknown input: ^moments_9/sufficient_statistics/var_ss" "Unknown input: ^moments_10/sufficient_statistics/mean_ss" "Unknown input: ^moments_10/sufficient_statistics/var_ss" "Unknown input: ^moments_11/sufficient_statistics/mean_ss" "Unknown input: ^moments_11/sufficient_statistics/var_ss" "Unknown input: ^moments_12/sufficient_statistics/mean_ss" "Unknown input: ^moments_12/sufficient_statistics/var_ss" "Unknown input: ^moments_13/sufficient_statistics/mean_ss" "Unknown input: ^moments_13/sufficient_statistics/var_ss" "Set up takes:: 1.77025699615479 (0.564889731927128 per second)"

Screenshot of the example after clicking run of course: http://imgur.com/a/WIUHE

santiagofm commented 7 years ago

Hey @AndreJFBico ! I am wondering how you could run on an iOS 11 device from Xcode 8.3.2 as that isn't possible, you need xcode 9. Can you check the version of Xcode you use ? Cheers

AndreJFBico commented 7 years ago

Hey! thanks for the fast answer, and yes its possible. https://stackoverflow.com/questions/44381860/use-xcode-8-with-ios-11

But on the other hand i should have tested it with Xcode 9 just to make sure, ill do it and report back.

santiagofm commented 7 years ago

Oh my bad then ! I don't have any device with iOS 11 in hand so I can't really test 😞 but please do using Xcode 9 beta 3 (with beta 2 Bender won't compile) and don't mind those logs as they are just warnings and they'll get ignored once #44 gets merged.

chanmi commented 7 years ago

In my case, I have similar error with this: "Unknown input".

I'm trying to run object detection with MobileNet which is recently presented by Google. They provided trained model of SSD with MobileNets in https://github.com/tensorflow/models/blob/master/object_detection/g3doc/detection_model_zoo.md. What I want to run is _ssd_mobilenet_v1coco.

I started the work with change the model from "mnist_full.pb" to this model in MNISTTestController.swift in the example code. The following error occurs.

2017-07-24 17:41:23.925148+0900 Example[28045:7594089] [DYMTLInitPlatform] platform initialization successful
2017-07-24 17:41:24.057314+0900 Example[28045:7594049] Metal GPU Frame Capture Enabled
2017-07-24 17:41:24.057884+0900 Example[28045:7594049] Metal API Validation Enabled
2017-07-24 17:41:24.114991+0900 Example[28045:7594049] libMobileGestalt MobileGestaltSupport.m:153: pid 28045 (Example) does not have sandbox access for frZQaeyWLUvLjeuEK43hmg and IS NOT appropriately entitled
2017-07-24 17:41:24.115131+0900 Example[28045:7594049] libMobileGestalt MobileGestalt.c:550: no access to InverseDeviceID (see <rdar://problem/11744455>)
Hello world in ViewController.swift
Running test: Example.TextureConversionTest
Running test: Example.LocalResponseNormTest
Running test: Example.InstanceNormTest
Running test: Example.ConcatTest
ALL TEST PASSED
2017-07-24 17:41:25.785573+0900 Example[28045:7594049] [MC] System group container for systemgroup.com.apple.configurationprofiles path is /private/var/containers/Shared/SystemGroup/systemgroup.com.apple.configurationprofiles
2017-07-24 17:41:25.786231+0900 Example[28045:7594049] [MC] Reading from public effective user settings.
"Unknown input: Preprocessor/map/TensorArray:1"
"Unknown input: Preprocessor/map/TensorArray_1:1"
"Unknown input: Preprocessor/map/while/Switch:1"
"Unknown input: Preprocessor/map/while/Switch_1:1"
"Unknown input: ^Preprocessor/map/while/Identity"
"Unknown input: ^Preprocessor/map/while/Identity"
"Unknown input: ^Preprocessor/map/while/Identity"
"Unknown input: ^FeatureExtractor/Assert/Assert"
"Unknown input: ^FeatureExtractor/Assert/Assert"
"Unknown input: ^FeatureExtractor/Assert/Assert"

... (same line is repeated many times) ...

"Unknown input: ^MultipleGridAnchorGenerator/assert_equal/Assert/Assert"
"Unknown input: ^assert_equal/Assert/Assert"
"Unknown input: ^assert_equal/Assert/Assert"
"Unknown input: Postprocessor/Decode/get_center_coordinates_and_sizes/unstack:3"
"Unknown input: Postprocessor/Decode/get_center_coordinates_and_sizes/unstack:1"
"Unknown input: Postprocessor/Decode/get_center_coordinates_and_sizes/unstack:2"
"Unknown input: Postprocessor/Decode/get_center_coordinates_and_sizes/unstack:1"
"Unknown input: Postprocessor/Decode/unstack:1"
"Unknown input: Postprocessor/Decode/unstack:2"
"Unknown input: Postprocessor/Decode/unstack:3"
"Unknown input: ^Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/Assert/Assert"
"Unknown input: ^Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/Assert/Assert"
"Unknown input: ^Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/Assert/Assert"
"Unknown input: Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/ClipToWindow/unstack:2"
"Unknown input: Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/ClipToWindow/split:2"
"Unknown input: Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/ClipToWindow/unstack:2"
"Unknown input: Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/ClipToWindow/split:1"
"Unknown input: Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/ClipToWindow/unstack:3"
"Unknown input: Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/ClipToWindow/unstack:1"
"Unknown input: Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/ClipToWindow/split:3"
"Unknown input: Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/ClipToWindow/unstack:3"
"Unknown input: Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/ClipToWindow/unstack:1"
"Unknown input: Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/ClipToWindow/Area/split:2"
"Unknown input: Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/ClipToWindow/Area/split:3"
"Unknown input: Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/ClipToWindow/Area/split:1"
"Unknown input: ^Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/Assert/Assert"
"Unknown input: ^Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/Assert/Assert"
"Unknown input: ^Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/Assert/Assert"

... (similar lines are repeated many times) ...

assertion failed: Seems you might have a cyclic dependency in your graph. That is not supported!: file /Users/naver/Desktop/tf_project/Bender/Sources/Graph/Graph.swift, line 30
2017-07-24 17:43:29.594486+0900 Example[28045:7594049] assertion failed: Seems you might have a cyclic dependency in your graph. That is not supported!: file /Users/naver/Desktop/tf_project/Bender/Sources/Graph/Graph.swift, line 30
(lldb)

I run on an iPhone7, iOS10 device from Xcode 8.3.3 and I don't think it's the version problem as above. So I would like to ask you for advice on how to do this. Let someone know where to start for it.

AndreJFBico commented 7 years ago

That seems to me that your model has unimplemented layers, operations that have no equivalent in Benders side?

chanmi commented 7 years ago

Hmm.. then in the error message, the layer or operations( TensorArray, switch, identity, assert, get_center_coordinates_and_sizes, unstack, MultiClassNonMaxSuppression) are unimplemented.

In Bender/Sources/Layers folder, we can find identity Identity.swift. It means, for identity layer -> I have to connect to the system. for other layers -> I have to implement.

Do you know how can I connect the input name with corresponding operation? I mean, input name: (Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/Assert/Assert) -> to (operation nonmaxsuppression).

bryant1410 commented 7 years ago

I haven't seen the graph, but according to that output:

  1. There seem to be unimplemented operations. For those, we have no ETA to implement them. You can do it yourself :smile: We can provide help, but take a look at the existing operations first to gain insight.
  2. There's a cycle in your graph. We don't support that yet and we don't have an ETA. You can also work on this if you want. We are always open to contributions!
bryant1410 commented 7 years ago

Hey! thanks for the fast answer, and yes its possible. https://stackoverflow.com/questions/44381860/use-xcode-8-with-ios-11

But on the other hand i should have tested it with Xcode 9 just to make sure, ill do it and report back.

Were you able to make it work?

chanmi commented 7 years ago

I have made the system print NOT "Unknown input". As insert a line print("temp input: \(input)")
after line 40 in https://github.com/xmartlabs/Bender/blob/master/Sources/Adapters/Tensorflow/TFGraph.swift. This is a part of my error message.

...

"Unknown input: ^Postprocessor/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/cond_2/cond/switch_t"
temp input: Postprocessor/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/cond_2/ExpandDims
temp input: Postprocessor/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/cond_2/cond/pred_id
"Unknown input: Postprocessor/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/cond_2/cond/concat/Switch:1"
temp input: Postprocessor/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/cond_2/cond/strided_slice
temp input: Postprocessor/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/cond_2/cond/concat/axis
"Unknown input: ^Postprocessor/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/cond_2/cond/switch_f"
temp input: Postprocessor/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/cond_2/strided_slice
temp input: Postprocessor/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/cond_2/cond/pred_id
temp input: Postprocessor/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/cond_2/cond/sub/x
temp input: Postprocessor/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/cond_2/cond/sub/Switch

...

By checking "BatchMultiClassNonMaxSuppression" is in "temp input: " part, it seems like problem of unmatching with the node name and operation (or something like that.) rather than unimplemented layers.(I think unimplemented layer problem is later thing.)

Is there any help to cover this? (such as... how to match the name and operation in graph..?) (related to graphDef.node.map in Sources/Adapters/Tensorflow/TFGraph.swift line 28... ) Thank you

AndreJFBico commented 7 years ago

About the current issue we managed to make it work properly on iphone 6 using xcode 9, we had to compile the swift protobuf library using swift 3.2 as the carthage update builds it using Swift 3.1

bryant1410 commented 7 years ago

@chanmi can you open a new issue for that?

AndreJFBico commented 7 years ago

Ok im going to close this down then.

backnotprop commented 7 years ago

Anyone run into the issue of using an image larger than 256x256 and having it replicate the styled image 4 times inside of its actual dimensions (like columns, vs one image) ?

bryant1410 commented 7 years ago

Anyone run into the issue of using an image larger than 256x256 and having it replicate the styled image 4 times inside of its actual dimensions (like columns, vs one image) ?

Not me. Maybe you should open a new issue if you detect some bug.