ENCP / CNNdroid

Open Source Library for GPU-Accelerated Execution of Trained Deep Convolutional Neural Networks on Android
MIT License
539 stars 181 forks source link

Are you planning to develop the project further? #1

Open mkaskov opened 8 years ago

mkaskov commented 8 years ago

Nice project very intresting projects. I tested it on nexus5 (snapdragon 800)

Are you planning to develop the project further?

for example

matinhashemi commented 8 years ago

Hi

Yes. In fact, we are 1) developing faster mobile GPU algorithms for currently supported layers, and 2) adding compressed neural networks. You are more than welcome to join the project.

Large models, e.g., googlenet, do not fit in the mobile memory and therefore are not part of our development plans, but compressed models, e.g., squeezenet, do fit and we are working on them.

Currently we are focused more on CNNs not RNNs.

Matin

mkaskov commented 8 years ago

Hi Im glad to see it.

  1. On my device (nexus5) CNNdroid in parallel mode time to recognize one image ~800-900 ms (CaffeNet)
  2. Then I tried to add SqueezeNet. but when I create a network I found that the CNNdroid does not support the concat and dropout layers. Now I think how to implement them for calculations on Renderscript.
  3. Google TensorFlow has a mobile app for android with pruned inception5h network. https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android I tested it on my device. time to recognize ~300-500ms but their application does not support mGPU. inception5h has 55mb size on storage.
  4. I tested few other apps without mGPU support android native C++ and it result 700-1500 ms
matinhashemi commented 8 years ago

Yes, the computation times are about what we have measured as well. About 1 second for every image.

As I have mentioned, we are adding support for compressed models, e.g., SqueezeNet.

mkaskov commented 8 years ago

This is great news. it will be interesting to try. Pruning or/and quantize network model gives performance boost? did you test? (preparing scripts from compact models deploy large * .msg files)

pavelgonchar commented 7 years ago

Hi, Is there an advantage of using GPU-Accelerated CNNdroid over CPU only tensorflow? Do you have any idea on how faster is that when running on comparable networks?

mkaskov commented 7 years ago

in tensorflow app google uses inception network. it is one of the best performance network by calculation cost. closed to squeezenet but more accuracy. google uses pruning and quantization to decrease calculation cost also. @matinhashemi said that they planning realize squeezenet. with prunned and quatizated model it will provide performance boost, maybe ....

matinhashemi commented 7 years ago

Hi, We have not tested tensorflow yet.

shrutisharmavsco commented 7 years ago

@mkaskov were you able to run the demo projects on Nexus 5 without any changes? I am having trouble running them as is on a Nexus 5 with Android 6.0 - I've had to make a lot of changes to the project already.

mkaskov commented 7 years ago

@shrutisharmavsco I did a lot of additions to run the code.

AlexandreBriot commented 7 years ago

Regarding CNNdroid / Android Tensorflow demo comparison, I did some runtime measurements on HTC One M9.

As mentionned in your paper, with CNNdroid you can process a forward pass within 700 ms for a 16 images batchsize in optimal conditions. For a single image batchsize, runtime is around 1 second for me.

I tested the same AlexNet model by converting it from bvlc caffe to tensorflow .pb file. Running this model instead of default inception model, I checked the mean value of inference time displayed in the tensorflow demo logcat.

I got similar difference when I compare inference times for tensorflow demo with inception model building it with bazel/gradle (480ms vs 990ms).

I have very little knowledge about gradle/bazel, does this seems surprising to you to observe such a difference ? Do you think we could get similar improvement building CNNdroid with bazel ?

filipetrocadoferreira commented 7 years ago

Quantization and Pruning are on the plan to CNNdroid? Quantized SqueezeNet in CNNDroid should be nice :3

latifisalar commented 7 years ago

Yes, quantization and pruning are in the pipeline and we're working to add the support for them.

michaelholm-ce commented 7 years ago

Roughly how close are you to squeezenet support? I am interested in trying that when the time comes.

latifisalar commented 7 years ago

Sorry for the delayed response, the implementation of the version 1 is kind of ready, and hopefully will be pushed to the git next month.

llhe commented 7 years ago

The underlying library for TensorFlow is Eigen and gemmlowp. Is there benchmark done between the CPU version conv/matmul and GPU shader version?

latifisalar commented 7 years ago

Hi, Unfortunately, we have not done any of the mentioned benchmarks.

siftr commented 7 years ago

Hello I was looking forward for squeezenet support in CNNdroid, roughly when will it be available?

latifisalar commented 7 years ago

Hi, It should be ready really soon, will give you the update about the approximate release time shortly.