search

dbolya / yolact

A simple, fully convolutional model for real-time instance segmentation.
MIT License
5k stars 1.32k forks source link

Tenserflow & TPU (coral) #326

Open doit-ceo opened 4 years ago

doit-ceo commented 4 years ago

First, this is work is WOW and I'm newbie in all this AI stuff. But has been playing with TF for a while and was going to use it along with TPU (Google Coral), now I see and tried your project, I'm sold (managed to create my own video interference on trained data). But then I still need to run interference on small devices (like mobile or Raspberry Pi, Coral), so questions

  1. Can we develop Yolact as TF ?
  2. Would Yolact benefit from TPU (Coral) and what FPS would I expect ?
  3. Would Yolact ever run on mobile device with TF lite , what FPS would I expect ?
  4. I read that even with jetson nano, people are getting 5 FPS , is that normal ?

Thanks and sorry for posting this topic as Issues

dbolya commented 4 years ago
  1. I don't plan on porting this to TF, but I've heard of other people trying. Though, I Pytorch likely already has all the features you're looking for.
  2. Since most of the computation of YOLACT is centered in the backbone (just vanilla resnet), and since TPUs result in huge performance gains in exactly that situation, I'd expect it to be fairly powerul there. There are articles on using TPUs in Pytorch, but it seems pretty hacky right now.
  3. You might want to check out pytorch mobile: https://pytorch.org/mobile/home/. I have no idea what fps to expect on mobile though. These models run at 45 fps on beefy GPUs, so I'd assume it would be <= 5 fps, but that's pure conjecture.
  4. ¯\_(ツ)_/¯ 45 fps on a $1200 graphics card v.s. 5 fps on a $99 graphics card seems pretty reasonable to me. I've never used a jetson nano myself though, so I can't verify if everything's working properly.
lbq779660843 commented 4 years ago

First, this is work is WOW and I'm newbie in all this AI stuff. But has been playing with TF for a while and was going to use it along with TPU (Google Coral), now I see and tried your project, I'm sold (managed to create my own video interference on trained data). But then I still need to run interference on small devices (like mobile or Raspberry Pi, Coral), so questions

  1. Can we develop Yolact as TF ?
  2. Would Yolact benefit from TPU (Coral) and what FPS would I expect ?
  3. Would Yolact ever run on mobile device with TF lite , what FPS would I expect ?
  4. I read that even with jetson nano, people are getting 5 FPS , is that normal ?

Thanks and sorry for posting this topic as Issues

I also get 5 FPS in my jetson nano when I test a video, I think it maybe work by using torch2trt to convert PyTorch to TensorRT in somewhere of this project if we want to accelerate the test speed.

abhigoku10 commented 4 years ago

@lbq779660843 @doit-ceo i am newbo in this work can you please share the code base or process to do this , it would be much helpful THanks in advance