Q: Implement YOLACT compatible with a big DNN framework API (tensorflow,..?)

[breznak]

With regards to #299 and problem with early stopping #298 , would you agree to add API compatibility to YOLACT with some of the large DNN frameworks (tensorflow, caffee, keras,..?)

• Which would be the most suitable?
• Would it be feasible work?
• Is YOLACT more or less stable, or are some wild swings in the code expected?

Motivation is that this would a) make Yolact more widespread, easy to use, b) better experience using yolact, since all the "side" problems (training, evaluation, persistency, visualization) are already covered by the frameworks

This would be a lot of work, so I'd like to know if @dbolya thinks this is a good idea and if someone wants to join forces on this?

[breznak]

It would be interesting to join forces with https://github.com/open-mmlab/mmdetection/issues/1230

[dbolya]

I'm at risk of announcing this prematurely, but we have a YOLACT implementation ready to be merged into mmdetection. We're just waiting on a couple more experiments to verify performance and for the mmdetection team to let us know how they want to merge.

I'm not going to make a keras, caffee, or tensorflow implementation, since that would be a lot of work for not much gain, but what's the problem with Pytorch? Caffee has now been merged with pytorch so it's at least as big >.>

[breznak]

Wow, this is great news! :100: I've just briefly reviewed mmdetection repo, this could be relevant for the "calling train/eval from code" issue #299 , as there's an API in mmdet for eval : https://github.com/open-mmlab/mmdetection/blob/master/docs/GETTING_STARTED.md#high-level-apis-for-testing-images

[abhigoku10]

@breznak @dbolya is there any pros of merging yolact with mmdetection ? since currently yolact as itself standalone is the best y dont @dbolya think of adding features as darknet community perform , this repo can be a pioneer in implementation of new ideas !!!! just a thought

[breznak]

'm not going to make a keras, caffee, or tensorflow implementation, since that would be a lot of work for not much gain, but what's the problem with Pytorch?

No, I was not suggesting that the internals for YOLACT were rewritten in another framework, I wanted to make yolact Model a (better-fitting) part of a framework (pytorch?)

I'm not that well versed in pytorch and what its Module can do? https://pytorch.org/docs/stable/nn.html#torch.nn.Module

But my suggestion is intended for eval/train issue #299 , and ie letting the learning-rate, early stopping (#298 ) be done by the framework automatically, for example in tf it works like this https://lambdalabs.com/blog/tensorflow-2-0-tutorial-04-early-stopping/

So can pytorch do similar functionality? And then we should adhere (implement more completely) its API so people can treat Yolact as any other pytorch.Module

the goal is to

better experience using yolact, since all the "side" problems (training, evaluation, persistency, visualization) are already covered by the frameworks

[dbolya]

@breznak Yeah, Pytorch has similar functionality that I just wasn't aware of at the time. The LR scheduling and likely early stopping can be done through the optimizer, so I'm planning on switching the hard-coded stuff out for that as you suggested in another issue.

@abhigoku10, @breznak Even though YOLACT will be offered in mmdetect, I won't stop maintaning this repo because I can perform exploratory stuff here that I couldn't on some super polished repo like mmdetect.

[abhigoku10]

@dbolya this was just my perceptive , it would good if this repo can grow better than darknet , since YOLACT address the problem of both accuracy and performance , we can add other backbones like ghostnet , mixnet , shufflenet and new augmentations techniques like cutout mixup , new ideas like adding orientation of teh object , depth feature and so on is it possible to list out the improvements and people in the community can work on it ??

[dbolya]

@abhigoku10 That's a good idea. I'd like to clean up the code first though, since there are a bunch of settings in coco_base_config that we tested for one experiment once, found that it didn't work, and just left it in there for backward compatibility with old experiments (for instance, our baseline was the first thing I implemented, and then a year later the exact same config worked to train a new baseline because I lost the old weights). Most of those should be removed so that the community can more easily work on the project (I mean just look at the size of coco_base_config, that's way too much).

[abhigoku10]

@dbolya sure please do clean up the code and let us knw i want to work on few things with collaboration from others too @breznak @fanyix

1. anchor free detection and segmentation 2.adding ghostnet,shufflenet,thundernet,matrixnet as feature backbone 3.performing semantic segmentation on cityscapes data 4adding rotation roi and depth estimation @dbolya would like to hear ur thoughts on this

[innat]

@breznak Tensorflow-YOLACT