search

cscribano / CERBERUS

Simple and Effective All-In-One Automotive Perception Model with Multi Task Learning
20 stars 1 forks source link

Quantized model available? #1

Closed learnermaxRL closed 2 years ago

learnermaxRL commented 2 years ago

Hi, looking for a quantized model if you can make avaialble

cscribano commented 2 years ago

Hi, thank you for your interest on our work!

Regarding quantized inference, currently we only deployed the model on TensorRT, which supports FP16 post-training quantization. To better understand you request, please let me now which kind of quantization do you need and the intended inference platform and framework for inference.

learnermaxRL commented 2 years ago

Hey,thanks for your reply I am trying to run it on - https://nanoreview.net/en/soc/qualcomm-snapdragon-450 ,

I calculated that this mode takes around 6.99 macs in compute complexity,which i am trying to bring down, it seems BIFPN layera are contributing the most so i had to remove 2 layers and add an upsample layer to match the dimesnions which brough the MACs ~ 1.8 MACs

I am seeking a multitask architecture which can run on A53 Cortex family actually.It would be great if you can suggest some optimizations as well

Also can you please provide details of train and eval losses with your configuration?

learnermaxRL commented 2 years ago

is there a way we can add few classes to ob detction head,i am assuming we can freeze neck and other heads and ony leave ob det head trainable ,would that work?

cscribano commented 2 years ago

Regarding your first comment, I'm not familiar with the exact platform, but I assume hat you are referring to an android smartphone. In that case, I assume that you want to use TFLite for inference. Can you please confirm?

Regarding the second question: the model is trained on all the detection classes in BDD100K, is not in general possible to add new classes to a pretrained detector by only training on new classes. You will incur in the problem known as catastrophic forgetting of the previously learned tasks.

learnermaxRL commented 2 years ago

freezing everything except the ob det head , and passing original images plus images with our objects ,(other objects can be tagged using pretrained heavier model in those images or using this model itself for e.g car ,lanes etc),shouldnt it wokr ?

learnermaxRL commented 2 years ago

with TFlite the mbv2 model is giving roughly 1 fps

cscribano commented 2 years ago

freezing everything except the ob det head , and passing original images plus images with our objects ,(other objects can be tagged using pretrained heavier model in those images or using this model itself for e.g car ,lanes etc),shouldnt it wokr ?

You can give it a try, but keep in mind that the detection head alone has way too few trainable parameters to lead to satisfactory results. It might be an interesting experiment, however.