Model Optimization Chapter 15

[adhiiisetiawan]

Hi everyone, i with the team just discuss about model optimization chapter, and we make an outline for this chapter. Overall we plan like this

Module 1: Introduction to model optimization for deployment

What is model optimization? Why is it important for deployment in computer vision? Different types of model optimization techniques Trade-offs between accuracy, performance, and resource usage

Module 2: Model compression techniques

Quantization Pruning Knowledge distillation Low-rank approximation Model compression with hardware accelerators

Module 3: Model deployment considerations

Different deployment platforms (cloud, edge, mobile) Model serialization and packaging Model serving and inference Best practices for deployment in production

Module 4: Model optimization tools and frameworks

TensorFlow Model Optimization Toolkit (TMO) PyTorch Quantization ONNX Runtime OpenVINO TensorRT Edge TPU

Module 5: Model optimization case studies (Hands on, choose one is okay)

Deploying a real-time object detection model on a mobile device Deploying a semantic segmentation model on an edge device Deploying a face recognition model for authentication in the cloud Case studies of using model optimization techniques for specific deployment challenges

Module 6: Future trends in model optimization for deployment

Federated learning Continual learning Model compression for new hardware architectures

Module means like section in each chapter, so for example in NLP course chapter 1, there are 10 module. And we plan to create like that, and for each module contains topics like above.

And than for module 5, we still discussing what the hands on project use case should implement. But for starter we plan to object detection on mobile device.

Let us know if the outline need to be revise or all of you guys have any suggestions, it will be very helpful @johko @lunarflu @merveenoyan

cc: @mfidabel

[merveenoyan]

Hello @adhiiisetiawan thanks for the outline! My points are mostly supportive with additions:

1. Overall module 1 looks good :)
2. Knowledge distillation is covered under transfer learning section given it's also a transfer learning technique so I wouldn't go very in-depth with it. What do you want to cover under compression given you have a different section below? I didn't apply a lot of quantization in computer vision, AFAIK there's post-training quantization, but if there's quantization aware training it might be worth covering both under different sections.
3. Third and fourth sections seem very comprehensive :)
4. Same with 5th. I was in ultralytics conference and there was someone from OpenVINO team presenting about real-time object detection on edge (quantized model) you can watch here I think you could demonstrate both quantization and real-time serving in one example and it would be very comprehensive.
5. This section is very cool as well, also for continual learning, check out this where folks build active learning pipeline (if you're interested).

[adhiiisetiawan]

Hi @merveenoyan thanks for feedback. So, is knowledge distillation should we drop from this chapter? or we just review it again like in "transfer learning" chapter? and about quantization, yap I mean post-training quantization, we want to explain different technique in that "compression" module. thank you again for resources, it's cool.

So, is this overall outline good?, if yes, we will move to make the materials.

[johko]

hey @adhiiisetiawan,

the outline looks great and I don't really have anything to add to @merveenoyan's suggestions. I think you can keep the knowledge distillation part, but keep it short. It is good to mention it, in case people didn't check the transfer learning chapter(s) and you can make sure to reference them for more information.

It feels like a lot of content to me, so make sure to divide it up well in your group and try to focus on some important parts of each module first. Connected to that, how is your team going? You only marked @mfidabel up there. What about the other 3 people, are they responsive? Because I think with that much (awesome) content planned, you will need more than 2 people working on it.

[adhiiisetiawan]

Thank you @johko for your suggestions. Hmm 3 people is unreachable, 2 not accept my friends request to added in group chat and 1 people already joined, but not say anything. Maybe they are busy or something. I will ask to them first.

[mfidabel]

the outline looks great and I don't really have anything to add to @merveenoyan's suggestions. I think you can keep the knowledge distillation part, but keep it short. It is good to mention it, in case people didn't check the transfer learning chapter(s) and you can make sure to reference them for more information.

We could reference the transfer learning chapter and add some examples of distilled models very briefly. As per the other techniques, we could also take a look at what the transfer learning chapter is doing and try to apply the other techniques to the same problem, so that we can compare them.

CC: @adhiiisetiawan

[johko]

@adhiiisetiawan let me know in discord if the members still don't respond very soon (I'd say by tomorrow). Then we can free up their slots and find other people who might have more time to work on it.

[adhiiisetiawan]

okay @johko