search

microsoft / only_train_once

OTOv1-v3, NeurIPS, ICLR, TMLR, DNN Training, Compression, Structured Pruning, Erasing Operators, CNN, Diffusion, LLM
MIT License
25 stars 5 forks source link

Pruning problem of deep grouped convolution models. #15

Open zbzb-dlut opened 2 months ago

zbzb-dlut commented 2 months ago

Hello author! May I ask if this model can handle deep grouped convolution models? Additionally, for reparameterized models, is this method applicable during the training stage?

aponte411 commented 1 month ago

hi @zbzb-dlut thanks for the question. Our response is late due to recent business travel.

Currently, for grouped conv, we only support the case where groups = out_channels, e.g., the depthwisw conv. However, adding support for different scenarios should be doable, which can be achieved by further development on https://github.com/microsoft/only_train_once/blob/c5ec214833b5f5b9433f3f59ae21564b75c35886/only_train_once/operation/operator.py#L153

As for the reparameterization, it depends on what type of reparameterization, in general, the answer is yes.