Official support for finetune-based methods, e.g., vit adapter, and multi-GPU training with DataParallel.

albumentations-team / autoalbument

AutoML for image augmentation. AutoAlbument uses the Faster AutoAugment algorithm to find optimal augmentation policies. Documentation - https://albumentations.ai/docs/autoalbument/

https://albumentations.ai/docs/autoalbument/

MIT License

198 stars 20 forks source link

Official support for finetune-based methods, e.g., vit adapter, and multi-GPU training with DataParallel. #48

Open TimandXiyu opened 9 months ago

TimandXiyu commented 9 months ago

The implementation of the gradient update in faa_model.py seems to be very constrainted at best. It does not factor in the case where I want to obtain the policy for a finetuning model, it only naively set all layer's parameter to either require or not require gradient. Besides, this is also problematic if the user want to use DP to enable multi-GPU training.

This repo really needs some updates to address the mentioned issues as well as other issues that has been mentioned in other issues.

saigontrade88 commented 3 months ago

@TimandXiyu : can you provide more information related to your issues? It is best if you can provide detailed information related to their implementation. Thanks.