๐๐๐ฎ๐ผ๐ธ๐ถ๐ฎ ๐๐ช๐ป๐ช๐ถ๐ฎ๐ฝ๐ฎ๐ป-๐๐ฏ๐ฏ๐ฒ๐ฌ๐ฒ๐ฎ๐ท๐ฝ ๐ฃ๐ป๐ช๐ท๐ผ๐ฏ๐ฎ๐ป ๐๐ฎ๐ช๐ป๐ท๐ฒ๐ท๐ฐ
    [](https://github.com/sindresorhus/awesome) [](https://GitHub.com/Naereen/StrapDown.js/graphs/commit-activity)
๐ ๐ฌ๐ธ๐ต๐ต๐ฎ๐ฌ๐ฝ๐ฒ๐ธ๐ท ๐ธ๐ฏ ๐ป๐ฎ๐ผ๐ธ๐พ๐ป๐ฌ๐ฎ๐ผ ๐ธ๐ท ๐น๐ช๐ป๐ช๐ถ๐ฎ๐ฝ๐ฎ๐ป-๐ฎ๐ฏ๐ฏ๐ฒ๐ฌ๐ฒ๐ฎ๐ท๐ฝ ๐ฝ๐ป๐ช๐ท๐ผ๐ฏ๐ฎ๐ป ๐ต๐ฎ๐ช๐ป๐ท๐ฒ๐ท๐ฐ.โญ Citation
If you find our survey and repository useful for your research, please cite it below:
@article{xin2024parameter,
title={Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey},
author={Xin, Yi and Luo, Siqi and Zhou, Haodi and Du, Junlong and Liu, Xiaohong and Fan, Yue and Li, Qing and Du, Yuntao},
journal={arXiv preprint arXiv:2402.02242},
year={2024}
}
๐ฅ News
-
[2024/03/01] "Visual PEFT Library/Benchmark" repo is created.
-
[2024/02/01] "Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey" is released.
-
[2023/01/01] "Awesome-Parameter-Efficient-Transfer-Learning" repo is created.
๐ Table of Contents
๐ Introduction
- Parameter-Efficient Fine-Tuning (PEFT) seeks to exceed the performance of full fine-tuning with minimal parameter modifications.
- This repository provides a comprehensive overview and offer a systematic review of the latest advancements. It introduces a categorization criterion that classifies existing methods into three categories: Addition-based Tuning, Partial-based Tuning, and Unified-based Tuning.
- This repository also introduces commonly used datasets and applications.
๐ฌ Keywords
๐ Papers
Addition-based Tuning
Adapter Tuning
-
[1] AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition, NeurIPS 2022.
Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, Ping Luo.
-
[2] Convolutional Bypasses are Better Vision Transformer Adapters, Arxiv 2022.
Jie, Shibo and Deng, Zhi-Hong.
-
[3] ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning, NeurIPS 2022.
Pan, Junting and Lin, Ziyi and Zhu, Xiatian and Shao, Jing and Li, Hongsheng.
-
[4] AIM: Adapting Image Models for Efficient Video Action Recognition, ICLR 2023.
Yang, Taojiannan and Zhu, Yi and Xie, Yusheng and Zhang, Aston and Chen, Chen and Li, Mu.
-
[5] Lossless Adaptation of Pretrained Vision Models For Robotic Manipulation, ICLR 2023.
Sharma, Mohit and Fantacci, Claudio and Zhou, Yuxiang and Koppula, Skanda and others.
-
[6] 1% VS 100%: Parameter-Efficient Low Rank Adapter for Dense Predictions, CVPR 2023.
Yin, Dongshuo and Yang, Yiran and Wang, Zhechao and Yu, Hongfeng and Wei, Kaiwen and Sun, Xian.
[Paper][Code]
-
[7] Polyhistor: Parameter-Efficient Multi-Task Adaptation for Dense Vision Tasks, NeurIPS 2022.
Yen-Cheng Liu, Chih-Yao Ma, Junjiao Tian, Zijian He, Zsolt Kira.
[Paper][Code]
-
[8] VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding, AAAI 2024.
Yi Xin, Junlong Du, Qiang Wang, Zhiwen Lin, Ke Yan.
[Paper][[Code]()]
-
[9] SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels, IJCV 2023.
Henry Hengyuan Zhao, Pichao Wang, Yuyang Zhao, Hao Luo, Fan Wang, Mike Zheng Shou.
-
[10] Important Channel Tuning, Openreview 2023.
Hengyuan Zhao, Pichao WANG, Yuyang Zhao, Fan Wang, Mike Zheng Shou.
[Paper][Code]
-
[11] Revisit Parameter-Efficient Transfer Learning: A Two-Stage Paradigm, Arxiv 2023.
Zhao, Hengyuan and Luo, Hao and Zhao, Yuyang and Wang, Pichao and Wang, Fan and Shou, Mike Zheng.
[Paper][Code]
-
[12] Compacter: Efficient Low-Rank Hypercomplex Adapter Layer, NeurIPS 2021.
Karimi Mahabadi, Rabeeh and Henderson, James and Ruder, Sebastian.
-
[13] Parameter-efficient and student-friendly knowledge distillation, NeurIPS 2022.
Rao, Jun and Meng, Xv and Ding, Liang and Qi, Shuhan and Tao, Dacheng.
[Paper][Code]
-
[14] VL-adapter: Parameter-efficient transfer learning for vision-and-language tasks, CVPR 2022.
Sung, Yi-Lin and Cho, Jaemin and Bansal, Mohit.
-
[15] UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling, ICLR 2024.
Haoyu Lu, Mingyu Ding, Yuqi Huo, Guoxing Yang, Zhiwu Lu, Masayoshi Tomizuka, Wei Zhan.
-
[16] Parameter Efficient Fine-tuning via Cross Block Orchestration for Segment Anything Model, Arxiv 2023.
Zelin Peng, Zhengqin Xu, Zhilin Zeng, Lingxi Xie, Qi Tian, and Wei Shen.
[Paper][Code]
-
[17] Hydra: Multi-head Low-rank Adaptation for Parameter Efficient Fine-tuning, Arxiv 2023.
Sanghyeon Kim, Hyunmo Yang, Younghyun Kim, Youngjoon Hong, Eunbyung Park.
-
[18] MixPHM: Redundancy-Aware Parameter-Efficient Tuning for Low-Resource Visual Question Answering, CVPR 2023.
Jingjing Jiang, Nanning Zheng.
-
[19] Vision Transformers are Parameter-Efficient Audio-Visual Learners, CVPR 2023.
Yan-Bo Lin, Yi-Lin Sung, Jie Lei, Mohit Bansal, Gedas Bertasius.
-
[20] SAM-Adapter: Adapting Segment Anything in Underperformed Scenes, ICCVW 2023.
Chen, Tianrun and Zhu, Lanyun and Deng, Chaotao and Cao, Runlong and Wang, Yan and Zhang, Shangzhan and Li, Zejian and Sun, Lingyun and Zang, Ying and Mao, Papa.
-
[21] T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models, AAAI 2024.
Mou, Chong and Wang, Xintao and Xie, Liangbin and Zhang, Jian and Qi, Zhongang and others.
-
[22] I2V-Adapter: A General Image-to-Video Adapter for Video Diffusion Models, Arxiv 2023.
Guo, Xun and Zheng, Mingwu and Hou, Liang and Gao, Yuan and Deng, Yufan and others.
-
[23] AdaptIR: Parameter Efficient Multi-task Adaptation for Pre-trained Image Restoration Models, Arxiv 2023.
Hang Guo, Tao Dai, Yuanchao Bai, Bin Chen, Shu-Tao Xia, Zexuan Zhu.
-
[24] A Closer Look at Parameter-Efficient Tuning in Diffusion Models, Arxiv 2023.
Chendong Xiang, Fan Bao, Chongxuan Li, Hang Su, Jun Zhu.
-
[25] CAST: Cross-Attention in Space and Time for Video Action Recognition, NeurIPS 2023.
Lee, Dongho and Lee, Jongseo and Choi, Jinwoo.
-
[26] Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis, CVPR 2024.
Xin Zhou , Dingkang Liang , Wei Xu, Xingkui Zhu ,Yihan Xu, Zhikang Zou, Xiang Bai.
-
[27] MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation, ArXiv 2024.
Kunpeng Song and Yizhe Zhu and Bingchen Liu and Qing Yan and Ahmed Elgammal and Xiao Yang.
-
[28] Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentation, ICCV 2023.
Zunnan Xu, Zhihong Chen, Yong Zhang, Yibing Song, Xiang Wan, Guanbin Li.
-
[29] Enhancing Fine-grained Multi-modal Alignment via Adapters: A Parameter-Efficient Training Framework for Referring Image Segmentation, WANT @ ICML 2024.
Zunnan Xu, Jiaqi Huang, Ting Liu, Yong Liu, Haonan Han, Kehong Yuan, Xiu Li.
-
[30] Sparse-Tuning: Adapting Vision Transformers with Efficient Fine-tuning and Inference, ArXiv 2024.
Ting Liu, Xuyang Liu, Liangtao Shi, Zunnan Xu, Siteng Huang, Yi Xin, Quanjun Yin.
Prompt Tuning
-
[1] Visual Prompt Tuning, ECCV 2022.
Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, Ser-Nam Lim.
-
[2] Visual Prompt Tuning for Test-time Domain Adaptation, Arxiv 2022.
Gao, Yunhe and Shi, Xingjian and Zhu, Yi and Wang, Hao and Tang, Zhiqiang and Zhou, Xiong and others.
[Paper][Code]
-
[3] LPT: Long-tailed Prompt Tuning for Image Classification, ICLR 2023.
Dong, Bowen and Zhou, Pan and Yan, Shuicheng and Zuo, Wangmeng.
-
[4] Pro-tuning: Unified Prompt Tuning for Vision Tasks, TCSVT 2023.
Nie, Xing and Ni, Bolin and Chang, Jianlong and Meng, Gaofeng and Huo, Chunlei and others.
[Paper][Code]
-
[5] Instance-aware Dynamic Prompt Tuning for Pre-trained Point Cloud Models, ICCV 2023.
Zha, Yaohua and Wang, Jinpeng and Dai, Tao and Chen, Bin and Wang, Zhi and Xia, Shu-Tao.
-
[6] Visual Prompt Multi-Modal Tracking, CVPR 2023.
Zhu, Jiawen and Lai, Simiao and Chen, Xin and Wang, Dong and Lu, Huchuan.
-
[7] LION: Implicit Vision Prompt Tuning, AAAI 2024.
Wang, Haixin and Chang, Jianlong and Luo, Xiao and Sun, Jinan and Lin, Zhouchen and Tian, Qi.
[Paper][Code]
-
[8] Convolutional Visual Prompt for Robust Visual Perception, NeurIPS 2023.
Tsai, Yun-Yun and Mao, Chengzhi and Yang, Junfeng.
[Paper][Code]
-
[9] ProSFDA: Prompt Learning based Source-free Domain Adaptation for Medical Image Segmentation, Arxiv 2023.
Hu, Shishuai and Liao, Zehui and Xia, Yong.
-
[10] Explicit Visual Prompting for Low-Level Structure Segmentations, CVPR 2023.
Liu, Weihuang and Shen, Xi and Pun, Chi-Man and Cun, Xiaodong.
-
[11] P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting, NeurIPS 2022.
Wang, Ziyi and Yu, Xumin and Rao, Yongming and Zhou, Jie and Lu, Jiwen.
-
[12] Exploring Visual Prompts for Adapting Large-Scale Models, Arxiv 2022.
Bahng, Hyojin and Jahanian, Ali and Sankaranarayanan, Swami and Isola, Phillip.
-
[13] Unleashing the Power of Visual Prompting At the Pixel Level, Arxiv 2023.
Wu, Junyang and Li, Xianhang and Wei, Chen and Wang, Huiyu and Yuille, Alan and Zhou, Yuyin and Xie, Cihang.
-
[14] Understanding and Improving Visual Prompting: A Label-Mapping Perspective, CVPR 2023.
Chen, Aochuan and Yao, Yuguang and Chen, Pin-Yu and Zhang, Yihua and Liu, Sijia.
-
[15] Learning to Prompt for Vision-Language Models, IJCV 2022.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, Ziwei Liu.
-
[16] Hyperprompt: Prompt-based task-conditioning of transformers, ICML 2022.
He, Yun and Zheng, Steven and Tay, Yi and Gupta, Jai and Du, Yu and Aribandi, Vamsi and others.
[Paper][Code]
-
[17] MaPLe: Multi-modal Prompt Learning, CVPR 2023.
Khattak, Muhammad Uzair and Rasheed, Hanoona and Maaz, Muhammad and others.
-
[18] Hierarchical Prompt Learning for Multi-Task Learning, CVPR 2023.
Liu, Yajing and Lu, Yuning and Liu, Hao and An, Yaozu and Xu, Zhuoran and Yao, Zhuokun and others.
[Paper][Code]
-
[19] Visual Exemplar Driven Task-Prompting for Unified Perception in Autonomous Driving, CVPR 2023.
Liang, Xiwen and Niu, Minzhe and Han, Jianhua and Xu, Hang and Xu, Chunjing and Liang, Xiaodan.
[Paper][Code]
-
[20] Dual Modality Prompt Tuning for Vision-Language Pre-Trained Model, TMM 2023.
Xing, Yinghui and Wu, Qirui and Cheng, De and Zhang, Shizhou and Liang, Guoqiang and others.
-
[21] Tokenize Anything via Prompting, Arxiv 2023.
Pan, Ting and Tang, Lulu and Wang, Xinlong and Shan, Shiguang.
-
[22] MmAP : Multi-modal Alignment Prompt for Cross-domain Multi-task Learning, AAAI 2024.
Yi Xin, Junlong Du, Qiang Wang, Ke Yan, Shouhong Ding.
[Paper][Code]
-
[23] Diversity-Aware Meta Visual Prompting, CVPR 2023.
Qidong Huang, Xiaoyi Dong, Dongdong Chen, Weiming Zhang, Feifei Wang, Gang Hua, Nenghai Yu.
-
[24] Cross-modal Prompts: Adapting Large Pre-trained Models for Audio-Visual Downstream Tasks, NeurIPS 2023.
Duan, Haoyi and Xia, Yan and Mingze, Zhou and Tang, Li and Zhu, Jieming and Zhao, Zhou.
-
[25] Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Models, AAAI 2024.
Yiwen Tang, Ray Zhang, Zoey Guo, Xianzheng Ma, Dong Wang, Zhigang Wang, Bin Zhao, Xuelong Li.
[Paper][Code]
-
[26] E2VPT: An Effective and Efficient Approach for Visual Prompt Tuning, ICCV 2023.
Cheng, Han and Qifan, Wang and Yiming, Cui and Zhiwen, Cao and Wenguan, Wang and Siyuan, Qi and Dongfang, Liu
-
[27] DGL: Dynamic Global-Local Prompt Tuning for Text-Video Retrieval, AAAI 2024.
Xiangpeng Yang and Linchao Zhu and Xiaohan Wang and Yi Yang
Prefix Tuning
-
[1] Prefix-Tuning: Optimizing Continuous Prompts for Generation, ACL 2021.
Li, Xiang Lisa and Liang, Percy.
-
[2] Towards a Unified View on Visual Parameter-Efficient Transfer Learning, Arxiv 2023.
Yu, Bruce XB and Chang, Jianlong and Liu, Lingbo and Tian, Qi and Chen, Chang Wen.
-
[3] Exploring Efficient Few-shot Adaptation for Vision Transformers, TMLR 2023.
Xu, Chengming and Yang, Siqian and Wang, Yabiao and Wang, Zhanxiong and Fu, Yanwei and Xue, Xiangyang.
-
[4] Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learning, CVPR 2023.
Tu, Cheng-Hao and Mai, Zheda and Chao, Wei-Lun.
-
[5] A Unified Continual Learning Framework with General Parameter-Efficient Tuning, ICCV 2023.
Tu, Cheng-Hao and Mai, Zheda and Chao, Wei-Lun.
Side Tuning
-
[1] Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks, ECCV 2020.
Zhang, Jeffrey O and Sax, Alexander and Zamir, Amir and Guibas, Leonidas and Malik, Jitendra.
[Paper][Code]
-
[2] LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning, NeurIPS 2022.
Sung, Yi-Lin and Cho, Jaemin and Bansal, Mohit.
-
[3] Vision transformer adapter for dense predictions. ICLR 2023.
Chen, Zhe and Duan, Yuchen and Wang, Wenhai and He, Junjun and Lu, Tong and Dai, Jifeng and Qiao, Yu.
-
[4] Side Adapter Network for Open-Vocabulary Semantic Segmentation, CVPR 2023.
Xu, Mengde and Zhang, Zheng and Wei, Fangyun and Hu, Han and Bai, Xiang.
-
[5] Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone, NeurIPS 2023.
Jiang, Zeyinzi and Mao, Chaojie and Huang, Ziyuan and Ma, Ao and Lv, Yiliang and Shen, Yujun and Zhao, Deli and Zhou Jingren.
[Paper] [Code]
-
[6] DTL: Disentangled Transfer Learning for Visual Recognition, AAAI 2024.
Fu, Minghao and Zhu, Ke and Wu, Jianxin.
-
[7] Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictions, ACM MM 2024.
Yin, Dongshuo and Han, Xueting and Li, Bin and Feng, Hao and Bai, Jing.
[Paper][Code]
-
[8] Ladder Fine-tuning approach for SAM integrating complementary network, Arxiv 2023.
Chai, Shurong and Jain, Rahul Kumar and Teng, Shiyu and Liu, Jiaqing and Li, Yinhao and others.
-
[9] End-to-End Temporal Action Detection with 1B Parameters Across 1000 Frames, CVPR 2024.
Liu, Shuming and Zhang, Chen-Lin and Zhao, Chen and Ghanem, Bernard.
[Paper] [Code]
-
[10] Time-, Memory- and Parameter-Efficient Visual Adaptation, CVPR 2024.
Mercea, Otniel-Bogdan and Gritsenko, Alexey and Schmid, Cordelia and Arnab, Anurag.
[Paper] [Code]
-
[11] Low-rank Attention Side-Tuning for Parameter-Efficient Fine-Tuning, ArXiv 2024.
Tang, Ningyuan and Fu, Minghao and Zhu, Ke and Wu, Jianxin.
[Paper] [Code]
-
[12] LoSA: Long-Short-range Adapter for Scaling End-to-End Temporal Action Localization, ArXiv 2024.
Gupta, Akshita and Mittal, Gaurav and Magooda, Ahmed and Yu, Ye and Taylor, Graham W and Chen, Mei.
[Paper] [Code]
-
[13] BarLeRIa: An Efficient Tuning Framework for Referring Image Segmentation, ICLR 2024.
Wang, Yaoming and Li, Jin and ZHANG, XIAOPENG and Shi, Bowen and Li, Chenglin and Dai, Wenrui and Xiong, Hongkai and Tian, Qi.
-
[14] UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memory, CVPR 2024.
Haiwen Diao, Bo Wan, Ying Zhang, Xu Jia, Huchuan Lu, Long Chen.
-
[15] SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning, ECCV 2024.
Haiwen Diao, Bo Wan, Xu Jia, Yunzhi Zhuge, Ying Zhang, Huchuan Lu, Long Chen.
Partial-based Tuning
Specification Tuning
-
[1] Do Better ImageNet Models Transfer Better?, CVPR 2019.
Kornblith, Simon and Shlens, Jonathon and Le, Quoc V.
-
[2] BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models. ACL 2022.
Zaken, Elad Ben and Ravfogel, Shauli and Goldberg, Yoav.
-
[3] Differentially Private Bias-Term only Fine-tuning of Foundation Models, Arxiv 2022.
Bu, Zhiqi and Wang, Yu-Xiang and Zha, Sheng and Karypis, George.
[Paper][Code]
-
[4] AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks, NAACL 2022.
Fu, Chin-Lun and Chen, Zih-Ching and Lee, Yun-Ru and Lee, Hung-yi.
-
[5] Strong Baselines for Parameter Efficient Few-Shot Fine-tuning, AAAI 2024.
Basu, Samyadeep and Massiceti, Daniela and Hu, Shell Xu and Feizi, Soheil.
[Paper][Code]
-
[6] DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning, ICCV 2023.
Enze Xie, Lewei Yao, Han Shi, Zhili Liu, Daquan Zhou, Zhaoqiang Liu, Jiawei Li, Zhenguo Li.
-
[7] Gradient-based Parameter Selection for Efficient Fine-Tuning, Arxiv 2023.
Zhi Zhang, Qizhe Zhang, Zijun Gao, Renrui Zhang, Ekaterina Shutova, Shiji Zhou, Shanghang Zhang.
[Paper][[Code]()]
-
[8] Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning, ICCV 2023.
Haoyu He, Jianfei Cai, Jing Zhang, Dacheng Tao, Bohan Zhuang.
Reparameter Tuning
-
[1] LoRA: Low-Rank Adaptation of Large Language Models. NeurIPS 2021.
Hu, Edward J and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and others.
-
[2] Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning, NeurIPS 2022.
Dongze Lian, Daquan Zhou, Jiashi Feng, Xinchao Wang.
-
[3] KronA: Parameter Efficient Tuning with Kronecker Adapter, Arxiv 2023.
Ali Edalati, Marzieh Tahaei, Ivan Kobyzev, Vahid Partovi Nia, James J. Clark, Mehdi Rezagholizadeh.
[Paper)][Code]
-
[4] FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer, AAAI 2023.
Jie, Shibo and Deng, Zhi-Hong.
[Paper][Code]
-
[5] Aggregate, Decompose, and Fine-Tune: A Simple Yet Effective Factor-Tuning Method for Vision Transformer, Arxiv 2023.
Chen, Dongping.
-
[6] Strong Baselines for Parameter Efficient Few-Shot Fine-tuning, AAAI 2024.
Basu, Samyadeep and Massiceti, Daniela and Hu, Shell Xu and Feizi, Soheil.
[Paper][Code]
-
[7] Parameter-efficient Model Adaptation for Vision Transformers, AAAI 2023.
He, Xuehai and Li, Chunyuan and Zhang, Pengchuan and Yang, Jianwei and Wang, Xin Eric.
-
[8] DnA: Improving Few-Shot Transfer Learning with Low-Rank Decomposition and Alignment, ECCV 2022.
Jiang, Ziyu and Chen, Tianlong and Chen, Xuxi and Cheng, Yu and Zhou, Luowei and Yuan, Lu and others.
-
[9] Towards Efficient Visual Adaption via Structural Re-parameterization, Arxiv 2023.
Luo, Gen and Huang, Minglang and Zhou, Yiyi and Sun, Xiaoshuai and Jiang, Guannan and Wang, Zhiyu and Ji, Rongrong.
-
[10]SAM-PARSER: Fine-tuning SAM Efficiently by Parameter Space Reconstruction, AAAI 2024.
Zelin Peng, Zhengqin Xu, Zhilin Zeng, Xiaokang Yang, Wei Shen.
[Paper][[Code]()]
-
[10]DiffuseKronA: A Parameter Efficient Fine-tuning Method for Personalized Diffusion Models, Arxiv 2023.
Shyam Marjit, Harshit Singh, Nityanand Mathur, Sayak Paul, Chia-Mu Yu, Pin-Yu Chen.
Unified Tuning
-
[1] Towards a Unified View of Parameter-Efficient Transfer Learning, ICLR 2022.
Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, Graham Neubig.
-
[2] Towards a Unified View on Visual Parameter-Efficient Transfer Learning, Arxiv 2023.
Yu, Bruce XB and Chang, Jianlong and Liu, Lingbo and Tian, Qi and Chen, Chang Wen.
-
[3] Neural Prompt Search, Arxiv 2022.
Zhang, Yuanhan and Zhou, Kaiyang and Liu, Ziwei.
-
[4] Rethinking Efficient Tuning Methods from a Unified Perspective, Arxiv 2023.
Jiang, Zeyinzi and Mao, Chaojie and Huang, Ziyuan and Lv, Yiliang and Zhao, Deli and Zhou, Jingren.
[Paper][Code]
-
[5] A Unified Continual Learning Framework with General Parameter-Efficient Tuning, ICCV 2023.
Gao, Qiankun and Zhao, Chen and Sun, Yifan and Xi, Teng and Zhang, Gang and Ghanem, Bernard and Zhang, Jian.
-
[6] GIST: Improving Parameter Efficient Fine Tuning via Knowledge Interaction, Arxiv 2023.
Jiacheng Ruan, Jingsheng Gao, Mingye Xie, Suncheng Xiang, Zefang Yu, Ting Liu, Yuzhuo Fu.
[Paper][Code]
