Migrate to `peft` from `opendelta` for parameter efficient tuning methods

[jon-tow]

🚀 The feature, motivation, and pitch

Let's migrate to peft.

Tasks

Doing so will require the following updates:

1. Replace the opendelta setup in the AccelerateBaseTrainer with a peft backed setup: https://github.com/CarperAI/trlx/blob/92b68e4d8c5d59e6ba25d12fd9acfe10287be689/trlx/trainer/accelerate_base_trainer.py#L145-L155

2. Handle fine-grained layer capturing to only modify the upper trunk layers of hydra architectures as handled below: https://github.com/CarperAI/trlx/blob/92b68e4d8c5d59e6ba25d12fd9acfe10287be689/trlx/utils/modeling.py#L414-L428

Motivation

Citing @ethankim00's concerns with opendelta:

• opendelta import fails due to an unnecessary turtle package import. Even if pip installed, users may be required to have sudo privileges to install the corresponding base graphics package https://github.com/thunlp/OpenDelta/issues/47
• Doesn’t seem to work with DeepSpeed ZeRO 3
• Additional inference overhead from not merging in the LoRA adapters layers
• Incompatibility with int8 training
• Less actively maintained than the peft library, which has been growing rapidly
• Sharing adapter weights on the HuggingFace Hub is less convenient with opendelta

Alternatives

No response

Additional context

No response

[jon-tow]

Assignee: @glerzing will be taking a go at this :)

[loganlebanoff]

I'm curious as to the status of this issue/PR

[glerzing]

I'm developing automated tests for it, there should be a PR soon.

[akk-123]

@glerzing look forward it

[akk-123]

@glerzing when will have a pr?

[glerzing]

It should be ready for PR tomorrow, sorry for the wait.