Lora

[mrudat-iais]

What does this PR do?

This PR ..

General Changes

• ..

Breaking Changes

• ..

Checklist before submitting final PR

• [ ] My PR is minimal and addresses one issue in isolation
• [ ] I have merged the latest version of the target branch into this feature branch
• [ ] I have reviewed my own code w.r.t. correct implementation, missing type hints, proper documentation, etc.
• [ ] I have run a sample config for model training
• [ ] I have checked that all tests run through (python tests/tests.py)

[mrudat-iais]

Open TODOs

• [x] finish conversion function
• [x] think about the configuration on what modules to convert (e.g. "attn")
• [ ] convert model before training if lora config parameters exist
• [x] complete tests
• [ ] create PR with medhi / michael / max as reviewers