[Minor] Remove ortho init for DAS - Githubissues

stanfordnlp / pyvene

Stanford NLP Python Library for Understanding and Improving PyTorch Models via Interventions

http://pyvene.ai

Apache License 2.0

545 stars 46 forks source link

[Minor] Remove ortho init for DAS #160

Closed frankaging closed 1 month ago

frankaging commented 1 month ago

Description

we might not need this as registering line below might trigger this call internally: https://github.com/pytorch/pytorch/blob/main/torch/nn/utils/parametrize.py#L421

kudos to peft library PR: https://github.com/huggingface/peft/pull/1657#discussion_r1570154894

Testing Done

N.A.

Checklist:

[X] My PR title strictly follows the format: [Your Priority] Your Title
[X] I have attached the testing log above
[X] I provide enough comments to my code
[ ] I have changed documentations
[ ] I have added tests for my changes