stanfordnlp / pyvene

Stanford NLP Python Library for Understanding and Improving PyTorch Models via Interventions
http://pyvene.ai
Apache License 2.0
608 stars 59 forks source link

[Minor] Fix gradient backprop trainables with upstream interventions #97

Closed frankaging closed 8 months ago

frankaging commented 8 months ago

📝 Description

Currently, when using DAS with upstream interventions, gradient may throw arrow. This is due to some side-effect issues. We fix this by always-clone the passing activations in hook calls.

Testing Done

----------------------------------------------------------------------
Ran 49 tests in 43.641s

OK

Checklist: