logix-project / logix

AI Logging for Interpretability and Explainability🔬
Apache License 2.0
86 stars 6 forks source link

InfluenceFunction precondition function fails when logging does not set hessian=True #26

Closed YoungseogChung closed 11 months ago

YoungseogChung commented 11 months ago

When logging over the dataset as with analog(data_id=data_id, log=["grad"], save=True): instead of with analog(data_id=data_id, log=["grad"], hessian=True, save=True):, the precondition function of InfluenceFunction fails since module_eigvec["backward"] is empty but it still calls module_eigvec["backward"].t() in line 25 of influence_function.py:

File "/Users/youngsec/research/if/analog/analog/analysis/influence_function.py", line 26, in precondition module_eigvec["backward"].t(), AttributeError: 'collections.defaultdict' object has no attribute 't'

sangkeun00 commented 11 months ago

Previously, hessian is set to True by default, but I changed the default value to False in one of my recent commits. So, this is an expected behavior.

YoungseogChung commented 11 months ago

I see. Shouldn't the precondition function still handle the case where hessian is not logged? E.g. check if it's emtpy and exit

sangkeun00 commented 11 months ago

That's a good point. If you can add error handling for this, that would be great.