TRAIS-Lab / dattri

`dattri` is a PyTorch library for developing, benchmarking, and deploying efficient data attribution algorithms.
https://trais-lab.github.io/dattri/
24 stars 8 forks source link

[dattri.model_utils] Add utils for dropout #65

Closed tingwl0122 closed 3 months ago

tingwl0122 commented 3 months ago

Description

This PR is a prototype of a function that activates dropout layers in a PyTorch model.

3. What tests have been added/updated for the change?

tingwl0122 commented 3 months ago

Hi @TheaperDeng, I do see in our current design doc that this dropout helper only works for models that already have dropout layers. Should we adapt this to models that don't? We can probably insert dropout layers in specific places.

tingwl0122 commented 3 months ago

@TheaperDeng
fix the two comments together by raising an error whenever there is something in the layer_name is not a Dropout module.

tingwl0122 commented 3 months ago

LGTM, please merge this if you have nothing to add

Do you think we need to add the function for inserting dropout layers? Or we can just assume the original model already have some dropout layers and we are only able to open those during test time.

TheaperDeng commented 3 months ago

LGTM, please merge this if you have nothing to add

Do you think we need to add the function for inserting dropout layers? Or we can just assume the original model already have some dropout layers and we are only able to open those during test time.

I think we can now stick to assuming dropout layers are added by users.

tingwl0122 commented 3 months ago

LGTM, please merge this if you have nothing to add

Do you think we need to add the function for inserting dropout layers? Or we can just assume the original model already have some dropout layers and we are only able to open those during test time.

I think we can now stick to assuming dropout layers are added by users.

Got it. Then this should be ready for merging.