Closed tingwl0122 closed 3 months ago
Hi @TheaperDeng, I do see in our current design doc that this dropout helper only works for models that already have dropout layers. Should we adapt this to models that don't? We can probably insert dropout layers in specific places.
@TheaperDeng
fix the two comments together by raising an error whenever there is something in the layer_name
is not a Dropout module.
LGTM, please merge this if you have nothing to add
Do you think we need to add the function for inserting dropout layers? Or we can just assume the original model already have some dropout layers and we are only able to open those during test time.
LGTM, please merge this if you have nothing to add
Do you think we need to add the function for inserting dropout layers? Or we can just assume the original model already have some dropout layers and we are only able to open those during test time.
I think we can now stick to assuming dropout layers are added by users.
LGTM, please merge this if you have nothing to add
Do you think we need to add the function for inserting dropout layers? Or we can just assume the original model already have some dropout layers and we are only able to open those during test time.
I think we can now stick to assuming dropout layers are added by users.
Got it. Then this should be ready for merging.
Description
This PR is a prototype of a function that activates dropout layers in a PyTorch model.
3. What tests have been added/updated for the change?