berkeleydeeprlcourse / homework

Assignments for CS294-112.
MIT License
1.56k stars 1.04k forks source link

hw1 dropout implementation #23

Open nick6918 opened 6 years ago

nick6918 commented 6 years ago

In tf_utils.py HW1, I found dropout implementation as follows,

def dropout(x, pkeep, phase=None, mask=None):
    mask = tf.floor(pkeep + tf.random_uniform(tf.shape(x))) if mask is None else mask
    if phase is None:
        return mask * x
    else:
        return switch(phase, mask*x, pkeep*x)

In test phase, should it be x/pkeep instead of x*pkeep(based on inverse dropout theory)? If not, why?

Thanks for your explanation.