search

microsoft / CNTK

Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit
https://docs.microsoft.com/cognitive-toolkit/
Other
17.53k stars 4.28k forks source link

CNTK layer dropout #3510

Open christopher5106 opened 6 years ago

christopher5106 commented 6 years ago

Hi,

I would like to implement a layer dropout (not simple Dropout but full LayerDropout). For that, I need a random number generator operator and a switch operator, I believe. Here is the implementation in tensorflow:

    pred = tf.random_uniform([]) < self.dropout
    x_train = tf.cond(pred, lambda: residual, lambda: tf.nn.dropout(x, 1.0 - self.dropout) + residual)

How can I achieve the same in CNTK ?

Thank you for your answer

acifonelli commented 5 years ago

According to the, still opened, issue #1371 we can not random sampling from a distribution in CNTK.