openai / finetune-transformer-lm

Code and model for the paper "Improving Language Understanding by Generative Pre-Training"
https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf
MIT License
2.14k stars 499 forks source link

Question about "rf" parameter in conv1d() function #31

Open XisnZhang opened 5 years ago

XisnZhang commented 5 years ago
    if rf == 1:  # faster 1x1 conv
       c = tf.reshape(tf.matmul(tf.reshape(x, [-1, nx]), tf.reshape(w, [-1, nf])) + b, shape_list(x)[:-1] + [nf])
    else:  # was used to train LM
        c = tf.nn.conv1d(x, w, stride=1, padding=pad) + b  

when training LM, rf = ? and why?