How to turn on proper bucket support?

HadoopIt / rnn-nlu

A TensorFlow implementation of Recurrent Neural Networks for Sequence Classification and Sequence Labeling

483 stars 171 forks source link

How to turn on proper bucket support? #3

Closed MtDersvan closed 7 years ago

MtDersvan commented 7 years ago

Hi, thank you for the code. How to turn on properly a support for buckets of different size? This model works properly only with 1 bucket size right now.

HadoopIt commented 7 years ago

You may refer to the TensorFlow sequence-to-sequence model example for further details on bucketing and padding. Basically, you can define a number of buckets of various size according to the sequence length in your corpus, and put each data sample to the corresponding bucket. More details can be found at: https://www.tensorflow.org/tutorials/seq2seq/#bucketing_and_padding