search

alex-berard / seq2seq

Attention-based sequence to sequence learning
Apache License 2.0
388 stars 122 forks source link

Deep layer size #24

Closed jmp84 closed 5 years ago

jmp84 commented 5 years ago

According to the Librispeech AST config, a deep layer is used prior to output projection (https://github.com/eske/seq2seq/blob/master/config/LibriSpeech/AST.yaml#L42) and that layer size is 512 (https://github.com/eske/seq2seq/blob/master/config/LibriSpeech/AST.yaml#L45). However, running ./seq2seq.sh config/LibriSpeech/AST.yaml --train -v --purge shows that the actual deep layer size is 128 (the target embedding size). In https://github.com/eske/seq2seq/blob/master/translate/models.py#L776, decoder.pred_deep_layer_size is None so decoder.embedding_size is used. Would you be able to confirm which size is used for https://github.com/eske/seq2seq/blob/master/config/LibriSpeech/model-outputs.tar.xz ? Thanks in advance.

alex-berard commented 5 years ago

You're absolutely right. Thanks for bringing this to my attention. It seems like the attribute name is pred_deep_layer_size, and I have been using deep_layer_size in the configuration files. In the models provided on the README page (which were used to generate the model outputs), the actual deep layer size was indeed 128 (I checked in the log files).