NVIDIA / OpenSeq2Seq

Toolkit for efficient experimentation with Speech Recognition, Text2Speech and NLP
https://nvidia.github.io/OpenSeq2Seq
Apache License 2.0
1.54k stars 369 forks source link

ds2_small_1gpu.py cannot compute modelsize #316

Open David-Levinthal opened 5 years ago

David-Levinthal commented 5 years ago

attempting to run the ds2 small training example per the instructions prints out a "warning" that it cannot determine model size ~/OpenSeq2Seq$ python3.6 run.py --config_file=example_configs/speech2text/ds2_small_1gpu.py --mode=train_eval> eval_ds2_2.log 2>&1 Building graph on GPU:0 Trainable variables: ForwardPass/ds2_encoder/conv1/kernel:0 shape: (11, 41, 1, 32), <dtype: 'float32_ref'> ForwardPass/ds2_encoder/conv1/bn/gamma:0 shape: (32,), <dtype: 'float32_ref'> ForwardPass/ds2_encoder/conv1/bn/beta:0 shape: (32,), <dtype: 'float32_ref'> ForwardPass/ds2_encoder/conv2/kernel:0 shape: (11, 21, 32, 32), <dtype: 'float32_ref'> ForwardPass/ds2_encoder/conv2/bn/gamma:0 shape: (32,), <dtype: 'float32_ref'> ForwardPass/ds2_encoder/conv2/bn/beta:0 shape: (32,), <dtype: 'float32_ref'> ForwardPass/ds2_encoder/cudnn_gru/opaque_kernel:0 shape: , <dtype: 'float32_ref'> ForwardPass/ds2_encoder/fully_connected/kernel:0 shape: (1024, 1024), <dtype: 'float32_ref'> ForwardPass/ds2_encoder/fully_connected/bias:0 shape: (1024,), <dtype: 'float32_ref'> ForwardPass/fully_connected_ctc_decoder/fully_connected/kernel:0 shape: (1024, 29), <dtype: 'float32_ref'> ForwardPass/fully_connected_ctc_decoder/fully_connected/bias:0 shape: (29,), <dtype: 'float32_ref'> Encountered unknown variable shape, can't compute total number of parameters. Building graph on GPU:0

being able to determine model size is important in checking the model is what is expected

David-Levinthal commented 5 years ago

This error occurs on all variations of the ds2 applications. this makes it impossible to really know what model is being used and what the size is. printing out the arrays is a rather critical feature