NVIDIA / OpenSeq2Seq

Toolkit for efficient experimentation with Speech Recognition, Text2Speech and NLP
https://nvidia.github.io/OpenSeq2Seq
Apache License 2.0
1.54k stars 369 forks source link

Tried to allocate nullptr when batch =1 #287

Closed bmwshop closed 5 years ago

bmwshop commented 5 years ago

We are getting a lot of warnings / errors below when running inference / eval with batch size = 1 on the ds2 ASR model. We understand that we should keep batch size to 1 per the docs

2018-11-20 17:45:39.130201: E tensorflow/core/common_runtime/bfc_allocator.cc:246] tried to allocate 0 bytes 2018-11-20 17:45:39.130247: W tensorflow/core/common_runtime/allocator_retry.cc:32] Request to allocate 0 bytes 2018-11-20 17:45:39.130256: E tensorflow/core/common_runtime/bfc_allocator.cc:246] tried to allocate 0 bytes 2018-11-20 17:45:39.130263: W tensorflow/core/common_runtime/allocator_retry.cc:32] Request to allocate 0 bytes 2018-11-20 17:45:39.130284: E tensorflow/core/common_runtime/bfc_allocator.cc:381] tried to deallocate nullptr 2018-11-20 17:45:39.130293: E tensorflow/core/common_runtime/bfc_allocator.cc:381] tried to deallocate nullptr 2018-11-20 17:45:39.408121: E tensorflow/core/common_runtime/bfc_allocator.cc:246] tried to allocate 0 bytes 2018-11-20 17:45:39.408166: W tensorflow/core/common_runtime/allocator_retry.cc:32] Request to allocate 0 bytes 2018-11-20 17:45:39.408175: E tensorflow/core/common_runtime/bfc_allocator.cc:246] tried to allocate 0 bytes 2018-11-20 17:45:39.408182: W tensorflow/core/common_runtime/allocator_retry.cc:32] Request to allocate 0 bytes 2018-11-20 17:45:39.408198: E tensorflow/core/common_runtime/bfc_allocator.cc:381] tried to deallocate nullptr 2018-11-20 17:45:39.408206: E tensorflow/core/common_runtime/bfc_allocator.cc:381] tried to deallocate nullptr 2018-11-20 17:45:39.496598: E tensorflow/core/common_runtime/bfc_allocator.cc:246] tried to allocate 0 bytes 2018-11-20 17:45:39.496640: W tensorflow/core/common_runtime/allocator_retry.cc:32] Request to allocate 0 bytes 2018-11-20 17:45:39.496650: E tensorflow/core/common_runtime/bfc_allocator.cc:246] tried to allocate 0 bytes 2018-11-20 17:45:39.496656: W tensorflow/core/common_runtime/allocator_retry.cc:32] Request to allocate 0 bytes 2018-11-20 17:45:39.496674: E tensorflow/core/common_runtime/bfc_allocator.cc:381] tried to deallocate nullptr 2018-11-20 17:45:39.496683: E tensorflow/core/common_runtime/bfc_allocator.cc:381] tried to deallocate nullptr 2018-11-20 17:45:39.934819: E tensorflow/core/common_runtime/bfc_allocator.cc:246] tried to allocate 0 bytes 2018-11-20 17:45:39.934863: W tensorflow/core/common_runtime/allocator_retry.cc:32] Request to allocate 0 bytes 2018-11-20 17:45:39.934873: E tensorflow/core/common_runtime/bfc_allocator.cc:246] tried to allocate 0 bytes 2018-11-20 17:45:39.934880: W tensorflow/core/common_runtime/allocator_retry.cc:32] Request to allocate 0 bytes 2018-11-20 17:45:39.934900: E tensorflow/core/common_runtime/bfc_allocator.cc:381] tried to deallocate nullptr 2018-11-20 17:45:39.934910: E tensorflow/core/common_runtime/bfc_allocator.cc:381] tried to deallocate nullptr 2018-11-20 17:45:39.995404: E tensorflow/core/common_runtime/bfc_allocator.cc:246] tried to allocate 0 bytes 2018-11-20 17:45:39.995446: W tensorflow/core/common_runtime/allocator_retry.cc:32] Request to allocate 0 bytes 2018-11-20 17:45:39.995455: E tensorflow/core/common_runtime/bfc_allocator.cc:246] tried to allocate 0 bytes 2018-11-20 17:45:39.995462: W tensorflow/core/common_runtime/allocator_retry.cc:32] Request to allocate 0 bytes 2018-11-20 17:45:39.995482: E tensorflow/core/common_runtime/bfc_allocator.cc:381] tried to deallocate nullptr 2018-11-20 17:45:39.995491: E tensorflow/core/common_runtime/bfc_allocator.cc:381] tried to deallocate nullptr [dbslp1827:00226] Process received signal