MTG / essentia

C++ library for audio and music analysis, description and synthesis, including Python bindings
http://essentia.upf.edu
GNU Affero General Public License v3.0
2.86k stars 534 forks source link

TF_FORCE_GPU_ALLOW_GROWTH seems to have no effect #1432

Open bthj opened 4 months ago

bthj commented 4 months ago

Hi,

When running inference with some of the classifiers while using the GPU support, all available GPU memory is utilised. This is a known TensorFlow behaviour which can be controlled by setting the environment variable TF_FORCE_GPU_ALLOW_GROWTH=true or directives in Pyhon code, which has worked for me so far, but when performing inference through Essentia, this doesn't seem to have any effect (and my processes are killed by Slurm on an HPC, reporting CUDA_ERROR_OUT_OF_MEMORY).

Could I control GPU memory usage with Essentia in some other way?

pip show essentia-tensorflow
Name: essentia-tensorflow
Version: 2.1b6.dev1110
Summary: Library for audio and music analysis, description and synthesis, with TensorFlow support
Home-page: http://essentia.upf.edu
Author: Dmitry Bogdanov
Author-email: dmitry.bogdanov@upf.edu
License: AGPLv3
Location: /usr/local/lib/python3.10/dist-packages
Requires: numpy, pyyaml, six
Required-by: