recommenders-team / recommenders

Best Practices on Recommendation Systems
https://recommenders-team.github.io/recommenders/intro.html
MIT License
19.12k stars 3.09k forks source link

[ASK] The Training using a GPU is very slow for NCF model #2131

Open orrimoch opened 3 months ago

orrimoch commented 3 months ago

I've set up an virtual environment based on the following commands: !conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1 !pip install recommenders[gpu]

Moreover I've added the following commands in the code:

tf.debugging.set_log_device_placement(True) gpus = tf.config.experimental.list_physical_devices('GPU') tf.config.set_visible_devices([], 'CPU') # hide the CPU tf.config.set_visible_devices(gpus[0], 'GPU') # unhide potentially hidden GPU tf.config.get_visible_devices()

Set the environment to use GPU

physical_devices = tf.config.list_physical_devices('GPU') if physical_devices: tf.config.experimental.set_memory_growth(physical_devices[0], True)

Non of the steps above helped me to increase the utilization of the gpu. Any suggestion would be helpful.

Thanks, Or