nnaisense / evotorch

Advanced evolutionary computation library built directly on top of PyTorch, created at NNAISENSE.
https://evotorch.ai
Apache License 2.0
1.01k stars 63 forks source link

Hard-code network_device property to cpu for GymNE #6

Closed NaturalGradient closed 2 years ago

NaturalGradient commented 2 years ago

Currently there are scenarios where the user can have the network_device property return some form of torch.device('cuda:x') when using the evotorch.neuroevolution.GymNE neuroevolution class.

This class is not designed to support Policy networks on the GPU, as the environments' observations are returned as numpy arrays which are then cast to torch tensors on the CPU. This then causes a bug (for example: see #4 ) where the user is unable to run simple GymNE experiments.

In future releases, we should consider the possibility that the user may want the Policy network on the GPU -- especially if we look to supporting other (GPU-based) environments. However, in the short term, there is no mechanism for observations even being placed on a non-cpu device. This pull-request therefore offers a transparent short-term fix that complies with our current expectation of GymNE's usage:

codecov-commenter commented 2 years ago

Codecov Report

Merging #6 (5168b9c) into master (fb71a8a) will increase coverage by 0.01%. The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master       #6      +/-   ##
==========================================
+ Coverage   50.51%   50.52%   +0.01%     
==========================================
  Files          33       33              
  Lines        4702     4705       +3     
==========================================
+ Hits         2375     2377       +2     
- Misses       2327     2328       +1     
Impacted Files Coverage Δ
src/evotorch/neuroevolution/gymne.py 27.50% <66.66%> (+0.59%) :arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

flukeskywalker commented 2 years ago

Let's add a note to the Gym section in the User Guide clarifying that all computation happens on CPU when using GymNE.