d2l-ai / d2l-en

Interactive deep learning book with multi-framework code, math, and discussions. Adopted at 500 universities from 70 countries including Stanford, MIT, Harvard, and Cambridge.
https://D2L.ai
Other
23.24k stars 4.27k forks source link

try_gpu(), try_gpu(10), try_all_gpus() do not provide meaningful outputs in TF #1290

Closed astonzhang closed 4 years ago

astonzhang commented 4 years ago

http://preview.d2l.ai.s3-website-us-west-2.amazonaws.com/d2l-en/master/chapter_deep-learning-computation/use-gpu.html

Text: Now we define two convenient functions that allow us to run code even if the requested GPUs do not exist.

Code:

try_gpu(), try_gpu(10), try_all_gpus()

(<tensorflow.python.eager.context._EagerDeviceContext at 0x7f7de4269750>,
 <tensorflow.python.eager.context._EagerDeviceContext at 0x7f7de42698d0>,
 [<tensorflow.python.eager.context._EagerDeviceContext at 0x7f7de4269910>,
  <tensorflow.python.eager.context._EagerDeviceContext at 0x7f7de4269a10>])

Thus, the output is inconsistent with text descriptions. Is it possible to show similar outputs as in mx and pt?

terrytangyuan commented 4 years ago

I thought about this earlier. Something like the following would give more info:

str(try_gpu()._ctx)
> 'Eager TensorFlow Context with 2 devices\n   Device 0: /job:localhost/replica:0/task:0/device:CPU:0\n   Device 1: /job:localhost/replica:0/task:0/device:XLA_CPU:0'

However, this is using an internal method and not stable. Also, for try_gpus(), we can format the string inside the function definition but for cases like tf.device('/CPU:0'), tf.device('/GPU:0'), tf.device('/GPU:1'), it will look very inelegant and inconsistent with other implementations.

astonzhang commented 4 years ago

Got it. In that case, do you suggest that we just modify text for tf?

terrytangyuan commented 4 years ago

I am not sure which part of the text is inconsistent but yes feel free to update the text if necessary and leave the code as is for now.

terrytangyuan commented 4 years ago

@astonzhang The text looks fine though since we did not mention anything related to the output of these functions: "convenient functions that allow us to run code even if the requested GPUs do not exist." Could we close this then?

astonzhang commented 4 years ago

@terrytangyuan Sure, though the output is not very illustrative.