search

mlsys-seo / ooo-backprop

Apache License 2.0
21 stars 5 forks source link

Question about ‘cuda_graph_run’ #1

Closed Dwade3 closed 1 year ago

Dwade3 commented 2 years ago

The key function 'prepare_cuda_graph_capture' (in ./tensorflow/python/training/raining_ops.py) invokes the function 'cuda_graph_run'. But I can not find where this function is defined.

92hyungjun commented 1 year ago

The operation is defined in this line (https://github.com/mlsys-seo/ooo-backprop/blob/772446f2df1154c890d4d12860c1981600d639cc/tensorflow/tensorflow/core/ops/training_ops.cc#L25).

The actual implementation is in this line (https://github.com/mlsys-seo/ooo-backprop/blob/772446f2df1154c890d4d12860c1981600d639cc/tensorflow/tensorflow/core/kernels/training_ops_gpu.cu.cc#L70).

Based on Tensorflow convention, all Upper case letters are converted to lower case with underscore when you compile the source code. ex) "CudaGraphRun" -> "cuda_graph_run"