search

rapidsai / cudf

cuDF - GPU DataFrame Library
https://docs.rapids.ai/api/cudf/stable/
Apache License 2.0
7.99k stars 866 forks source link

[QST] Question about the cuDF.pandas interception #16119

Open shou123 opened 3 days ago

shou123 commented 3 days ago

** Hi there,

I am working on understanding the cuDF.pandas source code and have a few questions:

  1. When does cuDF.pandas enable interception to run fast_lib on the GPU? Is this logic defined in module_accelerator.py or fast_slow_proxy.py?
  2. Is there a call stack map available for cuDF.pandas? **
Matt711 commented 3 days ago

Hi @shou123 , thanks for question!

  1. When does cuDF.pandas enable interception to run fast_lib on the GPU? Is this logic defined in module_accelerator.py or fast_slow_proxy.py?

cudf.pandas works via a fast-slow proxy mechanism. It wraps Pandas types and functions into proxy types and functions that include their fast (cuDF) and slow (Pandas) versions. The proxy objects are executed on the GPU and fallback to the CPU when needed. You can find more information in the documentation here. The functionality you're referring to is in _fast_slow_function_call.

  1. Is there a call stack map available for cuDF.pandas?

There is a profiler which can show which functions executed where (GPU or CPU). And we're working on more ways to better track proxied function calls.