nv-morpheus / Morpheus

Morpheus SDK
Apache License 2.0
333 stars 124 forks source link

[FEA]: Limit pandas conversions in DFP modules/stages #1635

Open dagardner-nv opened 4 months ago

dagardner-nv commented 4 months ago

Is this a new feature, an improvement, or a change to existing functionality?

Improvement

How would you describe the priority of this feature request

Low (would be nice)

Please provide a clear description of problem this feature solves

The DFP pipeline contains a few pandas conversions which can be costly in terms of performance, and requires storing a second copy of the cudf df.

Describe your ideal solution

Perform as many operations as possible in cudf.

Additional context

No response

Code of Conduct