rapidsai / cuml

cuML - RAPIDS Machine Learning Library
https://docs.rapids.ai/api/cuml/stable/
Apache License 2.0
4.25k stars 534 forks source link

Switch to native traceback in `cuml` #6078

Open galipremsagar opened 1 month ago

galipremsagar commented 1 month ago

In cudf we have observed a ~10% speed up of pytest suite execution by switching pytest traceback to --native:

currently:

102474 passed, 2117 skipped, 902 xfailed in 892.16s (0:14:52)

--tb=short:

102474 passed, 2117 skipped, 902 xfailed in 898.99s (0:14:58)

--tb=no:

102474 passed, 2117 skipped, 902 xfailed in 815.98s (0:13:35)

--tb=native:

102474 passed, 2117 skipped, 902 xfailed in 820.92s (0:13:40)

This PR makes similar change to cuml.

xref: https://github.com/rapidsai/cudf/pull/16851

dantegd commented 1 month ago

@galipremsagar those times are for the cuDF pytest suite? It might be worth to check the time savings for cuML, I got a fresh container so can check those times in the next couple of days if so

galipremsagar commented 1 month ago

@dantegd Yes, the timing improvements I posted above are of cudf pytest suite. cuml seems to have even better speedups, almost ~20% faster:

This PR:

==== 466 skipped, 9 xfailed, 201 xpassed, 25 warnings in 1432.43s (0:23:52) ====

link: https://github.com/rapidsai/cuml/actions/runs/11002114744/job/30550052027?pr=6078#step:7:1802

Other most recently merged PR:

==== 466 skipped, 9 xfailed, 201 xpassed, 3 warnings in 1696.39s (0:28:16) =====

link: https://github.com/rapidsai/cuml/actions/runs/10999896809/job/30543198516#step:7:1622

trivialfis commented 1 month ago

Super nice! I should try this for XGBoost as well.

dantegd commented 1 month ago

/merge

divyegala commented 1 month ago

@galipremsagar could you please check what's going wrong with the pytests in this PR? I can't quite figure out what the error is exactly.