Closed leewyang closed 4 days ago
python qualx_main.py train @leewyang There is a CLI for train
spark_rapids train
, right? just making sure that the CLI is not falling behind to be used.
Yes, I recently tested spark_rapids train
CLI per #1140. It pretty much just wraps the same code, so I think it's fine.
This PR adds a plugin mechanism to invoke dataset-specific code to modify the pandas dataframe returned by the qualx
load_profiles()
function. This is intended to allow custom handlers for one-off cases which shouldn't be introduced into the main codebase.The path to the plugin module should be specified within the dataset JSON file with the "load_profiles_hook" key, e.g.
The plugin module should define a function with the following signature:
Changes
load_profiles_hook
.--output-sql-ids-aligned
argument to Profiler invocations (for future use).Test
Following CMDs have been tested:
Internal Usage: