NVIDIA / spark-rapids-tools

User tools for Spark RAPIDS
Apache License 2.0
44 stars 34 forks source link

Port training model code into user tools #1060

Closed parthosa closed 1 month ago

parthosa commented 1 month ago

Fixes #909.

We want to import the training module from QualX and update the prediction code with latest changes. This will be done in four steps.

Steps

  1. Import the source code and pretrained models as it as. Exclude these from pytest and flake8 linters.
  2. Make changes in the qualx files to connect them with tools.
  3. Make changes in the tools to use the latest changes in qualx.
  4. Add entry point for train by adding a new CMD for training.

Note:

Directory structure for QualX:

Test:

leewyang commented 1 month ago

LGTM