search

NVIDIA / spark-rapids-ml

Spark RAPIDS MLlib – accelerate Apache Spark MLlib with GPUs
https://nvidia.github.io/spark-rapids-ml/
Apache License 2.0
68 stars 30 forks source link

initial attempt at enabling no package import change #725

Closed eordentlich closed 2 months ago

eordentlich commented 2 months ago

still needs documentation, test, lint type checking etc. The file python/test_no_import_change.py can be tried for some basic testing.

# to run on cpu
CUDA_VISIBLE_DEVICES=0 python python/test_no_import_change.py 0.2
# run on gpu 
CUDA_VISIBLE_DEVICES=0 python -m spark_rapids_ml python/test_no_import_change.py 0.2
# gpu using spark-submit (for cpu just omit the __main__.py)
CUDA_VISIBLE_DEVICES=0 spark-rapids-submit --master local[2] \
python/test_no_import_change.py 0.2
eordentlich commented 2 months ago

build

eordentlich commented 2 months ago

build

wbo4958 commented 2 months ago

I just executed the python -m spark_rapids_ml but showed nothing. Is this expected? or should print the help usage?

eordentlich commented 2 months ago

Yes we could print usage in that case. Good suggestion.

eordentlich commented 2 months ago

build

eordentlich commented 2 months ago

build