search

dmmiller612 / sparktorch

Train and run Pytorch models on Apache Spark.
MIT License
339 stars 44 forks source link

Why is nobody using this? #26

Open adsk2050 opened 2 years ago

adsk2050 commented 2 years ago

I am new to doing distributed training and inference of ML models. If I have say 4 GPU nodes in a spark cluster, will this library help me train and do inference on the models without having to go into too much detail? If yes, I am wondering why nobody is using this project. If no, what exactly is the function of this library?

dmmiller612 commented 2 years ago

It does have very limited publicity, but based on pypi activity people are using the library. Since the modeling is focused around spark ml/dataframes, if your objective fits within that ecosystem, it could be a fit.