Open Data-drone opened 3 years ago
Hi @Data-drone! Sure, that will be nice to have but for the future. Right now, I'm focused on making AutoML work nicely with good results on the CPU. Most tabular datasets that I've seen are small < 4GB. Working on AutoML and at the same time fighting with constantly changing drivers for GPU might be a nightmare.
For Neural Networks at first, I was using Keras+TF. But I have some problems with them:
That's why I switched to MLP from scikit-learn.
Anyway, in the future, it will be nice to have support for GPU.
I think you can consider fast.ai -> tabular NN with GPU. As far I remember for xgboost GPU is about a parameter or two in configuration. Probably similar for catboost and lightGBM.
@mglowacki100 have you tested fast.ai tabular NN? do they have good performance?
I don't like the fact that I need to transform pandas data frame to their own table format before training. Look's like a huge framework, so I'm a little skeptical.
Yes, switch GBM algorithms to GPU should be straightforward, but there additional constraints that need to be implemented. For example, not all parameters/metrics work with GPU. Would you like to take a look at it?
@pplonski I'll ve some time in the next month for gbm algos. I think the easiest and most flexible approach is just to clone e.g. xgboost as separate algo with gpu-adjusted params and let's call it Xgboost_GPU in algorithms parameters.
Yeah there is https://github.com/rapidsai/cuml for some of the other algorithms as well
I have problem with Tensorflow on my server as well (seems like it has to do with GPU + CUDA version). However Pytorch installation is a breeze. Maybe we can use Pytorch instead of Tensorflow? Plus the transformers from the hugging face library are written in Pytorch anyway.
@pplonski any updates on this? Catboost supports GPU nicely out of the box so I think at least some of the training process could greatly be speed up in Mljar through the GPU
Any updates?
Seems like it might be one way to get more performance for larger problems