search

h2oai / h2o4gpu

H2Oai GPU Edition
Apache License 2.0
460 stars 95 forks source link

Port h2o4gpu to h2o-3 #753

Open jtelleria opened 5 years ago

jtelleria commented 5 years ago

Could it be possible to port h2o4gpu Algorithms to h2o-3?

mdymczyk commented 5 years ago

@jtelleria by port I'm assuming you're asking for GPU implementation in h2o-3 since h2o-3 already has almost all the algos h2o4gpu has.

This was actually the reason why we moved from using CTypes to SWIG so we could use it from Java and h2o-3 but I never had the time to integrate it and I doubt @sh1ng (who's now in charge of this repo) has.

Furthermore, h2o4gpu algos are not distributed (or well not in the h2o-3 sense) so they would still run only on a single machine.

You're welcome to try and contribute this to the h2o-3 repository, if you want though. We already did something similar with XGboost so you could use that as an example.

sh1ng commented 5 years ago

Due to compilation for multiple GPU's architectures h2o4gpu package is quite big(>300Mb). People who is not going to use it should not pay for it. We could implement lazy loading of the package, but that plus supporting Java binding and unification of the API will take some time. Not our first priority(in favour to more CUDA algos), but let's keep.

mdymczyk commented 5 years ago

@sh1ng the module system in h2o-3 is based on Java classpath mechanism so we probably could just make the h2o4gpu a separate Java jar and users would just pass it to the classpath when starting h2o-3, don't need to bundle it with h2o-3.jar. But the other issues you raised still stay :-)