combust / mleap

MLeap: Deploy ML Pipelines to Production
https://combust.github.io/mleap-docs/
Apache License 2.0
1.5k stars 310 forks source link

Will Mleap support Scala 2.13? #811

Open masi-ghodrati opened 2 years ago

masi-ghodrati commented 2 years ago

We have planned to use Mleap in our framework. Our framework uses Scala 2.13. We can't downgrade to 2.12, as we use some of the new features. I would like to know if you plan on upgrading to Scala 2.13 in the near future? ? And if there is any documentation/resource that can help us upgrade to the right Scala version and make Combust compatible with our framework?

jsleight commented 2 years ago

We are on spark 3.2 which is cross compatible for scala 2.12 and 2.13. If you (or anyone) wants to submit a PR to cross compile mleap for both scala 2.12 and 2.13, then I'm happy to review it and get it merged.

Skimming through the scala migration guide it seems the biggest changes are w.r.t. some collections APIs that I suspect mleap doesn't use too much.

masi-ghodrati commented 2 years ago

thank you for your prompt response.I'm gonna work on that.thanks again

masi-ghodrati commented 2 years ago

There are some dependencies which are not supported by Scala 2.13, such as xgboost4j. It's a blocker for updating the entire modules to Scala 2.13. Since our framework only uses the runtime module, do you think we could update it to 2.13 and make it compatible with our framework?

jsleight commented 2 years ago

Ah, that makes sense. In principle I think you can make only some of the modules be cross-compiled with scala 2.13 if you tweaked https://github.com/combust/mleap/blob/master/project/MleapProject.scala. Right nowthe build settings are all shared from Common.buildSettings, but if you want to make a new Common.buildSettingsNoScala2_13 and use that for the xgboost modules then I think that is ok for now.

masi-ghodrati commented 2 years ago

thank you

nicster commented 1 year ago

It seems that xgboost4j now added Scala 2.13 support: https://github.com/dmlc/xgboost/pull/9099. It isn't released yet though, but something to keep an eye on.

dotbg commented 1 year ago

This issue was one of the motivators to open that PR :) We can test things with the snapshot version.

"XGBoost4J Snapshot Repo" at "https://s3-us-west-2.amazonaws.com/xgboost-maven-repo/snapshot/" 

I'll try to ask whether there are plans on making a new xgboost release

scheruku-in commented 1 year ago

Hi @jsleight We are using Spark 3.3 and invoke the below mleap modules in our project:

bundle-ml
mleap-base
mleap-core
mleap-runtime

We plan to upgrade to Scala 2.13 , so I would like to know if the Scala 2.13 support is going to be added for mleap?

Thanks

jsleight commented 1 year ago

860 Is working on it.

scheruku-in commented 1 year ago

Thanks @jsleight Is there any ETA on when the 2.13 build will be available?

jsleight commented 1 year ago

Don't know, could ask on that PR.

dotbg commented 1 year ago

In PR 864 I've took slightly different approach, ensuring that all other mleap dependencies support scala 2.13. I will consider going forward after that MR is reviewed.

juliangp commented 11 months ago

@dotbg do you have an estimate for when you will have a Scala 2.13 version released?

dotbg commented 11 months ago

I'd love to be able to do that, but I am too busy at my current job at this moment, sorry.

juliangp commented 11 months ago

@dotbg is there something that we can do to help?

jsleight commented 11 months ago

864 is merged so in theory someone can make the project cross compile now without needing to worry about the dependencies being compatible. If anyone wants to start a PR, I'm happy to review and get things merged.

prashil1996 commented 11 months ago

@jsleight I need to use mleap with scala 2.13 due to some CVEs in my project and the same is not possible. Any idea when https://github.com/combust/mleap/pull/860 will be merged?

jsleight commented 11 months ago

Not sure, but once it has a clean build I'm happy to merge it. I see you've commented on that PR as well, which is good.

dotbg commented 10 months ago

Blocked by https://github.com/dmlc/xgboost/issues/9765

zhenglaizhang commented 4 months ago

It looks like xgboost has supported 2.13? https://github.com/dmlc/xgboost/issues/9765#issuecomment-1805029649

Can we proceed to merge that PR?