search

microsoft / SynapseML

Simple and Distributed Machine Learning
http://aka.ms/spark
MIT License
5.07k stars 831 forks source link

[FEATURE REQUEST]: Implement complete Spark bindings using Autogen Technology #1678

Open rrekapalli opened 2 years ago

rrekapalli commented 2 years ago

Dear Team:

Is there a possibility to implement complete .Net bindings (including core and MLLib etc) for Spark using Autogen technology? .Net for Spark .Net for Spark seems to only have core bindings and doesn't seem to implement MLLib from long time! Just wondering if it would be easy with Autogen technology to convert/port the whole Spark library to .Net.

Thanks & Regards.

github-actions[bot] commented 2 years ago

Hey @rrekapalli :wave:! Thank you so much for reporting the issue/feature request :rotating_light:. Someone from SynapseML Team will be looking to triage this issue soon. We appreciate your patience.

mhamilton723 commented 2 years ago

@serena-ruan is working on this item and will follow up when she has updates to share

rrekapalli commented 2 years ago

This is great news! Can't wait for the implementation!

serena-ruan commented 2 years ago

@rrekapalli Sorry for the late reply! It's definitely on the roadmap, and there is an initial PR https://github.com/dotnet/spark/pull/1096 that uses our existing codegen system to auto generate the code. I have talked to the owner that they're super interested. But we might need to wait as the dotnet spark team is quite busy recently, and we will work on polishing the existing .NET codegen bindings to make it usable outside of SynapseML :D

rrekapalli commented 2 years ago

Thank you for the update @serena-ruan !

Vislesha commented 1 year ago

Hi team, any update/guidance on this feature implementation!

rrekapalli commented 1 year ago

Dear @serena-ruan,
As .Net for Spark is not being active, is there any chance that a full Spark implementation be part of this library anytime soon? I think this is a great library that brings all pieces together and Spark should be part of this library as well. Greatly appreciate if you can share the future direction and/or any plans for full Spark implementation!

serena-ruan commented 1 year ago

Hi @rrekapalli Sorry for the inconvenience, .NET team has been real busy and hasn't actively responded to our feature requests. We'll reach out to them again offline and see what's in their roadmap. dotnet/spark repo is the official repository for support Spark with .NET, so there's no way for us to bringing the spark implementation directly into us. But you could use all the existing features with C# in SynapseML library now. @mhamilton723 if you have more comments :)

Vislesha commented 1 year ago

Hi @serena-ruan, @mhamilton723, Could you please let me know if there is any chance that we see a complete Spark binding whether in .Net for Spark or SynapseML? There hasn't been much traction on the .Net for Spark library from longtime and have been hearing there won't be support to C# (.Net) for SynapseML. Could you please provide some guidance on the support of .Net for these libraries.