search

adatao / tensorspark

TensorFlow on Spark
297 stars 98 forks source link

TensorSpark productionalized in yarn-cluster mode #8

Closed MasoodK closed 7 years ago

MasoodK commented 7 years ago

Hey Arimo contributors, Thanks for open-sourcing your TensorSpark! I have done few modifications to a fork of your repo that are mostly relevant to someone interested in taking TensorSpark to production in yarn-cluster mode with CPU machines. You can go through my commit comments to see if there is anything you’d like to bring to your repo; feel free to let me know and I’ll send a PR for the branch and you can then “git cherry-pick” the commits you’re interested in. I’ve been working with the MNIST dataset; if I find the time, I’ll try to create the ImageNet/AlexNet scenario which I see to be on your roadmap as well. Thanks.

illuzen commented 7 years ago

Hey @MasoodK ! Thanks for offering this back, and sorry for the late reply!

I'll look thru your commits this weekend and we can pull stuff in. I'm super stoked and grateful that you have done this work.

adatao-ddf commented 7 years ago

Good stuff!

On Fri, Nov 4, 2016, 8:14 AM Snake Charmer notifications@github.com wrote:

Hey @MasoodK https://github.com/MasoodK ! Thanks for offering this back, and sorry for the late reply!

I'll look thru your commits this weekend and we can pull stuff in. I'm super stoked and grateful that you have done this work.

—

You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/adatao/tensorspark/issues/8#issuecomment-258458606, or mute the thread https://github.com/notifications/unsubscribe-auth/AHu6XGzeKGh0p2LTdQiaCnj62iBJbOIxks5q60vdgaJpZM4KhVFe .

Algorithms of the Mind http://bit.ly/1ReQvEW http://bit.ly/1ReQvEW

Christopher Nguyen

CEO & Co-Founder

Arimo, Inc. http://www.arimo.com/(née Adatao)

2016 Fast Company World's 10 Most Innovative Companies in Data Science http://bit.ly/1QdFyH2

MasoodK commented 7 years ago

Thanks @snakecharmer1024 and @adatao-ddf for the comments!

After you have gone through my commits, let me know if there is anything aligned with the future direction of your project. If there is something, I'll try to work with you to bring stuff into your repo, if the current format is not ideal.

Have a good weekend!

illuzen commented 7 years ago

@MasoodK Sorry for another delay. I think this looks great. Make a pull request with everything?

aht commented 7 years ago

closing as PR is merged