search

yahoo / CaffeOnSpark

Distributed deep learning on Hadoop and Spark clusters.
Apache License 2.0
1.27k stars 358 forks source link

Support for Protocol buffers > 2.5 #231

Open dimon777 opened 7 years ago

dimon777 commented 7 years ago

Hi,

Are there plans to upgrade this repository to work on more recent protocol buffers? If not does anyone aware of any pulls of this repo which compiles against PB 2.6.1 ?

Thanks.

junshi15 commented 7 years ago

There is no plan to upgrade protobuf at this moment. Is there any reason you prefer 2.6?

dimon777 commented 7 years ago

Sorry for the long reply. Yes, there is a reason: protobuf 2.5 is a relic now. On my system: Arch 2015-01, which is already 2.5 years old, the default PB is 2.6.1. I don't know for other distros but they should be in similar situation. Hadoop 2.7 compiles just fine with PB2.6 and there is lot of movement to obsolete PB2.5. The dependency on it for Caffe or CaffeOnSpark makes it difficult to compile this product and requires deinstalling more recent PB version and compiling PB2.5 from source. Are these good reasons? If not, can someone explain to me how to upgrade this project to PB>2.5. I'd be happy to contribute.

Thanks.

GodLikeKrider commented 7 years ago

Hi, I am working to deploy distributed learning on Amazon EC2. The deep learning ubuntu16.04 AMI uses protobuf 3.0.0-beta-2. I also hope that someone can upgrade this project to support PB 3.0, since I cannot find other frame to support multi-gpu cloud learning on Caffe.

Many thanks in advance.