search

amplab / SparkNet

Distributed Neural Networks for Spark
MIT License
603 stars 172 forks source link

how to run on ppc64le platform? #137

Open liaocs2008 opened 8 years ago

liaocs2008 commented 8 years ago

I tried to build this project on ppc64le platform but unfortunately got following error:

java.lang.ExceptionInInitializerError at TensorFlowNetSpec$$anonfun$1.apply$mcV$sp(TensorFlowNetSpec.scala:15) at TensorFlowNetSpec$$anonfun$1.apply(TensorFlowNetSpec.scala:14)

The line 15 of "TensorFlowNetSpec" is to create an object from class GraphDef, which looks like coming from tensorflow package.

Do you think this would be the problem of javacpp-presets, i.e., not supporting ppc64le? Any help will be appreciated. Thanks!

pcmoritz commented 8 years ago

Currently, ppc64le is not supported. If you get TensorFlow to build with JavaCPP on ppc64le, you can follow these instructions to create JARs: https://github.com/amplab/SparkNet/blob/master/doc/creating-jars.md

liaocs2008 commented 8 years ago

All right.

In fact, I already got jar packages (opencv, tensorflow and caffe) from javacpp-presets. This is due to recent updates from JavaCPP. But there is another problem as following:

[info] Converting NDArray to Tensor and back [info] - should preserve its value * FAILED * [info] java.lang.Exception: The shape of t does not match the shape of array. t has shape Array() and array has shape Array(1000000)

To pass test, that line is commented in file TensorFlowNetSpec.scala. So finally a jar file can be generated. I run test on cifar dataset but following error pops out:

I0630 05:14:20.276917 161220 net.cpp:106] Creating Layer conv1 I0630 05:14:20.276932 161220 net.cpp:454] conv1 <- data I0630 05:14:20.276950 161220 net.cpp:411] conv1 -> conv1 #

A fatal error has been detected by the Java Runtime Environment:

#

SIGSEGV (0xb) at pc=0x00003ff5aa0e8c80, pid=160557, tid=70330884288928

#

JRE version: OpenJDK Runtime Environment (7.0_95) (build 1.7.0_95-b00)

Java VM: OpenJDK 64-Bit Server VM (24.95-b01 mixed mode linux-ppc64 compressed oops)

Derivative: IcedTea 2.6.4

Distribution: Ubuntu 14.04 LTS, package 7u95-2.6.4-0ubuntu0.14.04.2

Problematic frame:

C [libcaffe.so.1.0.0-rc3+0x328c80] caffe::Caffe::RNG::generator()+0x0

#

Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again

#

Do you have any idea what is wrong here? Anyway, do you have plan to support ppc64le in future?