search

h2oai / deepwater

Deep Learning in H2O using Native GPU Backends
Apache License 2.0
282 stars 93 forks source link

build failed at :deepwater-tensorflow:compileJava -> cannot find symbol #35

Closed everdark closed 7 years ago

everdark commented 7 years ago

I've followed the readme to install all the dependencies but can't build with success. It seems that something wrong with the java code.

My platform is macOS 10.12.3 The build message:

deepwater-backend-api
deepwater-mxnet
deepwater-tensorflow
:deepwater-backend-api:compileJava UP-TO-DATE
:deepwater-backend-api:processResources UP-TO-DATE
:deepwater-backend-api:classes UP-TO-DATE
:deepwater-backend-api:jar UP-TO-DATE
:deepwater-mxnet:make
rm -rf build/deepwater
mkdir -p build/deepwater/backends/mxnet
javac -sourcepath . -d build *.java build/*.java
install_name_tool -change lib/libmxnet.so @loader_path/libmxnet.so build/libNative.so
cp build/libNative.so build/deepwater/backends/mxnet
cp ../thirdparty/mxnet/lib/libmxnet.so build/deepwater/backends/mxnet
:deepwater-mxnet:compileJava UP-TO-DATE
:deepwater-mxnet:processResources UP-TO-DATE
:deepwater-mxnet:classes UP-TO-DATE
:deepwater-mxnet:jar UP-TO-DATE
:deepwater-tensorflow:compileJava
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/ModelFactory.java:9: error: cannot find symbol
import org.tensorflow.Graph;
                     ^
  symbol:   class Graph
  location: package org.tensorflow
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/TensorflowModel.java:7: error: cannot find symbol
import org.tensorflow.Graph;
                     ^
  symbol:   class Graph
  location: package org.tensorflow
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/TensorflowModel.java:8: error: cannot find symbol
import org.tensorflow.Session;
                     ^
  symbol:   class Session
  location: package org.tensorflow
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/TensorflowModel.java:9: error: cannot find symbol
import org.tensorflow.Tensor;
                     ^
  symbol:   class Tensor
  location: package org.tensorflow
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/TensorflowModel.java:22: error: cannot find symbol
    protected Session session;
              ^
  symbol:   class Session
  location: class TensorflowModel
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/TensorflowModel.java:23: error: cannot find symbol
    private Graph graph;
            ^
  symbol:   class Graph
  location: class TensorflowModel
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/TensorflowModel.java:37: error: cannot find symbol
    TensorflowModel(TensorflowMetaModel meta, Graph graph, byte[] definition) {
                                              ^
  symbol:   class Graph
  location: class TensorflowModel
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/TensorflowModel.java:44: error: cannot find symbol
    public Graph getGraph() {
           ^
  symbol:   class Graph
  location: class TensorflowModel
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/TensorflowModel.java:48: error: cannot find symbol
    public void setGraph(Graph g) {
                         ^
  symbol:   class Graph
  location: class TensorflowModel
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/TensorflowModel.java:52: error: cannot find symbol
    public Session getSession() {
           ^
  symbol:   class Session
  location: class TensorflowModel
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/TensorflowModel.java:56: error: cannot find symbol
    public void setSession(Session session) {
                           ^
  symbol:   class Session
  location: class TensorflowModel
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/TensorflowModel.java:83: error: cannot find symbol
    public float[] getPredictions(Tensor tensor) {
                                  ^
  symbol:   class Tensor
  location: class TensorflowModel
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:12: error: cannot find symbol
import org.tensorflow.Session;
                     ^
  symbol:   class Session
  location: package org.tensorflow
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:13: error: cannot find symbol
import org.tensorflow.Tensor;
                     ^
  symbol:   class Tensor
  location: package org.tensorflow
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:34: error: cannot find symbol
    private Session session;
            ^
  symbol:   class Session
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:104: error: package Session does not exist
    private void feedIfPresent(Session.Runner runner, String value, Tensor tensor) {
                                      ^
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:104: error: cannot find symbol
    private void feedIfPresent(Session.Runner runner, String value, Tensor tensor) {
                                                                    ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:402: error: package Session does not exist
    private List<Tensor> feedMLPData(TensorflowModel model, Session.Runner runner) {
                                                                   ^
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:402: error: cannot find symbol
    private List<Tensor> feedMLPData(TensorflowModel model, Session.Runner runner) {
                 ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/ModelFactory.java:145: error: cannot find symbol
        Graph g = new Graph();
        ^
  symbol:   class Graph
  location: class ModelFactory
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/models/ModelFactory.java:145: error: cannot find symbol
        Graph g = new Graph();
                      ^
  symbol:   class Graph
  location: class ModelFactory
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:73: error: cannot find symbol
        session = new Session(model.getGraph());
                      ^
  symbol:   class Session
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:91: error: package Session does not exist
            Session.Runner runner = session.runner();
                   ^
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:92: error: cannot find symbol
            Tensor isTrainingTensor = Tensor.create(false);
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:92: error: cannot find symbol
            Tensor isTrainingTensor = Tensor.create(false);
                                      ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:124: error: package Session does not exist
        Session.Runner runner = model.getSession().runner();
               ^
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:126: error: cannot find symbol
        try(Tensor filenameTensor = Tensor.create(param_path.getBytes());
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:126: error: cannot find symbol
        try(Tensor filenameTensor = Tensor.create(param_path.getBytes());
                                    ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:127: error: cannot find symbol
            Tensor isTrainingTensor = Tensor.create(false)){
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:127: error: cannot find symbol
            Tensor isTrainingTensor = Tensor.create(false)){
                                      ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:169: error: package Session does not exist
        Session.Runner runner = model.getSession().runner();
               ^
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:171: error: cannot find symbol
        try(Tensor filenameTensor = Tensor.create(param_path.getBytes());
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:171: error: cannot find symbol
        try(Tensor filenameTensor = Tensor.create(param_path.getBytes());
                                    ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:172: error: cannot find symbol
            Tensor isTrainingTensor = Tensor.create(false)) {
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:172: error: cannot find symbol
            Tensor isTrainingTensor = Tensor.create(false)) {
                                      ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:214: error: package Session does not exist
        Session.Runner runner = model.getSession().runner();
               ^
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:221: error: cannot find symbol
        try(Tensor dataTensor = Tensor.create(dataShape, dataMatrix)) {
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:221: error: cannot find symbol
        try(Tensor dataTensor = Tensor.create(dataShape, dataMatrix)) {
                                ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:225: error: cannot find symbol
            Tensor run = runner.run().get(0);
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:364: error: package Session does not exist
        Session.Runner runner = model.getSession().runner();
               ^
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:366: error: cannot find symbol
        try(Tensor dataTensor = Tensor.create(dataShape, dataMatrix);
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:366: error: cannot find symbol
        try(Tensor dataTensor = Tensor.create(dataShape, dataMatrix);
                                ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:367: error: cannot find symbol
            Tensor labelTensor = Tensor.create(labelShape, labelData);
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:367: error: cannot find symbol
            Tensor labelTensor = Tensor.create(labelShape, labelData);
                                 ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:368: error: cannot find symbol
            Tensor batchSizeTensor = Tensor.create((float) model.miniBatchSize);
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:368: error: cannot find symbol
            Tensor batchSizeTensor = Tensor.create((float) model.miniBatchSize);
                                     ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:369: error: cannot find symbol
            Tensor learningRateTensor = Tensor.create(model.getParameter("learning_rate", 0.001f));
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:369: error: cannot find symbol
            Tensor learningRateTensor = Tensor.create(model.getParameter("learning_rate", 0.001f));
                                        ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:370: error: cannot find symbol
            Tensor momentumTensor = Tensor.create(model.getParameter("momentum", 0.9f));
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:370: error: cannot find symbol
            Tensor momentumTensor = Tensor.create(model.getParameter("momentum", 0.9f));
                                    ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:371: error: cannot find symbol
            Tensor isTrainingTensor = Tensor.create(true)) {
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:371: error: cannot find symbol
            Tensor isTrainingTensor = Tensor.create(true)) {
                                      ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:378: error: cannot find symbol
            List<Tensor> mlpTensors = Collections.emptyList();
                 ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:394: error: cannot find symbol
            for(Tensor t : mlpTensors) {
                ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:409: error: cannot find symbol
        Tensor activationTensor = Tensor.create(act);
        ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:409: error: cannot find symbol
        Tensor activationTensor = Tensor.create(act);
                                  ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:411: error: cannot find symbol
        Tensor dropoutRatioTensor = Tensor.create(model.inputDropoutRatio);
        ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:411: error: cannot find symbol
        Tensor dropoutRatioTensor = Tensor.create(model.inputDropoutRatio);
                                    ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:413: error: cannot find symbol
        Tensor hiddenDropoutRatioTensor = Tensor.create(model.hiddenDropoutRatios);
        ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:413: error: cannot find symbol
        Tensor hiddenDropoutRatioTensor = Tensor.create(model.hiddenDropoutRatios);
                                          ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:416: error: cannot find symbol
        return new ArrayList<Tensor>() {{this.add(activationTensor); this.add(dropoutRatioTensor); this.add(hiddenDropoutRatioTensor);}};
                             ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:422: error: cannot find symbol
        Session session = model.getSession();
        ^
  symbol:   class Session
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:423: error: package Session does not exist
        Session.Runner runner = session.runner();
               ^
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:427: error: cannot find symbol
        List<Tensor> mlpTensors = Collections.emptyList();
             ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:432: error: cannot find symbol
        try(Tensor miniBatchSizeTensor = Tensor.create((float) model.miniBatchSize);
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:432: error: cannot find symbol
        try(Tensor miniBatchSizeTensor = Tensor.create((float) model.miniBatchSize);
                                         ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:433: error: cannot find symbol
            Tensor isTrainingTensor = Tensor.create(false);
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:433: error: cannot find symbol
            Tensor isTrainingTensor = Tensor.create(false);
                                      ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:434: error: cannot find symbol
            Tensor dataTensor = Tensor.create(dataShape, dataMatrix)) {
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:434: error: cannot find symbol
            Tensor dataTensor = Tensor.create(dataShape, dataMatrix)) {
                                ^
  symbol:   variable Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:441: error: cannot find symbol
            List<Tensor> results = runner.run();
                 ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:443: error: cannot find symbol
            for(Tensor t : mlpTensors) {
                ^
  symbol:   class Tensor
  location: class TensorflowBackend
/Users/kylechung/Gitrepo/deepwater/tensorflow/src/main/java/deepwater/backends/tensorflow/TensorflowBackend.java:447: error: cannot find symbol
            Tensor output = results.get(0);
            ^
  symbol:   class Tensor
  location: class TensorflowBackend
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
73 errors
:deepwater-tensorflow:compileJava FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':deepwater-tensorflow:compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 9.491 secs

Any help is appreciated. :)

mdymczyk commented 7 years ago

@everdark thanks for pointing this out. The README is not up to date, I'll update it.

Before building the project (with ./gradlew build -x test) please run ./gradlew clean tensorflowCompile. This will download the appropriate TF version and build it (the so libs and the jar).

everdark commented 7 years ago

yes now it works. thanks for the quick response.

fraserpal commented 7 years ago

Having the same error as @everdark , when running sudo ./gradlew -debug clean tensorflowCompile I get the following error:

14:01:04.788 [ERROR] [system.err] ../cppbuild.sh: line 45: ../../../tensorflow-1.1.0-rc0.patch: No such file or directory
14:01:04.831 [QUIET] [system.out] Using python library path: /usr/local/lib/python2.7/dist-packages
14:01:04.970 [ERROR] [system.err] ____Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.
14:01:05.032 [ERROR] [system.err] .......
14:01:06.712 [ERROR] [system.err] ____Downloading http://bazel-mirror.storage.googleapis.com/github.com/bazelbuild/rules_closure/archive/5ca1dab6df9ad02050f7ba4e816407f88690cf7d.tar.gz: 89,157 bytes
14:01:06.906 [ERROR] [system.err] ____Downloading http://bazel-mirror.storage.googleapis.com/github.com/bazelbuild/rules_closure/archive/5ca1dab6df9ad02050f7ba4e816407f88690cf7d.tar.gz: 182,153 bytes
14:01:07.108 [ERROR] [system.err] ____Downloading http://bazel-mirror.storage.googleapis.com/github.com/bazelbuild/rules_closure/archive/5ca1dab6df9ad02050f7ba4e816407f88690cf7d.tar.gz: 361,205 bytes
14:01:07.400 [ERROR] [system.err] ____Loading package: tensorflow/stream_executor
14:01:07.490 [ERROR] [system.err] ____Loading package: tensorflow/tensorboard/components/tf_dashboard_common
14:01:07.850 [ERROR] [system.err] ERROR: package contains errors: tensorflow/compiler/tests.
14:01:07.852 [ERROR] [system.err] ERROR: error loading package 'tensorflow/compiler/tests': Encountered error while reading extension file 'cuda/build_defs.bzl': no such package '@local_config_cuda//cuda': Traceback (most recent call last):
14:01:07.852 [ERROR] [system.err]   File "/home/azken/deepwater/thirdparty/tensorflow/cppbuild/linux-x86_64/tensorflow-1.1.0-rc0/third_party/gpus/cuda_configure.bzl", line 824
14:01:07.852 [ERROR] [system.err]       _create_cuda_repository(repository_ctx)
14:01:07.852 [ERROR] [system.err]   File "/home/azken/deepwater/thirdparty/tensorflow/cppbuild/linux-x86_64/tensorflow-1.1.0-rc0/third_party/gpus/cuda_configure.bzl", line 736, in _create_cuda_repository
14:01:07.852 [ERROR] [system.err]       _get_cuda_config(repository_ctx)
14:01:07.853 [ERROR] [system.err]   File "/home/azken/deepwater/thirdparty/tensorflow/cppbuild/linux-x86_64/tensorflow-1.1.0-rc0/third_party/gpus/cuda_configure.bzl", line 589, in _get_cuda_config
14:01:07.853 [ERROR] [system.err]       _cudnn_version(repository_ctx, cudnn_install_base..., ...)
14:01:07.853 [ERROR] [system.err]   File "/home/azken/deepwater/thirdparty/tensorflow/cppbuild/linux-x86_64/tensorflow-1.1.0-rc0/third_party/gpus/cuda_configure.bzl", line 312, in _cudnn_version
14:01:07.853 [ERROR] [system.err]       auto_configure_fail("cuDNN version detected from %s ...))
14:01:07.853 [ERROR] [system.err]   File "/home/azken/deepwater/thirdparty/tensorflow/cppbuild/linux-x86_64/tensorflow-1.1.0-rc0/third_party/gpus/cuda_configure.bzl", line 93, in auto_configure_fail
14:01:07.853 [ERROR] [system.err]       fail("
14:01:07.853 [ERROR] [system.err] %sAuto-Configuration Error:%s ...))
14:01:07.853 [ERROR] [system.err] 
14:01:07.854 [ERROR] [system.err] Auto-Configuration Error: cuDNN version detected from /usr/local/cuda/include/cudnn.h (6.0.21) does not match TF_CUDNN_VERSION (5)
14:01:07.854 [ERROR] [system.err] .
14:01:07.858 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: FAILED
14:01:07.858 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Process 'command 'bash'' finished with exit value 2 (state: FAILED)
14:01:07.859 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter] Finished executing task ':deepwater-tensorflow:tensorflowCompile'
14:01:07.859 [LIFECYCLE] [class org.gradle.TaskExecutionLogger] :deepwater-tensorflow:tensorflowCompile                                                                                                  FAILED
14:01:07.859 [INFO] [org.gradle.execution.taskgraph.AbstractTaskPlanExecutor] :deepwater-tensorflow:tensorflowCompile (Thread[main,5,main]) completed. Took 3.359 secs.
14:01:07.860 [DEBUG] [org.gradle.execution.taskgraph.AbstractTaskPlanExecutor] Task worker [Thread[main,5,main]] finished, busy: 3.382 secs, idle: 0.002 secs
14:01:07.867 [ERROR] [org.gradle.BuildExceptionReporter] 
14:01:07.868 [ERROR] [org.gradle.BuildExceptionReporter] FAILURE: Build failed with an exception.
14:01:07.868 [ERROR] [org.gradle.BuildExceptionReporter] 
14:01:07.868 [ERROR] [org.gradle.BuildExceptionReporter] * What went wrong:
14:01:07.868 [ERROR] [org.gradle.BuildExceptionReporter] Execution failed for task ':deepwater-tensorflow:tensorflowCompile'.
14:01:07.868 [ERROR] [org.gradle.BuildExceptionReporter] > Process 'command 'bash'' finished with non-zero exit value 2
14:01:07.869 [ERROR] [org.gradle.BuildExceptionReporter] 
14:01:07.869 [ERROR] [org.gradle.BuildExceptionReporter] * Try:
14:01:07.869 [ERROR] [org.gradle.BuildExceptionReporter] Run with --stacktrace option to get the stack trace. 
14:01:07.869 [LIFECYCLE] [org.gradle.BuildResultLogger] 
14:01:07.870 [LIFECYCLE] [org.gradle.BuildResultLogger] BUILD FAILED
14:01:07.870 [LIFECYCLE] [org.gradle.BuildResultLogger] 
14:01:07.870 [LIFECYCLE] [org.gradle.BuildResultLogger] Total time: 6.646 secs
14:01:07.872 [DEBUG] [org.gradle.cache.internal.DefaultCacheAccess] Cache task history cache (/home/azken/deepwater/.gradle/2.9/taskArtifacts) was closed 0 times.
14:01:07.874 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on cp_settings class cache for settings file '/home/azken/deepwater/settings.gradle' (/root/.gradle/caches/2.9/scripts/settings_2lez9x7bmh9b04ba7bo67cp8g/cp_settings).
14:01:07.874 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on settings class cache for settings file '/home/azken/deepwater/settings.gradle' (/root/.gradle/caches/2.9/scripts/settings_2lez9x7bmh9b04ba7bo67cp8g/settings).
14:01:07.875 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on cp_proj class cache for build file '/home/azken/deepwater/build.gradle' (/root/.gradle/caches/2.9/scripts/build_dah7py751ek7yl1exd679z80q/cp_proj).
14:01:07.875 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on proj class cache for build file '/home/azken/deepwater/build.gradle' (/root/.gradle/caches/2.9/scripts/build_dah7py751ek7yl1exd679z80q/proj).
14:01:07.875 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on cp_dsl class cache for script '/home/azken/deepwater/gradle/s3sync.gradle' (/root/.gradle/caches/2.9/scripts/s3sync_bd8wktjdyj6kaysp1k4aqg8xl/cp_dsl).
14:01:07.875 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on dsl class cache for script '/home/azken/deepwater/gradle/s3sync.gradle' (/root/.gradle/caches/2.9/scripts/s3sync_bd8wktjdyj6kaysp1k4aqg8xl/dsl).
14:01:07.876 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on cp_dsl class cache for script '/home/azken/deepwater/gradle/artifacts.gradle' (/root/.gradle/caches/2.9/scripts/artifacts_f4akapz4ulcaaw4lpim15bqx6/cp_dsl).
14:01:07.876 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on dsl class cache for script '/home/azken/deepwater/gradle/artifacts.gradle' (/root/.gradle/caches/2.9/scripts/artifacts_f4akapz4ulcaaw4lpim15bqx6/dsl).
14:01:07.876 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on cp_dsl class cache for script '/home/azken/deepwater/gradle/publish.gradle' (/root/.gradle/caches/2.9/scripts/publish_7co1r1w7c8aluspob3g6c9iql/cp_dsl).
14:01:07.876 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on dsl class cache for script '/home/azken/deepwater/gradle/publish.gradle' (/root/.gradle/caches/2.9/scripts/publish_7co1r1w7c8aluspob3g6c9iql/dsl).
14:01:07.876 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on cp_proj class cache for build file '/home/azken/deepwater/backend-api/build.gradle' (/root/.gradle/caches/2.9/scripts/build_ee0rd0x583me3m4cpi2r25g4d/cp_proj).
14:01:07.876 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on proj class cache for build file '/home/azken/deepwater/backend-api/build.gradle' (/root/.gradle/caches/2.9/scripts/build_ee0rd0x583me3m4cpi2r25g4d/proj).
14:01:07.876 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on cp_proj class cache for build file '/home/azken/deepwater/mxnet/build.gradle' (/root/.gradle/caches/2.9/scripts/build_6knbyg7srr0hu4j9o4eg3m260/cp_proj).
14:01:07.877 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on proj class cache for build file '/home/azken/deepwater/mxnet/build.gradle' (/root/.gradle/caches/2.9/scripts/build_6knbyg7srr0hu4j9o4eg3m260/proj).
14:01:07.877 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on cp_proj class cache for build file '/home/azken/deepwater/tensorflow/build.gradle' (/root/.gradle/caches/2.9/scripts/build_800niadndkhxa7uztodegvpio/cp_proj).
14:01:07.877 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on proj class cache for build file '/home/azken/deepwater/tensorflow/build.gradle' (/root/.gradle/caches/2.9/scripts/build_800niadndkhxa7uztodegvpio/proj).
14:01:07.877 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.store.CachedStoreFactory                                                                                                ]                                                                                                    Resolved configuration cache closed. Cache reads: 2, disk reads: 2 (avg: 0.001 secs, total: 0.003 secs)
14:01:07.878 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.store.CachedStoreFactory                                                                                                ]                                                                                                    Resolution result cache closed. Cache reads: 0, disk reads: 0 (avg: 0.0 secs, total: 0.0 secs)
14:01:07.878 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.store.ResolutionResultsStoreFactory                                                                                     ]                                                                                                    Deleted 2 resolution results binary files in 0.001 secs
14:01:07.878 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.memcache.InMemoryCachedRepositoryFactory                                                                                   ]                                                                                                    In-memory dependency metadata cache closed. Repos cached: 10, cache instances: 4, modules served from cache: 1, artifacts: 1
14:01:07.879 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache artifact-at-repository.bin (/root/.gradle/caches/modules-2/metadata-2.16/artifact-at-repository.bin)
14:01:07.879 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache module-metadata.bin (/root/.gradle/caches/modules-2/metadata-2.16/module-metadata.bin)
14:01:07.880 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache module-versions.bin (/root/.gradle/caches/modules-2/metadata-2.16/module-versions.bin)
14:01:07.880 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on artifact cache (/root/.gradle/caches/modules-2).
14:01:07.880 [DEBUG] [org.gradle.cache.internal.DefaultCacheAccess] Cache Plugin Resolution Cache (/root/.gradle/caches/2.9/plugin-resolution) was closed 0 times.
14:01:07.880 [DEBUG] [org.gradle.api.internal.tasks.compile.daemon.CompilerDaemonManager] Stopping 0 compiler daemon(s).
14:01:07.881 [INFO] [org.gradle.api.internal.tasks.compile.daemon.CompilerDaemonManager] Stopped 0 compiler daemon(s).

I have checked CUDA, cudnn and Tensorflow versions for compatibility problems but everything looks fine from my point of view. Any help will be appreciated!

mdymczyk commented 7 years ago

@fraserpal 14:01:07.854 [ERROR] [system.err] Auto-Configuration Error: cuDNN version detected from /usr/local/cuda/include/cudnn.h (6.0.21) does not match TF_CUDNN_VERSION (5)

You have cuDNN version 6.0.21 installed, which is not supported by TensorFlow. Please downgrade to version 5.x

fraserpal commented 7 years ago

Thanks @mdymczyk! Sorry for such a naive question but I am a beginner with all this installation process. Have a nice day!

fraserpal commented 7 years ago

@mdymczyk One more question. If they are not compatible, how it is possible that I run a Tensorflow Session and the GPUs are recognized by the system??

mdymczyk commented 7 years ago

@fraserpal depends which version of TF are you using, the newest release candidate might already have some sort of support for it or you might not be using any methods that use cuDNN.

Officially it's not yet fully supported as per this issue https://github.com/tensorflow/tensorflow/issues/8828

When they release a new, stable, version with cuDNN 6 support we might upgrade our dependency.

fraserpal commented 7 years ago

Thanks! now there is no error anymore! Anyway after following deepwater README, when I initialize an h2o cluster, it still only uses CPU and not GPUs. maybe the problem arises in this part:

"You need to check out the h2o-3. Copy the freshly created jar file build/libs/deepwater-all.jar from the previous step to H2O-3's library h2o-3/lib/deepwater-all.jar (create the directory if it's not there) and you're done!"

I am not sure if I have to copy the jar file somewhere in the already defined h2o directory that I made before, or make a new one called h2o-3 anywhere I want.

Thanks in advance for your help!

mdymczyk commented 7 years ago

@fraserpal not sure which part is not clear - after you successfully build deepwater there should be a jar in build/libs/deepwater-all.jar (in the deepwater project dir). Please copy it to your lib folder inside the h2o-3 project (if it does not exist then please create it). Then you build h2o-3 (./gradlew clean build -x test). That will build an h2o.jar which you can use. It should also create Python and R packages if you wish to use those.

Also be advised that GPU will only be used when building DeepWater models, H2O models (like random forest, GBM etc) are still CPU only. We are implementing them currently for GPUs.

fraserpal commented 7 years ago

Hello again @mdymczyk. Other error arises, that I am not able to solve. When I put build/libs/deepwater-all.jar in h2o-3 and build it, I get the following error: `:h2o-py:setProjectVersion INIT.path =/home/administrador/h2o-3/h2o-py/h2o/init.py :h2o-py:verifyDependencies

ERRORS:

Python module `pip` is missing: install it with `pip install pip`
Python module `setuptools` is missing: install it with `pip install setuptools`

:h2o-py:verifyDependencies FAILED

FAILURE: Build failed with an exception.

BUILD FAILED

Total time: 1 mins 17.77 secs

Task timings: 46,229 secs :h2o-assemblies:main:shadowJar 7,918 secs :h2o-web:jar 5,930 secs :h2o-core:compileJava 5,199 secs :h2o-bindings:runGenerateRESTAPIBindingsSrc 2,719 secs :h2o-assemblies:main:copyJar 1,168 secs :h2o-web:installNpmPackages 1,115 secs :h2o-algos:compileJava 6,489 secs All others `

mdymczyk commented 7 years ago

@fraserpal did you follow the build instructions https://github.com/h2oai/h2o-3#Building ?