Error when building tensorflow from source

shkr commented 8 years ago

I am trying to build and install tensorflow from source. However when building with bazel it returns an error stating Extension file 'tensorflow/tensorflow.bzl' has errors.

Here are the commands that I ran and the logs are included.


mkvirtualenv tensorflow_dev
brew install bazel swig
workon tensorflow_dev
pip install six numpy wheel ipython
export TF_DIR=/Users/shashank/Documents/repositories/tensorflow
cd /Users/shashank/Documents/repositories/
git clone git@github.com:tensorflow/tensorflow.git
cd $TF_DIR && ./configure

~/Documents/repositories/tensorflow ~/Documents/repositories/tensorflow
Please specify the location of python. [Default is /Users/shashank/.virtualenvs/tensorflow_dev/bin/python]:
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
No Google Cloud Platform support will be enabled for TensorFlow
Found possible Python library paths:
  /Users/shashank/Documents/py_config/
  /Users/shashank/.virtualenvs/tensorflow_dev/lib/python2.7/site-packages
Please input the desired Python library path to use.  Default is [/Users/shashank/Documents/py_config/]

/Users/shashank/Documents/py_config/
Do you wish to build TensorFlow with GPU support? [y/N]
No GPU support will be enabled for TensorFlow
Configuration finished

cd $TF_DIR && bazel build -c opt //tensorflow/tools/pip_package:build_pip_package
ERROR: /Users/shashank/Documents/repositories/tensorflow/tensorflow/tensorflow.bzl:568:26: Traceback (most recent call last):
        File "/Users/shashank/Documents/repositories/tensorflow/tensorflow/tensorflow.bzl", line 562
            rule(attrs = {"srcs": attr.label_list..."), <3 more arguments>)}, <2 more arguments>)
        File "/Users/shashank/Documents/repositories/tensorflow/tensorflow/tensorflow.bzl", line 568, in rule
            attr.label_list(cfg = "data", allow_files = True)
expected ConfigurationTransition or NoneType for 'cfg' while calling label_list but got string instead: data.
ERROR: com.google.devtools.build.lib.packages.BuildFileContainsErrorsException: error loading package '': Extension file 'tensorflow/tensorflow.bzl' has errors.
INFO: Elapsed time: 0.063s

Thanks for your help

CalaveraLoco commented 8 years ago

Try to update your bazel installation. I had this problem a few minutes ago, and this seems to have solved it on my side.

shkr commented 8 years ago

I reinstalled, as you have prescribed. Same error. I think there is a configuration step that I am missing with the tensorflow.bzl build file

moyi-d commented 8 years ago

I have the same issue

edgelord commented 8 years ago

I had the same issue with master master, but I could build after switching to the r0.10 branch.

brunodoamaral commented 8 years ago

I'm having the same error. It points out to an invalid configuration. A recent commit has changed the lines pointed by the error: https://github.com/tensorflow/tensorflow/commit/7bcdcbbf60fc08346fd8016270a0563f4b51362b

brunodoamaral commented 8 years ago

Update: upgrading bazel, as pointed out by @CalaveraLoco , seems to fix the problem. The issue I had is that I have two bazel installations and only upgraded the 'second' one.

shkr commented 8 years ago

Which version of bazel do I need to upgrade to ?

I installed bazel with homebrew.

Here are my build details

Build label: 0.2.1-homebrew
Build target: bazel-out/local_darwin-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Fri Apr 1 00:35:17 2016 (1459470917)
Build timestamp: 1459470917
Build timestamp as int: 1459470917

brunodoamaral commented 8 years ago

I used 0.3.0

shkr commented 8 years ago

I updated my bazel version to 0.3.1

Build label: 0.3.1-homebrew
Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Aug 4 09:58:27 2016 (1470304707)
Build timestamp: 1470304707
Build timestamp as int: 1470304707

Now when I execute this command in the root dir of tensorflow. I receive a different error.

export TF_DIR=/Users/shashank/Documents/repositories/tensorflow
cd /Users/shashank/Documents/repositories/
git clone git@github.com:tensorflow/tensorflow.git
cd $TF_DIR && ./configure

~/Documents/repositories/tensorflow ~/Documents/repositories/tensorflow
Please specify the location of python. [Default is /Users/shashank/.virtualenvs/tensorflow_dev/bin/python]:
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
No Google Cloud Platform support will be enabled for TensorFlow
Found possible Python library paths:
  /Users/shashank/Documents/py_config/
  /Users/shashank/.virtualenvs/tensorflow_dev/lib/python2.7/site-packages
Please input the desired Python library path to use.  Default is [/Users/shashank/Documents/py_config/]

/Users/shashank/Documents/py_config/
Do you wish to build TensorFlow with GPU support? [y/N]
No GPU support will be enabled for TensorFlow
Configuration finished

cd $TF_DIR && bazel build -c opt //tensorflow/tools/pip_package:build_pip_package
..
ERROR: /Users/shashank/Documents/repositories/tensorflow/tensorflow/BUILD:69:1: //tensorflow:all_files: invalid label '@' in element 0 of attribute 'srcs' in 'filegroup' rule: invalid fully-qualified label: @.
ERROR: /Users/shashank/Documents/repositories/tensorflow/tensorflow/BUILD:84:12: Target '//tensorflow:internal' contains an error and its package is in error (this is usually caused by a missing package group in the package-level visibility declaration).
ERROR: /Users/shashank/Documents/repositories/tensorflow/tensorflow/BUILD:82:1: Target '//tensorflow:__init__.py' contains an error and its package is in error and referenced by '//tensorflow:tensorflow_py'.
ERROR: /Users/shashank/Documents/repositories/tensorflow/tensorflow/tools/pip_package/BUILD:16:1: Target '//tensorflow:tensorflow_py' contains an error and its package is in error and referenced by '//tensorflow/tools/pip_package:simple_console'.
ERROR: /Users/shashank/Documents/repositories/tensorflow/tensorflow/tools/pip_package/BUILD:23:1: Target '//tensorflow:tensorflow_py' contains an error and its package is in error and referenced by '//tensorflow/tools/pip_package:build_pip_package'.
ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package' failed; build aborted.
INFO: Elapsed time: 2.489s

szsongyj commented 8 years ago

Hi , My environment is Ubuntu16.04 +gtx1080+CUDA8.0 +CUDNNV5.1+Python2.7+Bazel 0.3. 0 When I execute ./configure command in tensorflow-master directory, I experienced the same error with @shkr

How should this problem be fixed?

Thanks

gustavla commented 8 years ago

@szsongyj Easiest fix is to as people have said to upgrade bazel to 0.3.1. Another quick fix is to revert the offending commit:

git show -R 7bcdcbbf | git apply

szsongyj commented 8 years ago

@gustavla Thanks for your kindly help. I will try this solution and feedback here this evening.

Regards,

shkr commented 8 years ago

I cloned the git repo again, then also installed brew install coreutils (i do not have a GPU I did it anyways). After that the source build is working with bazel 0.3.1. Thanks @gustavla

I will investigate why in my other build I have an error with bazel 0.3.1.

szsongyj commented 8 years ago

@gustavla @shkr
I updated the bazel and confirmed the current version is 0.3.1(using command: "which bazel" and "bazel version"respectively )

and I run the configure process as @gustavla suggested:

git clone https://github.com/tensorflow/tensorflow cd tensorflow git show -R 7bcdcbbf | git apply ./configure But the following errors appeared and configured process failed

/home/syj/tensorflow/tensorflow/contrib/session_bundle/BUILD:107:1no such target '//tensorflow/core:android_lib_lite':target 'android_lib_lite'not declared in package.....

/home/syj/tensorflow/tensorflow/core/BUILD:899:1no such package '@zlib_archive//':Error downloading from http://zlib.net/zlib-1.2.8.tar.gz to ......

configuration of query "deps((//...union @bazel_tools//tools/jdk:toolchain))"failed:errors were encountered while computing transitive closure.

I am appreciated if anyone can tell me how to solve this problem.

Thank you

andrewharp commented 8 years ago

@szsongyj Is this an actual copy-paste?

//tensorflow/core:android_lib_lite does not exist, as the actual name is //tensorflow/core:android_tensorflow_lib_lite.

Even then, it would not be referenced from tensorflow/contrib/session_bundle/BUILD. Can you check to make sure the file itself has not been altered somehow?

gustavla commented 8 years ago

@szsongyj The problem you are seeing is an unrelated issue that has been resolved (see #4312). You should make sure you have the latest master branch. You can make sure your checked out copy has incorporated this commit by running git branch --contains f66b491 and make sure you see a line that starts with * (e.g. * master).

@andewharp The error I got in #4312 also said android_lib_lite. However, the whole premise of that issue was that those packages only exist in Google's internal branches, so I have no idea if that's a typo or not.

andrewharp commented 8 years ago

@gustavla Right, I probably should have remembered merging that commit :P

szsongyj commented 8 years ago

@andrewharp 1)The file itself has not been altered. Because i just issue the following commands: git clone https://github.com/tensorflow/tensorflow cd tensorflow ./configure

2)Is this an actual copy-paste? This is not the actual copy-paste ,because I run the configure last night at home with my home computer,Then I got to bed before the config process finished ,and I found the error results when I got up early in this morning , then I took some screen photos before I left home for work. (now i am in office)

So the error information is exactly like what I have posted althrough this is only part of error information @gustavla Thank you. Could you please tell me the exact commands assuming my current location is "/home/syj", then what commands one by one should I type?

Just like i have said what i have done is just typing the following commands,but failed .

git clone https://github.com/tensorflow/tensorflow cd tensorflow ./configure

by the way. Concerning my environment , CUDA8,CuDNN5.1 ,Caffe are all normal.

andrewharp commented 8 years ago

@shkr The internal fix https://github.com/tensorflow/tensorflow/commit/f66b491a06627510c1cf751fc11db2caf5aa1f25 was pushed today in https://github.com/tensorflow/tensorflow/pull/4360, so if you sync now I assume the issue will go away. I had just forgotten it was part of the push when I asked you about the copy-paste.

szsongyj commented 8 years ago

@gustavla

Please refer the following ,does that indicate my cloned tensorflow is the latest master branch or not? What is the reason of the error? How can i fix it ? Thank you&Regards

syj@syj-dl:~/tensorflow$ git branch

master syj@syj-dl:~/tensorflow$ git branch --contains f66b491 error: malformed object name f66b491

szsongyj commented 8 years ago

syj@syj-dl:~/tensorflow$ git branch

master (this is a star before master)

syj@syj-dl:~/tensorflow$ git branch --contains f66b491 error: malformed object name f66b491

szsongyj commented 8 years ago

syj@syj-dl:~$ git clone https://github.com/tensorflow/tensorflow 正克隆到 'tensorflow'... remote: Counting objects: 86413, done. remote: Compressing objects: 100% (6/6), done. remote: Total 86413 (delta 0), reused 0 (delta 0), pack-reused 86407 接收对象中: 100% (86413/86413), 56.90 MiB | 1.30 MiB/s, 完成. 处理 delta 中: 100% (63995/63995), 完成. 检查连接... 完成。 syj@syj-dl:~$ cd tensorflow syj@syj-dl:~/tensorflow$ git branch

master syj@syj-dl:~/tensorflow$ git branch -av
master 02a8d07 added threadpool.h to framework_headers in BUILD (#4108) remotes/origin/0.6.0 4be56f3 Merge pull request #1085 from tensorflow/vrv-fix-stale-doc remotes/origin/HEAD -> origin/master remotes/origin/go 5b11eb9 remove non idiomatic NewGraphFromReader(reader, bool) and add NewGraphFromBuffer and New Graph remotes/origin/issue_template_update 9d32095 Add instructions how to get commit hash remotes/origin/master 02a8d07 added threadpool.h to framework_headers in BUILD (#4108) remotes/origin/mrry-patch-1 56fc276 Delete unused boringssl.BUILD remotes/origin/mrry-patch-2 61542a5 Add -O2 to the adding an op HOWTO remotes/origin/r0.10 c715c31 Merge pull request #4284 from caisq/r0.10-fixes remotes/origin/r0.7 657f1d9 Python3 word2vec compatibility issue #1760 fixed remotes/origin/r0.8 9b69ec3 Fix broken link to Anaconda installation (#2679) remotes/origin/r0.9 9454b90 Merge pull request #3734 from RenatoUtsch/r0.9 remotes/origin/revert-3010-issue-2977 69e9316 Revert "Added complex type support to ops for GPU" remotes/origin/revert-3614-satok0 70846ec Revert "Fix prototype mismatch of ByteCount in env.cc" remotes/origin/tflearn-tutorial-updates 6e42687 New tutorial on tf.contrib.learn monitors. Change: 128589916 remotes/origin/tutorial-url-fix 51b4ea4 As of 2016-08-01, feature_columns argument is now required by DNNClassifier; updating tutorials/code accordingly. Change: 130785833 remotes/origin/vincentvanhoucke-patch-1 1862eb9 Remove confusing section remotes/origin/vincentvanhoucke-patch-2 1cf5af0 Make the default command use a named container remotes/origin/vrv-patch-1 35d2eea Update tf_op_files.txt to add tile_6.cc syj@syj-dl:~/tensorflow$ git branch --contains f66b491
master syj@syj-dl:~/tensorflow$ ./configure ~/tensorflow ~/tensorflow Please specify the location of python. [Default is /home/syj/anaconda2/bin/python]: Do you wish to build TensorFlow with Google Cloud Platform support? [y/N] y Google Cloud Platform support will be enabled for TensorFlow Found possible Python library paths: /home/syj/anaconda2/lib/python2.7/site-packages Please input the desired Python library path to use. Default is [/home/syj/anaconda2/lib/python2.7/site-packages]

/home/syj/anaconda2/lib/python2.7/site-packages Do you wish to build TensorFlow with GPU support? [y/N] y GPU support will be enabled for TensorFlow Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to use system default]: 8.0 Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: Please specify the Cudnn version you want to use. [Leave empty to use system default]: 5 Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: Please specify a list of comma-separated Cuda compute capabilities you want to build with. You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus. Please note that each additional compute capability significantly increases your build time and binary size.

. INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes. . ERROR: /home/syj/tensorflow/tensorflow/core/BUILD:692:1: no such package 'base': BUILD file not found on package path and referenced by '//tensorflow/core:ios_tensorflow_test_lib

szsongyj commented 8 years ago

. INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes. . ERROR: /home/syj/tensorflow/tensorflow/core/BUILD:692:1: no such package 'base': BUILD file not found on package path and referenced by '//tensorflow/core:ios_tensorflow_test_lib'. ERROR: /home/syj/tensorflow/tensorflow/core/kernels/BUILD:2207:1: no such package 'base': BUILD file not found on package path and referenced by '//tensorflow/core/kernels:android_tensorflow_kernels_no_rtti_lite_runtime'. ERROR: /home/syj/tensorflow/tensorflow/core/kernels/BUILD:2207:1: no such target '//tensorflow/core:android_tensorflow_lib_lite_no_rtti_lite_runtime': target 'android_tensorflow_lib_lite_no_rtti_lite_runtime' not declared in package 'tensorflow/core' defined by /home/syj/tensorflow/tensorflow/core/BUILD and referenced by '//tensorflow/core/kernels:android_tensorflow_kernels_no_rtti_lite_runtime'. ERROR: /home/syj/tensorflow/tensorflow/core/kernels/BUILD:2207:1: no such target '//tensorflow/core:android_proto_lib_no_rtti_lite_runtime': target 'android_proto_lib_no_rtti_lite_runtime' not declared in package 'tensorflow/core' defined by /home/syj/tensorflow/tensorflow/core/BUILD and referenced by '//tensorflow/core/kernels:android_tensorflow_kernels_no_rtti_lite_runtime'. ERROR: Evaluation of query "deps((//... union @bazel_tools//tools/jdk:toolchain))" failed: errors were encountered while computing transitive closure. Configuration finished

szsongyj commented 8 years ago

The above is the whole steps and output? what's wrong?

andrewharp commented 8 years ago

@szsongyj If you're getting "malformed object name" it means your repo doesn't contain that commit. However your most recent clone seems to contain it.

Can you try patching in the https://github.com/okuchaiev/tensorflow-float16/commit/c1b6623154e14799419f91bd197b7900a78dbc2b commit referenced by https://github.com/tensorflow/tensorflow/issues/4312? I think that's what you're running into now.

BTW, you can make your console output more readable by using the "insert code" button in the editor when pasting.

szsongyj commented 8 years ago

Thank you @andrewharp

in https://github.com/tensorflow/tensorflow/issues/4312, @ibab said

You can fix this by commenting out the if_android and if_ios calls in contrib/session_bundle/BUILD and the android_tensorflow_lib_lite_no_rtti_lite_runtime target in the core BUILD file.These seem to be based on Google internal targets.

Does that means I should edit BUILD file under /home/syj/tensorflow , or some other file? Thank you syj@syj-dl:~/tensorflow$ vi BUILD

szsongyj commented 8 years ago

syj@syj-dl:~/tensorflow$ ls ACKNOWLEDGMENTS bzip2.BUILD grpc.BUILD png.BUILD util ADOPTERS.md configure ISSUE_TEMPLATE.md README.md WORKSPACE AUTHORS CONTRIBUTING.md jpeg.BUILD RELEASE.md zlib.BUILD avro.BUILD eigen.BUILD jsoncpp.BUILD six.BUILD boost.BUILD farmhash.BUILD LICENSE tensorflow bower.BUILD gif.BUILD nanopb.BUILD third_party BUILD gmock.BUILD navbar.md tools It seems that in my side , thers is no

tensorflow/core/kernels/BUILD

andrewharp commented 8 years ago

@szsongyj You are listing only the contents of the local directory. Use ls -R to see the entire tree recursively, or ls tensorflow/core/kernels to see the contents of the relevant directory.

To edit the actual file from your current location try vi tensorflow/core/kernels/BUILD.

szsongyj commented 8 years ago

@andrewharp sorry I got it . it is a foolish question

syj@syj-dl:~/tensorflow/tensorflow/core/kernels$ ls BUILD

BUILD

andrewharp commented 8 years ago

@szsongyj If you select the content after pasting and then press "Insert code", github will pick the right escape chars (it needs three quotes for multi-line chunks).

Regarding the question, you may need to apply the entire patch if you continue to run into problems after commenting out only that one lib. I would just try it and see what happens.

szsongyj commented 8 years ago

@andrewharp Thank you for your guidance. NowI have commented out “android_tensorflow_kernels_no_rtti_lite_runtime” section，and the configure is running . I will paste the errors here if problem still exists. Thanks again.

szsongyj commented 8 years ago

ERROR: /home/syj/tensorflow/tensorflow/core/BUILD:692:1: no such package 'base': BUILD file not found on package path and referenced by '//tensorflow/core:ios_tensorflow_test_lib'.

andrewharp commented 8 years ago

@szsongyj ios_tensorflow_test_lib is also commented out in the patch. I would just go ahead apply the entire thing.

szsongyj commented 8 years ago

@ andrewharp OK I have commented " ios_tensorflow_test_lib" section

szsongyj commented 8 years ago

ERROR: /home/syj/tensorflow/tensorflow/workspace.bzl:85:3: no such package '@six_archive//': Error downloading from https://pypi.python.org/packages/source/s/six/six-1.10.0.tar.gz#md5=34eed507548117b2ab523ab14b2f8b55 to /home/syj/.cache/bazel/_bazel_syj/47b860f752d3ee48e8d208f528a56395/external/six_archive: Error downloading https://pypi.python.org/packages/source/s/six/six-1.10.0.tar.gz#md5=34eed507548117b2ab523ab14b2f8b55 to /home/syj/.cache/bazel/_bazel_syj/47b860f752d3ee48e8d208f528a56395/external/six_archive/six-1.10.0.tar.gz: Connection timed out and referenced by '//external:six'. ERROR: Evaluation of query "deps((//... union @bazel_tools//tools/jdk:toolchain))" failed: errors were encountered while computing transitive closure.

andrewharp commented 8 years ago

Can you try git pull --recurse-submodules?

szsongyj commented 8 years ago

At this moment, I have totally made modification as guidance of https://github.com/okuchaiev/tensorflow-float16/commit/c1b6623154e14799419f91bd197b7900a78dbc2b

so I am reconfiguring right now. Let's wait to see the result.

szsongyj commented 8 years ago

@andrewharp you mean i execute the command of

git pull --recurse-submodules

under /home/syj/tensorflow ?

andrewharp commented 8 years ago

Yes. I usually just add the --recurse-submodules flag when I clone to make things easier.

szsongyj commented 8 years ago

@ andrewharp
syj@syj-dl:~/tensorflow$ git pull --recurse-submodules remote: Counting objects: 767, done. remote: Total 767 (delta 633), reused 633 (delta 633), pack-reused 133 接收对象中: 100% (767/767), 121.42 KiB | 16.00 KiB/s, 完成. 处理 delta 中: 100% (641/641), 完成 234 个本地对象. 来自 https://github.com/tensorflow/tensorflow 8f91632..4addf4b master -> origin/master 更新 8f91632..4addf4b error: Your local changes to the following files would be overwritten by merge: tensorflow/core/BUILD Please, commit your changes or stash them before you can merge. Aborting

szsongyj commented 8 years ago

Just as https://github.com/okuchaiev/tensorflow-float16/commit/c1b6623154e14799419f91bd197b7900a78dbc2b , I modified the following three files

tensorflow/contrib/session_bundle/BUILD tensorflow/core/BUILD tensorflow/core/kernels/BUILD

So when i execute syj@syj-dl:~/tensorflow$ git pull --recurse-submodules

It prompts me

error: Your local changes to the following files would be overwritten by merge: tensorflow/core/BUILD Aborting

What should I do?

szsongyj commented 8 years ago

@ andrewharp I need leave and be back tomorrow. Hope to still get your help regarding the pending issues. I will try when i am back ,and give you feedback I have been stucked in this issue for several days. Hope to get it solved ASAP. Thank you & Best Regards.

andrewharp commented 8 years ago

@szsongyj Perhaps try one of the methods suggested here: http://stackoverflow.com/questions/14318234/how-to-ignore-error-on-git-pull-about-my-local-changes-would-be-overwritten-by-m

Alternatively follow the xkcd method and start over, this time with --recurse-submodules :)

gustavla commented 8 years ago

Just to make sure we don't lose sight of this issue, there should be some discussion of what the best solution to this is. As I see it, the two easiest fixes is either to revert 7bcdcbbf or simply bump the officially supported version of bazel to 0.3.1.

shkr commented 8 years ago

If someone could also provide explanation for why this commit is necessary https://github.com/tensorflow/tensorflow/commit/7bcdcbbf60fc08346fd8016270a0563f4b51362b, that will also be useful.

szsongyj commented 8 years ago

`syj@syj-dl:~$ cd tensorflow/ syj@syj-dl:~/tensorflow$ git reset --hard HEAD 现在位于 8f91632 fix comment grammar (#4381)

syj@syj-dl:~/tensorflow$ git pull --recurse-submodules remote: Counting objects: 12, done. remote: Total 12 (delta 6), reused 6 (delta 6), pack-reused 6 展开对象中: 100% (12/12), 完成. 来自 https://github.com/tensorflow/tensorflow 4addf4b..503a202 master -> origin/master 更新 8f91632..503a202 Fast-forward README.md | 8 +- grpc.BUILD | 3 + tensorflow/cc/framework/cc_op_gen.cc | 5 + tensorflow/contrib/factorization/BUILD | 1 - tensorflow/contrib/factorization/init.py | 6 +- tensorflow/contrib/factorization/examples/mnist.py | 7 +- .../python/kernel_tests/clustering_ops_test.py | 15 +- .../python/kernel_tests/wals_solver_ops_test.py | 4 +- .../python/ops/factorization_ops_test.py | 19 +- .../factorization/python/ops/gmm_ops_test.py | 19 +- .../contrib/factorization/python/ops/gmm_test.py | 74 +- .../factorization/python/ops/kmeans_test.py | 93 +- tensorflow/contrib/framework/BUILD | 1 - tensorflow/contrib/framework/init.py | 2 +- .../contrib/framework/python/framework/init.py | 1 + .../framework/python/framework/deprecation.py | 80 ++ .../framework/python/framework/deprecation_test.py | 207 ++++ .../contrib/framework/python/ops/init.py | 1 - .../contrib/framework/python/ops/embedding_ops.py | 80 -- tensorflow/contrib/layers/BUILD | 1 - tensorflow/contrib/layers/init.py | 1 + .../contrib/layers/python/layers/embedding_ops.py | 14 +- .../contrib/layers/python/layers/feature_column.py | 52 +- .../contrib/layers/python/layers/optimizers.py | 7 +- .../layers/python/layers/optimizers_test.py | 11 +- tensorflow/contrib/learn/BUILD | 13 + .../estimators/dnn_sampled_softmax_classifier.py | 513 +++++++++ .../dnn_sampled_softmax_classifier_test.py | 382 +++++++ .../learn/python/learn/estimators/estimator.py | 4 +- .../learn/python/learn/estimators/linear.py | 16 +- .../learn/python/learn/estimators/random_forest.py | 55 +- .../learn/python/learn/learn_io/graph_io.py | 7 +- tensorflow/contrib/learn/python/learn/monitors.py | 4 +- .../learn/python/learn/tests/estimators_test.py | 26 +- tensorflow/contrib/learn/python/learn/trainable.py | 5 +- tensorflow/contrib/makefile/Dockerfile | 17 + tensorflow/contrib/makefile/build_with_docker.sh | 54 + tensorflow/contrib/rnn/BUILD | 44 +- tensorflow/contrib/rnn/ops/gru_ops.cc | 35 + tensorflow/contrib/rnn/ops/gru_ops_test.cc | 63 ++ tensorflow/contrib/rnn/ops/lstm_ops.cc | 84 ++ tensorflow/contrib/rnn/ops/lstm_ops_test.cc | 196 ++++ tensorflow/contrib/rnn/python/ops/gru_ops.py | 24 +- tensorflow/contrib/rnn/python/ops/lstm_ops.py | 67 +- tensorflow/contrib/slim/python/slim/evaluation.py | 50 +- .../contrib/slim/python/slim/evaluation_test.py | 5 + tensorflow/contrib/tensor_forest/data/data_ops.py | 37 +- tensorflow/core/BUILD | 2 + tensorflow/core/common_runtime/constant_folding.cc | 6 + tensorflow/core/debug/BUILD | 11 + tensorflow/core/debug/debug_service.proto | 35 + tensorflow/core/distributed_runtime/BUILD | 12 + .../core/distributed_runtime/master_session.cc | 214 +++- tensorflow/core/distributed_runtime/scheduler.cc | 319 ++++++ tensorflow/core/distributed_runtime/scheduler.h | 118 ++ tensorflow/core/framework/cost_graph.proto | 7 +- .../core/framework/shape_inference_testutil.cc | 2 +- tensorflow/core/kernels/constant_op.cc | 2 + tensorflow/core/kernels/cwise_op_abs.cc | 1 + tensorflow/core/kernels/cwise_op_div.cc | 7 +- tensorflow/core/kernels/cwise_op_gpu_abs.cu.cc | 2 +- tensorflow/core/kernels/cwise_op_gpu_div.cu.cc | 3 +- tensorflow/core/kernels/cwise_op_gpu_mul.cu.cc | 4 +- tensorflow/core/kernels/cwise_op_gpu_sign.cu.cc | 2 +- tensorflow/core/kernels/cwise_op_mul.cc | 8 +- tensorflow/core/kernels/cwise_op_sign.cc | 4 +- tensorflow/core/kernels/cwise_ops_gpu_common.cu.h | 3 + tensorflow/core/kernels/fft_ops.cc | 53 +- tensorflow/core/kernels/reverse_op.cc | 18 +- tensorflow/core/kernels/reverse_op_gpu.cu.cc | 1 - tensorflow/core/kernels/shape_ops.cc | 4 +- tensorflow/core/kernels/slice_op.cc | 4 + tensorflow/core/kernels/slice_op_cpu_impl.h | 2 +- tensorflow/core/kernels/slice_op_gpu.cu.cc | 2 + tensorflow/core/kernels/stack_ops.cc | 10 +- tensorflow/core/kernels/tensor_array_ops.cc | 2 + tensorflow/core/lib/io/match_test.cc | 5 +- tensorflow/core/lib/io/path.cc | 81 ++ tensorflow/core/lib/io/path.h | 9 + tensorflow/core/lib/io/path_test.cc | 20 + .../core/lib/io/snappy/snappy_buffers_test.cc | 2 +- tensorflow/core/lib/io/table_builder.cc | 2 +- tensorflow/core/lib/io/table_builder.h | 3 +- tensorflow/core/lib/io/zlib_buffers_test.cc | 2 +- tensorflow/core/lib/io/zlib_outputbuffer.cc | 5 +- tensorflow/core/lib/png/png_io.cc | 2 +- tensorflow/core/lib/strings/str_util.h | 1 + tensorflow/core/ops/compat/ops_history.v0.pbtxt | 215 ++-- tensorflow/core/ops/data_flow_ops.cc | 197 ++-- tensorflow/core/ops/data_flow_ops_test.cc | 63 +- tensorflow/core/ops/io_ops.cc | 53 +- tensorflow/core/ops/io_ops_test.cc | 49 +- tensorflow/core/ops/math_ops.cc | 117 +- tensorflow/core/ops/math_ops_test.cc | 28 +- tensorflow/core/ops/ops.pbtxt | 113 +- tensorflow/core/platform/cloud/gcs_file_system.cc | 243 ++++- tensorflow/core/platform/cloud/gcs_file_system.h | 7 +- .../core/platform/cloud/gcs_file_system_test.cc | 394 +++++-- .../platform/cloud/google_auth_provider_test.cc | 2 +- tensorflow/core/platform/cloud/http_request.cc | 82 +- tensorflow/core/platform/cloud/http_request.h | 33 +- tensorflow/core/platform/cloud/http_request_fake.h | 56 +- .../core/platform/cloud/http_request_test.cc | 75 +- tensorflow/core/platform/cloud/oauth_client.cc | 4 +- tensorflow/core/platform/env.cc | 3 +- tensorflow/core/platform/env_test.cc | 9 +- tensorflow/core/platform/file_system.cc | 7 +- tensorflow/core/platform/file_system.h | 3 +- tensorflow/core/protobuf/config.proto | 3 + tensorflow/core/public/version.h | 3 +- tensorflow/core/util/example_proto_fast_parsing.cc | 6 +- .../g3doc/api_docs/python/contrib.framework.md | 86 +- tensorflow/g3doc/api_docs/python/contrib.layers.md | 51 +- tensorflow/g3doc/api_docs/python/framework.md | 4 +- .../shard0/tf.contrib.framework.deprecated_args.md | 36 + ...ntrib.framework.safe_embedding_lookup_sparse.md | 44 - .../python/functions_and_classes/shard0/tf.fft.md | 10 +- .../functions_and_classes/shard0/tf.fft2d.md | 10 +- .../functions_and_classes/shard0/tf.ifft3d.md | 9 +- .../python/functions_and_classes/shard0/tf.mul.md | 2 +- .../functions_and_classes/shard0/tf.nn.ctc_loss.md | 17 +- .../functions_and_classes/shard1/tf.Tensor.md | 4 +- .../functions_and_classes/shard1/tf.batch_fft.md | 18 - .../shard1/tf.nn.embedding_lookup_sparse.md | 2 +- .../shard2/tf.contrib.layers.optimize_loss.md | 3 +- ....contrib.layers.safe_embedding_lookup_sparse.md | 45 + .../shard3/tf.batch_ifft3d.md | 18 - .../functions_and_classes/shard3/tf.ifft2d.md | 9 +- .../functions_and_classes/shard4/tf.batch_ifft.md | 18 - .../python/functions_and_classes/shard5/tf.div.md | 2 +- .../functions_and_classes/shard6/tf.batch_fft3d.md | 18 - .../shard6/tf.batch_ifft2d.md | 18 - .../functions_and_classes/shard6/tf.nn.elu.md | 2 +- .../shard6/tf.self_adjoint_eig.md | 8 +- .../functions_and_classes/shard7/tf.fft3d.md | 10 +- .../python/functions_and_classes/shard7/tf.ifft.md | 9 +- .../functions_and_classes/shard8/tf.Variable.md | 4 +- .../functions_and_classes/shard9/tf.batch_fft2d.md | 18 - tensorflow/g3doc/api_docs/python/index.md | 9 +- tensorflow/g3doc/api_docs/python/math_ops.md | 129 +-- tensorflow/g3doc/api_docs/python/nn.md | 21 +- tensorflow/g3doc/api_docs/python/state_ops.md | 4 +- tensorflow/python/BUILD | 1141 ++++++++++++++++---- tensorflow/python/build_defs.bzl | 27 + tensorflow/python/debug/BUILD | 2 + tensorflow/python/framework/function.py | 192 +++- tensorflow/python/framework/function_test.py | 109 +- tensorflow/python/framework/importer.py | 30 +- tensorflow/python/framework/importer_test.py | 32 + tensorflow/python/kernel_tests/BUILD | 2 +- tensorflow/python/kernel_tests/cwise_ops_test.py | 10 + tensorflow/python/kernel_tests/fft_ops_test.py | 149 +-- tensorflow/python/kernel_tests/reader_ops_test.py | 199 +++- .../kernel_tests/self_adjoint_eig_op_test.py | 45 + tensorflow/python/kernel_tests/stack_ops_test.py | 11 +- tensorflow/python/lib/io/file_io.py | 7 +- tensorflow/python/lib/io/py_record_reader.cc | 5 + tensorflow/python/lib/io/py_record_writer.cc | 5 + tensorflow/python/lib/io/tf_record.py | 21 +- tensorflow/python/ops/data_flow_ops.py | 134 +-- tensorflow/python/ops/embedding_ops.py | 7 +- tensorflow/python/ops/image_ops_test.py | 19 + tensorflow/python/ops/linalg_grad.py | 44 + tensorflow/python/ops/linalg_ops.py | 8 +- tensorflow/python/ops/math_grad.py | 60 +- tensorflow/python/ops/math_ops.py | 13 +- tensorflow/python/training/adam.py | 3 +- tensorflow/python/training/adam_test.py | 19 + tensorflow/tensorboard/README.md | 9 - .../components/vz-projector/scatterWebGL.ts | 84 +- .../vz-projector/scatterWebGLPointsCanvasLabels.ts | 194 ++-- tensorflow/tensorflow.bzl | 7 +- tensorflow/tools/docker/Dockerfile.devel | 2 +- tensorflow/tools/docker/Dockerfile.devel-gpu | 2 +- .../bin/crosstool_wrapper_driver_is_not_gcc.tpl | 3 + 175 files changed, 6287 insertions(+), 2114 deletions(-) delete mode 100644 tensorflow/contrib/framework/python/ops/embedding_ops.py create mode 100644 tensorflow/contrib/learn/python/learn/estimators/dnn_sampled_softmax_classifier.py create mode 100644 tensorflow/contrib/learn/python/learn/estimators/dnn_sampled_softmax_classifier_test.py create mode 100644 tensorflow/contrib/makefile/Dockerfile create mode 100755 tensorflow/contrib/makefile/build_with_docker.sh create mode 100644 tensorflow/contrib/rnn/ops/gru_ops_test.cc create mode 100644 tensorflow/contrib/rnn/ops/lstm_ops_test.cc create mode 100644 tensorflow/core/debug/debug_service.proto create mode 100644 tensorflow/core/distributed_runtime/scheduler.cc create mode 100644 tensorflow/core/distributed_runtime/scheduler.h create mode 100644 tensorflow/g3doc/api_docs/python/functions_and_classes/shard0/tf.contrib.framework.deprecated_args.md delete mode 100644 tensorflow/g3doc/api_docs/python/functions_and_classes/shard0/tf.contrib.framework.safe_embedding_lookup_sparse.md delete mode 100644 tensorflow/g3doc/api_docs/python/functions_and_classes/shard1/tf.batch_fft.md create mode 100644 tensorflow/g3doc/api_docs/python/functions_and_classes/shard2/tf.contrib.layers.safe_embedding_lookup_sparse.md delete mode 100644 tensorflow/g3doc/api_docs/python/functions_and_classes/shard3/tf.batch_ifft3d.md delete mode 100644 tensorflow/g3doc/api_docs/python/functions_and_classes/shard4/tf.batch_ifft.md delete mode 100644 tensorflow/g3doc/api_docs/python/functions_and_classes/shard6/tf.batch_fft3d.md delete mode 100644 tensorflow/g3doc/api_docs/python/functions_and_classes/shard6/tf.batch_ifft2d.md delete mode 100644 tensorflow/g3doc/api_docs/python/functions_and_classes/shard9/tf.batch_fft2d.md create mode 100644 tensorflow/python/build_defs.bzl

syj@syj-dl:~/tensorflow$ vi tensorflow/contrib/session_bundle/BUILD syj@syj-dl:~/tensorflow$ vi tensorflow/core/BUILD syj@syj-dl:~/tensorflow$ vi tensorflow/core/kernels/BUILD`

ERROR: /home/syj/tensorflow/tensorflow/tensorboard/bower/BUILD:5:1: no such package '@iron_autogrow_textarea//': Error downloading from https://github.com/polymerelements/iron-autogrow-textarea/archive/v1.0.12.tar.gz to /home/syj/.cache/bazel/_bazel_syj/47b860f752d3ee48e8d208f528a56395/external/iron_autogrow_textarea: Error downloading https://github.com/polymerelements/iron-autogrow-textarea/archive/v1.0.12.tar.gz to /home/syj/.cache/bazel/_bazel_syj/47b860f752d3ee48e8d208f528a56395/external/iron_autogrow_textarea/v1.0.12.tar.gz: Connection timed out and referenced by '//tensorflow/tensorboard/bower:bower'. ERROR: /home/syj/tensorflow/tensorflow/tensorboard/bower/BUILD:5:1: no such package '@iron_autogrow_textarea//': Error downloading from https://github.com/polymerelements/iron-autogrow-textarea/archive/v1.0.12.tar.gz to /home/syj/.cache/bazel/_bazel_syj/47b860f752d3ee48e8d208f528a56395/external/iron_autogrow_textarea: Error downloading https://github.com/polymerelements/iron-autogrow-textarea/archive/v1.0.12.tar.gz to /home/syj/.cache/bazel/_bazel_syj/47b860f752d3ee48e8d208f528a56395/external/iron_autogrow_textarea/v1.0.12.tar.gz: Connection timed out and referenced by '//tensorflow/tensorboard/bower:bower'. ERROR: Evaluation of query "deps((//... union @bazel_tools//tools/jdk:toolchain))" failed: errors were encountered while computing transitive closure. Configuration finished '.

`BUT,Using wget I can download successfully.

syj@syj-dl:~$ wget https://github.com/polymerelements/iron-autogrow-textarea/archive/v1.0.12.tar.gz wget: /usr/local/lib/libcrypto.so.1.0.0: no version information available (required by wget) wget: /usr/local/lib/libssl.so.1.0.0: no version information available (required by wget) wget: /usr/local/lib/libssl.so.1.0.0: no version information available (required by wget) --2016-09-16 23:53:20-- https://github.com/polymerelements/iron-autogrow-textarea/archive/v1.0.12.tar.gz 正在解析主机 github.com (github.com)... 192.30.253.112 正在连接 github.com (github.com)|192.30.253.112|:443... 已连接。已发出 HTTP 请求，正在等待回应... 302 Found 位置：https://codeload.github.com/PolymerElements/iron-autogrow-textarea/tar.gz/v1.0.12 [跟随至新的 URL] --2016-09-16 23:53:21-- https://codeload.github.com/PolymerElements/iron-autogrow-textarea/tar.gz/v1.0.12 正在解析主机 codeload.github.com (codeload.github.com)... 192.30.253.121 正在连接 codeload.github.com (codeload.github.com)|192.30.253.121|:443... 已连接。已发出 HTTP 请求，正在等待回应... 200 OK 长度： 8743 (8.5K) [application/x-gzip] 正在保存至: “v1.0.12.tar.gz”

v1.0.12.tar.gz 100%[===================>] 8.54K --.-KB/s in 0.001s

2016-09-16 23:53:23 (12.8 MB/s) - 已保存 “v1.0.12.tar.gz” [8743/8743]) `

aselle commented 8 years ago

@jart could you please take a look at this issue?

szsongyj commented 8 years ago

`syj@syj-dl:~$mv tensorflow tensorflow.bak0917

syj@syj-dl:~$ git clone https://github.com/tensorflow/tensorflow 正克隆到 'tensorflow'... remote: Counting objects: 87905, done. remote: Compressing objects: 100% (685/685), done. remote: Total 87905 (delta 384), reused 0 (delta 0), pack-reused 87215 接收对象中: 100% (87905/87905), 57.97 MiB | 1.28 MiB/s, 完成. 处理 delta 中: 100% (65026/65026), 完成. 检查连接... 完成。

syj@syj-dl:~$ cd tensorflow syj@syj-dl:~/tensorflow$ git pull --recurse-submodules Already up-to-date.

syj@syj-dl:~/tensorflow$ vi tensorflow/contrib/session_bundle/BUILD syj@syj-dl:~/tensorflow$ vi tensorflow/core/BUILD syj@syj-dl:~/tensorflow$ vi tensorflow/core/kernels/BUILD

syj@syj-dl:~/tensorflow$ ./configure ~/tensorflow ~/tensorflow Please specify the location of python. [Default is /home/syj/anaconda2/bin/python]: Do you wish to build TensorFlow with Google Cloud Platform support? [y/N] y Google Cloud Platform support will be enabled for TensorFlow Found possible Python library paths: /home/syj/anaconda2/lib/python2.7/site-packages Please input the desired Python library path to use. Default is [/home/syj/anaconda2/lib/python2.7/site-packages] /home/syj/anaconda2/lib/python2.7/site-packages Do you wish to build TensorFlow with GPU support? [y/N] y GPU support will be enabled for TensorFlow Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to use system default]: 8.0 Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: Please specify the Cudnn version you want to use. [Leave empty to use system default]: 5 Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: Please specify a list of comma-separated Cuda compute capabilities you want to build with. You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus. Please note that each additional compute capability significantly increases your build time and binary size.

INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes. . INFO: All external dependencies fetched successfully. Configuration finished `

szsongyj commented 8 years ago

Thank you all @andrewharp, @gustavla, @aselle, @shkr , with your kind help after over 10 days attempt the configuration passed. :)

szsongyj commented 8 years ago

Thank you @aselle , @jart

tensorflow / tensorflow

Error when building tensorflow from source #4319