eaplatanios
commented
6 years ago @lucaRadicalbit Can you try building TensorFlow using the node_def_fix branch in this repository and let me know if that works?
Closed lucataglia closed 6 years ago
eaplatanios
commented
6 years ago @lucaRadicalbit Can you try building TensorFlow using the node_def_fix branch in this repository and let me know if that works?
lucataglia
commented
6 years ago I compiled the tensorflow from the branch you told me but it's like nothing change. I compiled with bazel 0.5.4 and I gave the same configuration option I gave yesterday: python location: /usr/bin/python python library path: /ust/local/lib/python2.7/dist-packages build TensorFlow with jemalloc as malloc support: Y ALL other question: N
Than I run the bazel build command:
bazel build --config=opt //tensorflow:libtensorflow.so
Here below I include some terminal lines to make you sure I compile the right branch. As you can see after bazel end his work the prompt say git:(node_def_fix)
In file included from external/gemmlowp/meta/../internal/simd_wrappers.h:505:0,
from external/gemmlowp/meta/../internal/output.h:28,
from external/gemmlowp/meta/../internal/unpack.h:23,
from external/gemmlowp/meta/../internal/single_thread_gemm.h:29,
from external/gemmlowp/meta/../internal/multi_thread_gemm.h:26,
from external/gemmlowp/meta/multi_thread_common.h:18,
from external/gemmlowp/meta/multi_thread_gemm.h:18,
from ./tensorflow/core/kernels/meta_support.h:19,
from tensorflow/core/kernels/dequantize_op.cc:24:
external/gemmlowp/meta/../internal/simd_wrappers_sse.h:30:72: warning: ignoring attributes on template argument 'gemmlowp::Int32x4 {aka __vector(2) long long int}' [-Wignored-attributes]
typename std::conditional<ScalarCount >= 4, Int32x4, std::int32_t>::type;
^
external/gemmlowp/meta/../internal/simd_wrappers_sse.h:38:52: warning: ignoring attributes on template argument 'gemmlowp::Uint8x16 {aka __vector(2) long long int}' [-Wignored-attributes]
std::uint8_t>::type>::type;
^
Target //tensorflow:libtensorflow.so up-to-date:
bazel-bin/tensorflow/libtensorflow.so
INFO: Elapsed time: 5375.524s, Critical Path: 154.73s
➜ tensorflow_eaplatanios git:(node_def_fix) ll
total 224K
-rw-rw-r-- 1 osboxes osboxes 2.2K Oct 18 12:43 ACKNOWLEDGMENTS
-rw-rw-r-- 1 osboxes osboxes 542 Oct 18 12:43 ADOPTERS.md
-rw-rw-r-- 1 osboxes osboxes 1.2K Oct 18 12:43 arm_compiler.BUILD
-rw-rw-r-- 1 osboxes osboxes 347 Oct 18 12:43 AUTHORS
lrwxrwxrwx 1 osboxes osboxes 122 Oct 19 04:46 bazel-bin -> /home/osboxes/.cache/bazel/_bazel_osboxes/c41112d8ac886bef5093a3fb5077c388/execroot/org_tensorflow/bazel-out/local-opt/bin
lrwxrwxrwx 1 osboxes osboxes 127 Oct 19 04:46 bazel-genfiles -> /home/osboxes/.cache/bazel/_bazel_osboxes/c41112d8ac886bef5093a3fb5077c388/execroot/org_tensorflow/bazel-out/local-opt/genfiles
lrwxrwxrwx 1 osboxes osboxes 108 Oct 19 04:46 bazel-out -> /home/osboxes/.cache/bazel/_bazel_osboxes/c41112d8ac886bef5093a3fb5077c388/execroot/org_tensorflow/bazel-out
lrwxrwxrwx 1 osboxes osboxes 98 Oct 19 04:46 bazel-tensorflow_eaplatanios -> /home/osboxes/.cache/bazel/_bazel_osboxes/c41112d8ac886bef5093a3fb5077c388/execroot/org_tensorflow
lrwxrwxrwx 1 osboxes osboxes 127 Oct 19 04:46 bazel-testlogs -> /home/osboxes/.cache/bazel/_bazel_osboxes/c41112d8ac886bef5093a3fb5077c388/execroot/org_tensorflow/bazel-out/local-opt/testlogs
-rw-rw-r-- 1 osboxes osboxes 0 Oct 18 12:43 BUILD
-rw-rw-r-- 1 osboxes osboxes 5.2K Oct 18 12:43 CODE_OF_CONDUCT.md
-rw-rw-r-- 1 osboxes osboxes 2.4K Oct 18 12:43 CODEOWNERS
-rwxrwxr-x 1 osboxes osboxes 231 Oct 18 12:43 configure
-rw-rw-r-- 1 osboxes osboxes 37K Oct 19 04:44 configure.py
-rw-rw-r-- 1 osboxes osboxes 7.5K Oct 18 12:43 CONTRIBUTING.md
-rw-rw-r-- 1 osboxes osboxes 1.9K Oct 18 12:43 ISSUE_TEMPLATE.md
-rw-rw-r-- 1 osboxes osboxes 12K Oct 18 12:43 LICENSE
-rw-rw-r-- 1 osboxes osboxes 328 Oct 18 12:43 models.BUILD
-rw-rw-r-- 1 osboxes osboxes 8.9K Oct 19 04:44 README.md
-rw-rw-r-- 1 osboxes osboxes 71K Oct 19 04:44 RELEASE.md
drwxrwxr-x 17 osboxes osboxes 4.0K Oct 19 04:44 tensorflow
drwxrwxr-x 18 osboxes osboxes 4.0K Oct 19 04:44 third_party
drwxrwxr-x 2 osboxes osboxes 4.0K Oct 19 04:44 tools
drwxrwxr-x 3 osboxes osboxes 4.0K Oct 18 12:43 util
-rw-rw-r-- 1 osboxes osboxes 3.3K Oct 18 12:43 WORKSPACE
➜ tensorflow sudo vim /etc/ld.so.conf.d/libc.conf
➜ tensorflow sudo ldconfig
/sbin/ldconfig.real: /home/osboxes/Desktop/tensorflow_eaplatanios/bazel-bin/tensorflow/libtensorflow.so.runfiles_manifest is not an ELF file - it has the wrong magic bytes at the start.
/sbin/ldconfig.real: /home/osboxes/Desktop/tensorflow_eaplatanios/bazel-bin/tensorflow/libtensorflow.so-2.params is not an ELF file - it has the wrong magic bytes at the start.
/sbin/ldconfig.real: /home/osboxes/Desktop/tensorflow_eaplatanios/bazel-bin/tensorflow/libtensorflow_framework.so-2.params is not an ELF file - it has the wrong magic bytes at the start.
➜ tensorflow ldconfig -p | grep tensor
libtensorflow_framework.so (libc6,x86-64) => /home/osboxes/Desktop/tensorflow_eaplatanios/bazel-bin/tensorflow/libtensorflow_framework.so
libtensorflow.so (libc6,x86-64) => /home/osboxes/Desktop/tensorflow_eaplatanios/bazel-bin/tensorflow/libtensorflow.so
➜ tensorflow
After I add manualy the dynamic lib, I open Intellj and I try to run the LinearRegression example but I got the same error:
/usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -javaagent:/opt/idea-IC-172.4343.14/lib/idea_rt.jar=40089:/opt/idea-IC-172.4343.14/bin -Dfile.encoding=UTF-8 -classpath /usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/charsets.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/cldrdata.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/dnsns.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/icedtea-sound.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/jaccess.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/java-atk-wrapper.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/localedata.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/nashorn.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/sunec.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/sunjce_provider.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/sunpkcs11.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/zipfs.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/jce.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/jsse.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/management-agent.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/resources.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/rt.jar:/home/osboxes/IdeaProjects/HelloWorldScalaUbuntu/target/scala-2.11/classes:/home/osboxes/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.11.11.jar:/home/osboxes/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.2.3.jar:/home/osboxes/.ivy2/cache/ch.qos.logback/logback-core/jars/logback-core-1.2.3.jar:/home/osboxes/.ivy2/cache/com.chuusai/shapeless_2.11/bundles/shapeless_2.11-2.3.2.jar:/home/osboxes/.ivy2/cache/com.github.mpilquist/simulacrum_2.11/jars/simulacrum_2.11-0.10.0.jar:/home/osboxes/.ivy2/cache/com.google.protobuf/protobuf-java/bundles/protobuf-java-3.4.0.jar:/home/osboxes/.ivy2/cache/com.typesafe.scala-logging/scala-logging_2.11/bundles/scala-logging_2.11-3.7.2.jar:/home/osboxes/.ivy2/cache/io.circe/circe-core_2.11/jars/circe-core_2.11-0.8.0.jar:/home/osboxes/.ivy2/cache/io.circe/circe-generic_2.11/jars/circe-generic_2.11-0.8.0.jar:/home/osboxes/.ivy2/cache/io.circe/circe-jawn_2.11/jars/circe-jawn_2.11-0.8.0.jar:/home/osboxes/.ivy2/cache/io.circe/circe-numbers_2.11/jars/circe-numbers_2.11-0.8.0.jar:/home/osboxes/.ivy2/cache/io.circe/circe-parser_2.11/jars/circe-parser_2.11-0.8.0.jar:/home/osboxes/.ivy2/cache/junit/junit/jars/junit-4.12.jar:/home/osboxes/.ivy2/cache/org.hamcrest/hamcrest-core/jars/hamcrest-core-1.3.jar:/home/osboxes/.ivy2/cache/org.platanios/tensorflow-api_2.11/jars/tensorflow-api_2.11-0.1.0-SNAPSHOT.jar:/home/osboxes/.ivy2/cache/org.platanios/tensorflow_2.11/jars/tensorflow_2.11-0.1.0-SNAPSHOT.jar:/home/osboxes/.ivy2/cache/org.scala-lang/scala-reflect/jars/scala-reflect-2.11.11.jar:/home/osboxes/.ivy2/cache/org.scalactic/scalactic_2.11/bundles/scalactic_2.11-3.0.1.jar:/home/osboxes/.ivy2/cache/org.slf4j/slf4j-api/jars/slf4j-api-1.7.25.jar:/home/osboxes/.ivy2/cache/org.spire-math/jawn-parser_2.11/jars/jawn-parser_2.11-0.10.4.jar:/home/osboxes/.ivy2/cache/org.tensorflow/proto/jars/proto-1.3.0.jar:/home/osboxes/.ivy2/cache/org.typelevel/algebra_2.11/jars/algebra_2.11-0.7.0.jar:/home/osboxes/.ivy2/cache/org.typelevel/cats-core_2.11/jars/cats-core_2.11-0.9.0.jar:/home/osboxes/.ivy2/cache/org.typelevel/cats-kernel_2.11/jars/cats-kernel_2.11-0.9.0.jar:/home/osboxes/.ivy2/cache/org.typelevel/cats-macros_2.11/jars/cats-macros_2.11-0.9.0.jar:/home/osboxes/.ivy2/cache/org.typelevel/machinist_2.11/jars/machinist_2.11-0.6.1.jar:/home/osboxes/.ivy2/cache/org.typelevel/macro-compat_2.11/jars/macro-compat_2.11-1.1.1.jar:/home/osboxes/.ivy2/cache/org.typelevel/spire-macros_2.11/jars/spire-macros_2.11-0.14.1.jar:/home/osboxes/.ivy2/cache/org.typelevel/spire_2.11/jars/spire_2.11-0.14.1.jar:/home/osboxes/.ivy2/cache/org.platanios/tensorflow-jni_2.11/jars/tensorflow-jni_2.11-0.1.0-SNAPSHOT.jar org.platanios.tensorflow.examples.LinearRegression
2017-10-19 06:57:47.279 [main] INFO Examples / Linear Regression - Building linear regression model.
2017-10-19 06:57:48.249 [main] INFO TensorFlow Native - Extracting the 'tensorflow_jni' native library to /tmp/tensorflow_scala_native_libraries4287453170833863926/libtensorflow_jni.so.
2017-10-19 06:57:48.299 [main] INFO TensorFlow Native - Copied 557880 bytes to /tmp/tensorflow_scala_native_libraries4287453170833863926/libtensorflow_jni.so.
2017-10-19 06:57:48.573 [main] INFO TensorFlow Native - Extracting the 'tensorflow_ops' native library to /tmp/tensorflow_scala_native_libraries4287453170833863926/libtensorflow_ops.so.
2017-10-19 06:57:48.575 [main] INFO TensorFlow Native - Copied 48536 bytes to /tmp/tensorflow_scala_native_libraries4287453170833863926/libtensorflow_ops.so.
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.platanios.tensorflow.jni.Tensor$.<init>(Tensor.scala:24)
at org.platanios.tensorflow.jni.Tensor$.<clinit>(Tensor.scala)
at org.platanios.tensorflow.api.tensors.Context$.apply(Context.scala:50)
at org.platanios.tensorflow.api.package$.<init>(package.scala:47)
at org.platanios.tensorflow.api.package$.<clinit>(package.scala)
at org.platanios.tensorflow.examples.LinearRegression$.main(LinearRegression.scala:37)
at org.platanios.tensorflow.examples.LinearRegression.main(LinearRegression.scala)
Caused by: org.platanios.tensorflow.jni.NotFoundException: /tmp/tensorflow_scala_native_libraries4287453170833863926/libtensorflow_ops.so: undefined symbol: _ZN10tensorflow7strings8internal9CatPiecesESt16initializer_listINS_11StringPieceEE
at org.platanios.tensorflow.jni.TensorFlow$.loadOpLibrary(Native Method)
at org.platanios.tensorflow.jni.TensorFlow$$anonfun$load$6.apply(TensorFlow.scala:133)
at org.platanios.tensorflow.jni.TensorFlow$$anonfun$load$6.apply(TensorFlow.scala:133)
at scala.Option.foreach(Option.scala:257)
at org.platanios.tensorflow.jni.TensorFlow$.load(TensorFlow.scala:133)
at org.platanios.tensorflow.jni.TensorFlow$.<init>(TensorFlow.scala:181)
at org.platanios.tensorflow.jni.TensorFlow$.<clinit>(TensorFlow.scala)
... 7 more
Process finished with exit code 1
@lucaRadicalbit It looks like libtensorflow_framework.so is not loaded. Can you try copying your compiled libtensorflow.so and libtensorflow_framework.so to a path like /usr/local/lib and see if it works then? If it does, it means that for some reason the loader cannot find the dynamic libraries in your current setup.
eaplatanios
commented
6 years ago Also, when you say that you manually add the dynamic lib, what do you mean?
lucataglia
commented
6 years ago I mean since I need to add the bazel-bin/tensorflow directory that contain libtensorflow_framework.so and libtensorflow.so to the list of directory used by ldconfig in order to make these two libraries usable as shared object I read that there are two ways to do this: just add to LD_LIBRARY_PATH or rebuild cache. I chose the latter:
[1] - I add a line inside /etc/ld.so.conf.d/lib.conf with the path to the folder bazel-bin/tensorflow
[2] - run the commend sudo ldconfig
[3] - run the command ldconfig -p | grep tensorflow to check that the library was load correctly
As soon as possible I try your solution to move the libraries inside /usr/local/lib and I let you know
lucataglia
commented
6 years ago @eaplatanios I try to:
libtensorflow_framework.so and libtensorflow.so to /usr/local/libsudo ldconfig /usr/local/lib ldconfig -p | grep libtensorflow to check that the libraries were loadedeaplatanios
commented
6 years ago This is interesting. Can you try changing the library dependencies line in your build.sbt file to:
libraryDependencies += "org.platanios" %% "tensorflow" % "0.1.0-SNAPSHOT" classifier "linux-cpu-x86_64"
and try again? I'm just curious to see if it will work fine with the provided dynamic libraries.
lucataglia
commented
6 years ago 
If I use the provided dynamic libraries the error change
eaplatanios
commented
6 years ago Could you please send me the whole error? The line is cut off in the screenshot and I cannot see which shared object file it cannot open.
lucataglia
commented
6 years ago Sorry, this is the full error
/usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -javaagent:/opt/idea-IC-172.4343.14/lib/idea_rt.jar=42893:/opt/idea-IC-172.4343.14/bin -Dfile.encoding=UTF-8 -classpath /usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/charsets.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/cldrdata.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/dnsns.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/icedtea-sound.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/jaccess.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/java-atk-wrapper.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/localedata.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/nashorn.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/sunec.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/sunjce_provider.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/sunpkcs11.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/ext/zipfs.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/jce.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/jsse.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/management-agent.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/resources.jar:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/rt.jar:/home/osboxes/IdeaProjects/HelloWorldScalaUbuntu/target/scala-2.11/classes:/home/osboxes/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.11.11.jar:/home/osboxes/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.2.3.jar:/home/osboxes/.ivy2/cache/ch.qos.logback/logback-core/jars/logback-core-1.2.3.jar:/home/osboxes/.ivy2/cache/com.chuusai/shapeless_2.11/bundles/shapeless_2.11-2.3.2.jar:/home/osboxes/.ivy2/cache/com.github.mpilquist/simulacrum_2.11/jars/simulacrum_2.11-0.10.0.jar:/home/osboxes/.ivy2/cache/com.google.protobuf/protobuf-java/bundles/protobuf-java-3.4.0.jar:/home/osboxes/.ivy2/cache/com.typesafe.scala-logging/scala-logging_2.11/bundles/scala-logging_2.11-3.7.2.jar:/home/osboxes/.ivy2/cache/io.circe/circe-core_2.11/jars/circe-core_2.11-0.8.0.jar:/home/osboxes/.ivy2/cache/io.circe/circe-generic_2.11/jars/circe-generic_2.11-0.8.0.jar:/home/osboxes/.ivy2/cache/io.circe/circe-jawn_2.11/jars/circe-jawn_2.11-0.8.0.jar:/home/osboxes/.ivy2/cache/io.circe/circe-numbers_2.11/jars/circe-numbers_2.11-0.8.0.jar:/home/osboxes/.ivy2/cache/io.circe/circe-parser_2.11/jars/circe-parser_2.11-0.8.0.jar:/home/osboxes/.ivy2/cache/junit/junit/jars/junit-4.12.jar:/home/osboxes/.ivy2/cache/org.hamcrest/hamcrest-core/jars/hamcrest-core-1.3.jar:/home/osboxes/.ivy2/cache/org.platanios/tensorflow-api_2.11/jars/tensorflow-api_2.11-0.1.0-SNAPSHOT.jar:/home/osboxes/.ivy2/cache/org.scala-lang/scala-reflect/jars/scala-reflect-2.11.11.jar:/home/osboxes/.ivy2/cache/org.scalactic/scalactic_2.11/bundles/scalactic_2.11-3.0.1.jar:/home/osboxes/.ivy2/cache/org.slf4j/slf4j-api/jars/slf4j-api-1.7.25.jar:/home/osboxes/.ivy2/cache/org.spire-math/jawn-parser_2.11/jars/jawn-parser_2.11-0.10.4.jar:/home/osboxes/.ivy2/cache/org.tensorflow/proto/jars/proto-1.3.0.jar:/home/osboxes/.ivy2/cache/org.typelevel/algebra_2.11/jars/algebra_2.11-0.7.0.jar:/home/osboxes/.ivy2/cache/org.typelevel/cats-core_2.11/jars/cats-core_2.11-0.9.0.jar:/home/osboxes/.ivy2/cache/org.typelevel/cats-kernel_2.11/jars/cats-kernel_2.11-0.9.0.jar:/home/osboxes/.ivy2/cache/org.typelevel/cats-macros_2.11/jars/cats-macros_2.11-0.9.0.jar:/home/osboxes/.ivy2/cache/org.typelevel/machinist_2.11/jars/machinist_2.11-0.6.1.jar:/home/osboxes/.ivy2/cache/org.typelevel/macro-compat_2.11/jars/macro-compat_2.11-1.1.1.jar:/home/osboxes/.ivy2/cache/org.typelevel/spire-macros_2.11/jars/spire-macros_2.11-0.14.1.jar:/home/osboxes/.ivy2/cache/org.typelevel/spire_2.11/jars/spire_2.11-0.14.1.jar:/home/osboxes/.ivy2/cache/org.platanios/tensorflow-jni_2.11/jars/tensorflow-jni_2.11-0.1.0-SNAPSHOT.jar:/home/osboxes/.ivy2/cache/org.platanios/tensorflow_2.11/jars/tensorflow_2.11-0.1.0-SNAPSHOT-linux-cpu-x86_64.jar org.platanios.tensorflow.examples.LinearRegression
2017-10-19 11:52:40.890 [main] INFO Examples / Linear Regression - Building linear regression model.
2017-10-19 11:52:41.187 [main] INFO TensorFlow Native - Extracting the 'tensorflow_framework' native library to /tmp/tensorflow_scala_native_libraries8055538410184168067/libtensorflow_framework.so.
2017-10-19 11:52:41.314 [main] INFO TensorFlow Native - Copied 15408832 bytes to /tmp/tensorflow_scala_native_libraries8055538410184168067/libtensorflow_framework.so.
2017-10-19 11:52:41.330 [main] INFO TensorFlow Native - Extracting the 'tensorflow' native library to /tmp/tensorflow_scala_native_libraries8055538410184168067/libtensorflow.so.
2017-10-19 11:52:41.836 [main] INFO TensorFlow Native - Copied 45300776 bytes to /tmp/tensorflow_scala_native_libraries8055538410184168067/libtensorflow.so.
2017-10-19 11:52:41.898 [main] INFO TensorFlow Native - Extracting the 'tensorflow_jni' native library to /tmp/tensorflow_scala_native_libraries8055538410184168067/libtensorflow_jni.so.
2017-10-19 11:52:41.902 [main] INFO TensorFlow Native - Copied 557872 bytes to /tmp/tensorflow_scala_native_libraries8055538410184168067/libtensorflow_jni.so.
Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp/tensorflow_scala_native_libraries8055538410184168067/libtensorflow_jni.so: libtensorflow.so: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at org.platanios.tensorflow.jni.TensorFlow$$anonfun$load$5.apply(TensorFlow.scala:121)
at org.platanios.tensorflow.jni.TensorFlow$$anonfun$load$5.apply(TensorFlow.scala:119)
at scala.Option.foreach(Option.scala:257)
at org.platanios.tensorflow.jni.TensorFlow$.load(TensorFlow.scala:119)
at org.platanios.tensorflow.jni.TensorFlow$.<init>(TensorFlow.scala:181)
at org.platanios.tensorflow.jni.TensorFlow$.<clinit>(TensorFlow.scala)
at org.platanios.tensorflow.jni.Tensor$.<init>(Tensor.scala:24)
at org.platanios.tensorflow.jni.Tensor$.<clinit>(Tensor.scala)
at org.platanios.tensorflow.api.tensors.Context$.apply(Context.scala:50)
at org.platanios.tensorflow.api.package$.<init>(package.scala:47)
at org.platanios.tensorflow.api.package$.<clinit>(package.scala)
at org.platanios.tensorflow.examples.LinearRegression$.main(LinearRegression.scala:37)
at org.platanios.tensorflow.examples.LinearRegression.main(LinearRegression.scala)
Process finished with exit code 1@lucaRadicalbit Ok that's interesting. Let me investigate a bit later today (I have to attend a meeting now) and I'll update you by the end of the day today.
eaplatanios
commented
6 years ago @lucaRadicalbit Sorry I got super busy and just got a chance to get back to this. Are you still having that same issue?
mandar2812
commented
6 years ago @eaplatanios When I run on my Mac, this issue does not arise, but when I run on a managed (organisational) Fedora server, I get the exact issue. It tries to extract into /tmp/tensorflow_scala_native_libraries ... just like above.
eaplatanios
commented
6 years ago @mandar2812 That's interesting. The extraction happens if you do not have the native libraries installed on the machine you're using and you want to pull pre-compiled versions of them using the SBT classifier "linux..." flag. It seems that even though it extracts libtensorflow.so in the same temporary directory, the dynamic library loader cannot load it when loading libtensorflow_jni.so which is also extracted in the same directory. I remember that for Mac, I resolved this issue using a proper library name including @rpath for the TensorFlow native libraries. I think that similarly, maybe something is wrong with how the TensorFlow binaries are compiled for linux. Can you tell me if the same thing happens if you place pre-compiled libtensorflow.so and libtensorflow_framework.so files in a directory such as /usr/local/lib and avoid using the SBT classifier for the dependency? You might need to clear you ivy cache to do that because it will have already downloaded the JAR files containing the pre-compiled binaries.
eaplatanios
commented
6 years ago @lucaRadicalbit @mandar2812 I recently made some changes in how the dynamic libraries are loaded. Could you please let me know if you still have the issues you had?
lucataglia
commented
6 years ago @eaplatanios The LinearRegression example on my UbuntuMate now works
mandar2812
commented
6 years ago @eaplatanios Yes it works now on Linux, but it does not load anymore on Mac :(
DynaML>{
import org.platanios.tensorflow.api._
import org.platanios.tensorflow.api.tf.learn._
import org.platanios.tensorflow.data.loaders.MNISTLoader
}
import org.platanios.tensorflow.api._
import org.platanios.tensorflow.api.tf.learn._
import org.platanios.tensorflow.data.loaders.MNISTLoader
DynaML>val t2 = Tensor(-0.2, 1.1)
2017-11-06 18:01:03.471 [main] INFO TensorFlow Native - Extracting the 'tensorflow_framework' native library to /var/folders/5j/mhg359d57271g_hqg92jl5900000gn/T/tensorflow_scala_native_libraries8750870745264600420/libtensorflow_framework.so.
2017-11-06 18:01:03.560 [main] INFO TensorFlow Native - Copied 11419400 bytes to /var/folders/5j/mhg359d57271g_hqg92jl5900000gn/T/tensorflow_scala_native_libraries8750870745264600420/libtensorflow_framework.so.
2017-11-06 18:01:03.562 [main] INFO TensorFlow Native - Extracting the 'tensorflow' native library to /var/folders/5j/mhg359d57271g_hqg92jl5900000gn/T/tensorflow_scala_native_libraries8750870745264600420/libtensorflow.so.
2017-11-06 18:01:03.920 [main] INFO TensorFlow Native - Copied 102600676 bytes to /var/folders/5j/mhg359d57271g_hqg92jl5900000gn/T/tensorflow_scala_native_libraries8750870745264600420/libtensorflow.so.
2017-11-06 18:01:03.921 [main] INFO TensorFlow Native - Extracting the 'tensorflow_jni' native library to /var/folders/5j/mhg359d57271g_hqg92jl5900000gn/T/tensorflow_scala_native_libraries8750870745264600420/libtensorflow_jni.so.
2017-11-06 18:01:03.924 [main] INFO TensorFlow Native - Copied 612240 bytes to /var/folders/5j/mhg359d57271g_hqg92jl5900000gn/T/tensorflow_scala_native_libraries8750870745264600420/libtensorflow_jni.so.
java.lang.UnsatisfiedLinkError: /private/var/folders/5j/mhg359d57271g_hqg92jl5900000gn/T/tensorflow_scala_native_libraries8750870745264600420/libtensorflow_jni.so: dlopen(/private/var/folders/5j/mhg359d57271g_hqg92jl5900000gn/T/tensorflow_scala_native_libraries8750870745264600420/libtensorflow_jni.so, 1): Library not loaded: @rpath/libtensorflow.so
Referenced from: /private/var/folders/5j/mhg359d57271g_hqg92jl5900000gn/T/tensorflow_scala_native_libraries8750870745264600420/libtensorflow_jni.so
Reason: image not found
java.lang.ClassLoader$NativeLibrary.load(Native Method)
java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
java.lang.Runtime.load0(Runtime.java:809)
java.lang.System.load(System.java:1086)
org.platanios.tensorflow.jni.TensorFlow$$anonfun$load$3.apply(TensorFlow.scala:95)
org.platanios.tensorflow.jni.TensorFlow$$anonfun$load$3.apply(TensorFlow.scala:93)
scala.Option.foreach(Option.scala:257)
org.platanios.tensorflow.jni.TensorFlow$.load(TensorFlow.scala:93)
org.platanios.tensorflow.jni.TensorFlow$.<init>(TensorFlow.scala:155)
org.platanios.tensorflow.jni.TensorFlow$.<clinit>(TensorFlow.scala)
org.platanios.tensorflow.jni.Tensor$.<init>(Tensor.scala:24)
org.platanios.tensorflow.jni.Tensor$.<clinit>(Tensor.scala)
org.platanios.tensorflow.api.tensors.Context$.apply(Context.scala:50)
org.platanios.tensorflow.api.package$.<init>(package.scala:47)
org.platanios.tensorflow.api.package$.<clinit>(package.scala)
ammonite.$sess.cmd1$.<init>(cmd1.sc:1)
ammonite.$sess.cmd1$.<clinit>(cmd1.sc)
@mandar2812 I'm closing this and we can move the conversation to #41, which collects all related issues in one place.
ashwinravishankar
commented
5 years ago @eaplatanios When I run on my Mac, this issue does not arise, but when I run on a managed (organisational) Fedora server, I get the exact issue. It tries to extract into /tmp/tensorflow_scala_native_libraries ... just like above.
@mandar2812 Surprisingly, I'm currently facing this same issue. A program which works well on my mac, throws this same error when I run on a CentOS 7 machine. I'm using JDK 11, and the precompiled binaries - 0.4.1 of artifact tensorflow_2.21. I update the flag to linux-cpu-x86_64 on mac, package it, then ship to CentOS7.
StackTrace:
Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp/tensorflow_scala_native_libraries8675119614557414070/libtensorflow_jni.so: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /tmp/tensorflow_scala_native_libraries8675119614557414070/libtensorflow_jni.so)
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at org.platanios.tensorflow.jni.TensorFlow$.$anonfun$load$4(TensorFlow.scala:95)
at org.platanios.tensorflow.jni.TensorFlow$.$anonfun$load$4$adapted(TensorFlow.scala:93)
at scala.Option.foreach(Option.scala:274)
at org.platanios.tensorflow.jni.TensorFlow$.load(TensorFlow.scala:93)
at org.platanios.tensorflow.jni.TensorFlow$.<init>(TensorFlow.scala:155)
at org.platanios.tensorflow.jni.TensorFlow$.<clinit>(TensorFlow.scala)
at org.platanios.tensorflow.jni.Tensor$.<init>(Tensor.scala:24)
at org.platanios.tensorflow.jni.Tensor$.<clinit>(Tensor.scala)
at org.platanios.tensorflow.api.tensors.Tensor$.fill(Tensor.scala:835)
at org.platanios.tensorflow.api.tensors.Tensor$.zeros(Tensor.scala:726)
at com.cambridgesemantics.anzograph.tf.TFVersion.apply(TFVersion.scala:37)
at Test.main(Test.java:9)To get more insight on what dependencies am missing, I tried:
[ashwinravishankar@ash-test-00 so]$ ldd libtensorflow_jni.so
ldd: warning: you do not have execution permission for `./libtensorflow_jni.so'
./libtensorflow_jni.so: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by ./libtensorflow_jni.so)
./libtensorflow_jni.so: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/ashwinravishankar/so/./libtensorflow.so)
./libtensorflow_jni.so: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/ashwinravishankar/so/./libtensorflow.so)
./libtensorflow_jni.so: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /home/ashwinravishankar/so/./libtensorflow.so)
./libtensorflow_jni.so: /lib64/libm.so.6: version `GLIBC_2.23' not found (required by /home/ashwinravishankar/so/./libtensorflow.so)
./libtensorflow_jni.so: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /home/ashwinravishankar/so/./libtensorflow_framework.so)
./libtensorflow_jni.so: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/ashwinravishankar/so/./libtensorflow_framework.so)
./libtensorflow_jni.so: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/ashwinravishankar/so/./libtensorflow_framework.so)
linux-vdso.so.1 => (0x00007ffc72fd8000)
libtensorflow.so => /home/ashwinravishankar/so/./libtensorflow.so (0x00007f5e79fde000)
libtensorflow_framework.so => /home/ashwinravishankar/so/./libtensorflow_framework.so (0x00007f5e79204000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f5e78efd000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f5e78ce7000)
libc.so.6 => /lib64/libc.so.6 (0x00007f5e7891a000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f5e78716000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5e784fa000)
libm.so.6 => /lib64/libm.so.6 (0x00007f5e781f8000)
librt.so.1 => /lib64/librt.so.1 (0x00007f5e77ff0000)
/lib64/ld-linux-x86-64.so.2 (0x00007f5e821d4000)
I am using ubuntuMate 17.04 and I try use your TensorFlow Scala API installing the tensorflow as dynamic library. After recovering the tensorflow repository I try to compile first the last version available (tensorflow master: commit 27767d8e9 ) and than the release 1.0 (tensorflow r1.0: commit 905662a1c) both with bazel 0.6.1 first and than with bazel 0.5.4 since I read that bazel 0.6.1 compiling tensoflow has some problems.
When I run the configure I gave the following setup: python location: /usr/bin/python python library path: /ust/local/lib/python2.7/dist-packages build TensorFlow with jemalloc as malloc support: Y ALL other question: N
And than I run the bazel build command:
bazel build --config=opt //tensorflow:libtensorflow.soThan I created a IntellJ project in order to run you LinearRegression example, but the result was always:
I am using scala 2.11.11 and sbt 0.13.6. Here my build.sbt:
I can't figured out how to solve this runtime error. Am I missing something to do ?