search

eaplatanios / tensorflow_scala

TensorFlow API for the Scala Programming Language
http://platanios.org/tensorflow_scala/
Apache License 2.0
937 stars 95 forks source link

Libraries claiming to be extracted but not working #37

Closed siddhartha-gadgil closed 6 years ago

siddhartha-gadgil commented 6 years ago

I was trying to experiment with tensorflow_scala, but I get an error when running a tiny test program (in sbt) after a separate tensorflow installation

2017-11-03 10:35:30.468 [run-main-0] INFO  TensorFlow Native - Extracting the 'tensorflow_jni' native library to /tmp/tensorflow_scala_native_libraries1715329719143383324/libtensorflow_jni.so.
2017-11-03 10:35:30.475 [run-main-0] INFO  TensorFlow Native - Copied 628432 bytes to /tmp/tensorflow_scala_native_libraries1715329719143383324/libtensorflow_jni.so.
2017-11-03 10:35:30.476 [run-main-0] INFO  TensorFlow Native - Extracting the 'tensorflow_ops' native library to /tmp/tensorflow_scala_native_libraries1715329719143383324/libtensorflow_ops.so.
2017-11-03 10:35:30.477 [run-main-0] INFO  TensorFlow Native - Copied 48536 bytes to /tmp/tensorflow_scala_native_libraries1715329719143383324/libtensorflow_ops.so.
[error] (run-main-0) java.lang.ExceptionInInitializerError
java.lang.ExceptionInInitializerError
    at org.platanios.tensorflow.jni.Tensor$.<init>(Tensor.scala:24)
    at org.platanios.tensorflow.jni.Tensor$.<clinit>(Tensor.scala)
    at org.platanios.tensorflow.api.tensors.Context$.apply(Context.scala:50)
    at org.platanios.tensorflow.api.package$.<init>(package.scala:47)
    at org.platanios.tensorflow.api.package$.<clinit>(package.scala)
    at example.Hello$.delayedEndpoint$example$Hello$1(Hello.scala:9)
    at example.Hello$delayedInit$body.apply(Hello.scala:6)
    at scala.Function0.apply$mcV$sp(Function0.scala:34)
    at scala.Function0.apply$mcV$sp$(Function0.scala:34)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
    at scala.App.$anonfun$main$1$adapted(App.scala:76)
    at scala.collection.immutable.List.foreach(List.scala:389)
    at scala.App.main(App.scala:76)
    at scala.App.main$(App.scala:74)
    at example.Hello$.main(Hello.scala:6)
    at example.Hello.main(Hello.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
Caused by: org.platanios.tensorflow.jni.NotFoundException: /tmp/tensorflow_scala_native_libraries1715329719143383324/libtensorflow_ops.so: undefined symbol: _ZN10tensorflow7strings8internal9CatPiecesESt16initializer_listINS_11StringPieceEE
    at org.platanios.tensorflow.jni.TensorFlow$.loadOpLibrary(Native Method)
    at org.platanios.tensorflow.jni.TensorFlow$.$anonfun$load$8(TensorFlow.scala:138)
    at scala.Option.foreach(Option.scala:257)
    at org.platanios.tensorflow.jni.TensorFlow$.load(TensorFlow.scala:138)
    at org.platanios.tensorflow.jni.TensorFlow$.<init>(TensorFlow.scala:186)
    at org.platanios.tensorflow.jni.TensorFlow$.<clinit>(TensorFlow.scala)
    at org.platanios.tensorflow.jni.Tensor$.<init>(Tensor.scala:24)
    at org.platanios.tensorflow.jni.Tensor$.<clinit>(Tensor.scala)
    at org.platanios.tensorflow.api.tensors.Context$.apply(Context.scala:50)
    at org.platanios.tensorflow.api.package$.<init>(package.scala:47)
    at org.platanios.tensorflow.api.package$.<clinit>(package.scala)
    at example.Hello$.delayedEndpoint$example$Hello$1(Hello.scala:9)
    at example.Hello$delayedInit$body.apply(Hello.scala:6)
    at scala.Function0.apply$mcV$sp(Function0.scala:34)
    at scala.Function0.apply$mcV$sp$(Function0.scala:34)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
    at scala.App.$anonfun$main$1$adapted(App.scala:76)
    at scala.collection.immutable.List.foreach(List.scala:389)
    at scala.App.main(App.scala:76)
    at scala.App.main$(App.scala:74)
    at example.Hello$.main(Hello.scala:6)
    at example.Hello.main(Hello.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
[trace] Stack trace suppressed: run last compile:run for the full output.
java.lang.RuntimeException: Nonzero exit code: 1
    at scala.sys.package$.error(package.scala:27)
[trace] Stack trace suppressed: run last compile:run for the full output.
[error] (compile:run) Nonzero exit code: 1
[error] Total time: 0 s, completed 3 Nov, 2017 10:35:30 AM

I checked to see that Extracting the 'tensorflow_jni' native library to /tmp/tensorflow_scala_native_libraries1715329719143383324/libtensorflow_jni.so did not actually happen, i.e. there is no sub-directory in /tmp with the above (or any similar name).

I am using Ubuntu 16.04. I have checked that Tensorflow works with Python.

Any help is appreciated

eaplatanios commented 6 years ago

@siddhartha-gadgil The libraries are extracted fine but are deleted once the program terminates. That's why you don't see them in the tmp directory. The problem in your case has to do with which version of the compiled TensorFlow binaries you're working with. Could you try setting the following dependency in your build.sbt file?

libraryDependencies += "org.platanios" %% "tensorflow" % "0.1.0-SNAPSHOT" classifier "linux-cpu-x86_64"

This will pull the pre-compiled binaries I distribute with my releases. Otherwise, not that for now, if you want to provide your own binaries, you should compile TensorFlow yourself using master branch from the official TensorFlow repository, or you should download the nightly versions of their pre-compiled binaries.

Please let me know if this helps. :)

siddhartha-gadgil commented 6 years ago

Thanks. I have tried various combinations, and get slightly different errors. I have installed tensorflow from source and followed your instructions. If I include the bezel generated binary in the LD_LIBRARY_PATH, I get

java.lang.ExceptionInInitializerError
at org.platanios.tensorflow.jni.Tensor$.<init>(Tensor.scala:24)
at org.platanios.tensorflow.jni.Tensor$.<clinit>(Tensor.scala)
at org.platanios.tensorflow.api.tensors.Context$.apply(Context.scala:50)
at org.platanios.tensorflow.api.package$.<init>(package.scala:47)
at org.platanios.tensorflow.api.package$.<clinit>(package.scala)
at example.Hello$.delayedEndpoint$example$Hello$1(Hello.scala:9)
at example.Hello$delayedInit$body.apply(Hello.scala:6)
at scala.Function0.apply$mcV$sp(Function0.scala:34)
at scala.Function0.apply$mcV$sp$(Function0.scala:34)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App.$anonfun$main$1$adapted(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:389)
at scala.App.main(App.scala:76)
at scala.App.main$(App.scala:74)
at example.Hello$.main(Hello.scala:6)
at example.Hello.main(Hello.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
Caused by: org.platanios.tensorflow.jni.NotFoundException:
/tmp/tensorflow_scala_native_libraries5400773787127516037/libtensorflow_ops.so:
undefined symbol:
_ZN10tensorflow7strings8internal9CatPiecesESt16initializer_listINS_11StringPieceEE
at org.platanios.tensorflow.jni.TensorFlow$.loadOpLibrary(Native Method)
at
org.platanios.tensorflow.jni.TensorFlow$.$anonfun$load$8(TensorFlow.scala:138)
at scala.Option.foreach(Option.scala:257)
at org.platanios.tensorflow.jni.TensorFlow$.load(TensorFlow.scala:138)
at org.platanios.tensorflow.jni.TensorFlow$.<init>(TensorFlow.scala:186)
at org.platanios.tensorflow.jni.TensorFlow$.<clinit>(TensorFlow.scala)
at org.platanios.tensorflow.jni.Tensor$.<init>(Tensor.scala:24)
at org.platanios.tensorflow.jni.Tensor$.<clinit>(Tensor.scala)
at org.platanios.tensorflow.api.tensors.Context$.apply(Context.scala:50)
at org.platanios.tensorflow.api.package$.<init>(package.scala:47)
at org.platanios.tensorflow.api.package$.<clinit>(package.scala)
at example.Hello$.delayedEndpoint$example$Hello$1(Hello.scala:9)
at example.Hello$delayedInit$body.apply(Hello.scala:6)
at scala.Function0.apply$mcV$sp(Function0.scala:34)
at scala.Function0.apply$mcV$sp$(Function0.scala:34)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App.$anonfun$main$1$adapted(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:389)
at scala.App.main(App.scala:76)
at scala.App.main$(App.scala:74)
at example.Hello$.main(Hello.scala:6)
at example.Hello.main(Hello.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)

On the other hand, if I use the classifiers and exclude it from the path, I get

2017-11-03 14:08:07.071 [run-main-0] INFO  TensorFlow Native - Extracting
the 'tensorflow_framework' native library to
/tmp/tensorflow_scala_native_libraries5867038591854807869/libtensorflow_framework.so.
2017-11-03 14:08:07.152 [run-main-0] INFO  TensorFlow Native - Copied
15349648 bytes to
/tmp/tensorflow_scala_native_libraries5867038591854807869/libtensorflow_framework.so.
2017-11-03 14:08:07.165 [run-main-0] INFO  TensorFlow Native - Extracting
the 'tensorflow' native library to
/tmp/tensorflow_scala_native_libraries5867038591854807869/libtensorflow.so.
2017-11-03 14:08:07.309 [run-main-0] INFO  TensorFlow Native - Copied
40633448 bytes to
/tmp/tensorflow_scala_native_libraries5867038591854807869/libtensorflow.so.
2017-11-03 14:08:07.329 [run-main-0] INFO  TensorFlow Native - Extracting
the 'tensorflow_jni' native library to
/tmp/tensorflow_scala_native_libraries5867038591854807869/libtensorflow_jni.so.
2017-11-03 14:08:07.332 [run-main-0] INFO  TensorFlow Native - Copied
628432 bytes to
/tmp/tensorflow_scala_native_libraries5867038591854807869/libtensorflow_jni.so.
[error] (run-main-0) java.lang.UnsatisfiedLinkError:
/tmp/tensorflow_scala_native_libraries5867038591854807869/libtensorflow_jni.so:
libtensorflow.so: cannot open shared object file: No such file or directory
java.lang.UnsatisfiedLinkError:
/tmp/tensorflow_scala_native_libraries5867038591854807869/libtensorflow_jni.so:
libtensorflow.so: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at
org.platanios.tensorflow.jni.TensorFlow$.$anonfun$load$6(TensorFlow.scala:126)
at
org.platanios.tensorflow.jni.TensorFlow$.$anonfun$load$6$adapted(TensorFlow.scala:124)
at scala.Option.foreach(Option.scala:257)
at org.platanios.tensorflow.jni.TensorFlow$.load(TensorFlow.scala:124)
at org.platanios.tensorflow.jni.TensorFlow$.<init>(TensorFlow.scala:186)
at org.platanios.tensorflow.jni.TensorFlow$.<clinit>(TensorFlow.scala)
at org.platanios.tensorflow.jni.Tensor$.<init>(Tensor.scala:24)
at org.platanios.tensorflow.jni.Tensor$.<clinit>(Tensor.scala)
at org.platanios.tensorflow.api.tensors.Context$.apply(Context.scala:50)
at org.platanios.tensorflow.api.package$.<init>(package.scala:47)
at org.platanios.tensorflow.api.package$.<clinit>(package.scala)
at example.Hello$.delayedEndpoint$example$Hello$1(Hello.scala:9)
at example.Hello$delayedInit$body.apply(Hello.scala:6)
at scala.Function0.apply$mcV$sp(Function0.scala:34)
at scala.Function0.apply$mcV$sp$(Function0.scala:34)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App.$anonfun$main$1$adapted(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:389)
at scala.App.main(App.scala:76)
at scala.App.main$(App.scala:74)
at example.Hello$.main(Hello.scala:6)
at example.Hello.main(Hello.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
[trace] Stack trace suppressed: run last compile:run for the full output.
java.lang.RuntimeException: Nonzero exit code: 1
at scala.sys.package$.error(package.scala:27)
[trace] Stack trace suppressed: run last compile:run for the full output.
[error] (compile:run) Nonzero exit code: 1
[error] Total time: 0 s, completed 3 Nov, 2017 2:08:07 PM

It is probably mis-configuration at my end, but any help is appreciated. Incidentally, I set up watchers to confirm that the file libtensorflow_jni.so was indeed created.

regards

On Fri, Nov 3, 2017 at 1:48 PM Anthony Platanios notifications@github.com wrote:

@siddhartha-gadgil https://github.com/siddhartha-gadgil The libraries are extracted fine but are deleted once the program terminates. That's why you don't see them in the tmp directory. The problem in your case has to do with which version of the compiled TensorFlow binaries you're working with. Could you try setting the following dependency in your build.sbt file?

libraryDependencies += "org.platanios" %% "tensorflow" % "0.1.0-SNAPSHOT" classifier "linux-cpu-x86_64"

This will pull the pre-compiled binaries I distribute with my releases. Otherwise, not that for now, if you want to provide your own binaries, you should compile TensorFlow yourself using master branch from the official TensorFlow repository, or you should download the nightly versions of their pre-compiled binaries.

Please let me know if this helps. :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/eaplatanios/tensorflow_scala/issues/37#issuecomment-341642944, or mute the thread https://github.com/notifications/unsubscribe-auth/ADatpFqqM-zKqjKR8kOru35byozALg3aks5sysxkgaJpZM4QQqY- .

eaplatanios commented 6 years ago

@siddhartha-gadgil It's not an issue with your configuration. I was able to re-produce the problem and I'm currently looking into how to fix it. I'll update you soon. Thanks for finding that! :)

eaplatanios commented 6 years ago

@siddhartha-gadgil I think the issue has been fixed with commit #fc0428d, but please let me know if you still get the same error.

siddhartha-gadgil commented 6 years ago

I seem to have the same issue. I am trying to work with the source to get better diagnosis (I previously failed because of old protobuf version). I can compile but get the following error in a console session.

Welcome to Scala 2.12.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_101).
Type in expressions for evaluation. Or try :help.

scala> import org.platanios.tensorflow.api._
import org.platanios.tensorflow.api._

scala> import tensorflow._
import tensorflow._

scala> val tensor = Tensor.zeros(INT32, Shape(2, 5))
2017-11-06 08:45:26.349 [run-main-0] INFO  TensorFlow Native - Extracting
the 'tensorflow_jni' native library to
/tmp/tensorflow_scala_native_libraries690607158483966877/libtensorflow_jni.so.
2017-11-06 08:45:26.353 [run-main-0] INFO  TensorFlow Native - Copied
607880 bytes to
/tmp/tensorflow_scala_native_libraries690607158483966877/libtensorflow_jni.so.
2017-11-06 08:45:26.385 [run-main-0] INFO  TensorFlow Native - Extracting
the 'tensorflow_ops' native library to
/tmp/tensorflow_scala_native_libraries690607158483966877/libtensorflow_ops.so.
2017-11-06 08:45:26.385 [run-main-0] INFO  TensorFlow Native - Copied 51128
bytes to
/tmp/tensorflow_scala_native_libraries690607158483966877/libtensorflow_ops.so.
org.platanios.tensorflow.jni.NotFoundException:
/tmp/tensorflow_scala_native_libraries690607158483966877/libtensorflow_ops.so:
undefined symbol:
_ZN10tensorflow7strings8internal9CatPiecesESt16initializer_listINS_11StringPieceEE
  at org.platanios.tensorflow.jni.TensorFlow$.loadOpLibrary(Native Method)
  at
org.platanios.tensorflow.jni.TensorFlow$.$anonfun$load$6(TensorFlow.scala:107)
  at scala.Option.foreach(Option.scala:257)
  at org.platanios.tensorflow.jni.TensorFlow$.load(TensorFlow.scala:107)
  at org.platanios.tensorflow.jni.TensorFlow$.<init>(TensorFlow.scala:155)
  at org.platanios.tensorflow.jni.TensorFlow$.<clinit>(TensorFlow.scala)
  at org.platanios.tensorflow.jni.Tensor$.<init>(Tensor.scala:24)
  at org.platanios.tensorflow.jni.Tensor$.<clinit>(Tensor.scala)
  at org.platanios.tensorflow.api.tensors.Context$.apply(Context.scala:50)
  at org.platanios.tensorflow.api.package$.<init>(package.scala:47)
  at org.platanios.tensorflow.api.package$.<clinit>(package.scala)
  ... 36 elided

I have built the latest tensorflow from source. Is this an API change there?

Thanks

On Sun, Nov 5, 2017 at 2:01 PM Anthony Platanios notifications@github.com wrote:

@siddhartha-gadgil https://github.com/siddhartha-gadgil I think the issue has been fixed with commit #fc0428d, but please let me know if you still get the same error.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/eaplatanios/tensorflow_scala/issues/37#issuecomment-341957011, or mute the thread https://github.com/notifications/unsubscribe-auth/ADatpI3CERyk8643PHt8stc6LQFHxX9Lks5szXJ9gaJpZM4QQqY- .

eaplatanios commented 6 years ago

@siddhartha-gadgil As a first step I would suggest you remove your TensorFlow installation (the so files) and use this dependency on a test project:

libraryDependencies += "org.platanios" %% "tensorflow" % "0.1.0-SNAPSHOT" classifier "linux-cpu-x86_64"

You should also clear your Ivy cache before so you get the latest version of my API. You can do this (usually) by deleting the ~/.ivy2/cache/org.platanios directory.

Could you please tell me if this works? And then we can see what version of the TensorFlow source you should use.

siddhartha-gadgil commented 6 years ago

Works :) Thanks a lot. I am happy using the classifier since for now I am just experimenting.

eaplatanios commented 6 years ago

@siddhartha-gadgil That's great! :) So, whenever you get the chance, you could try compiling with the master branch of TensorFlow repository. That should normally work. If it doesn't, I'd more than happy to help figure out what's wrong. :)

siddhartha-gadgil commented 6 years ago

I had compiled with the master branch(CPU only) and got the issue reported above. It seems the same as https://github.com/tensorflow/fold/issues/57

eaplatanios commented 6 years ago

Given that I recently made changes to how the libraries are loaded. You could try again now that you cleared your Ivy cache and it might work. I'm actually curious.

siddhartha-gadgil commented 6 years ago

The error is back. In case this is relevant, while compliling in sbt (after clean) I get the warning

[info] Resolving org.platanios#tensorflow_2.12;0.1.0-SNAPSHOT ...
[warn] Unable to reparse org.platanios#tensorflow_2.12;0.1.0-SNAPSHOT from sonatype-snapshots, using Mon Nov 06 05:13:23 IST 2017
[info] Resolving org.platanios#tensorflow-jni_2.12;0.1.0-SNAPSHOT ...
[warn] Unable to reparse org.platanios#tensorflow-jni_2.12;0.1.0-SNAPSHOT from sonatype-snapshots, using Mon Nov 06 05:11:30 IST 2017
[info] Resolving org.platanios#tensorflow-api_2.12;0.1.0-SNAPSHOT ...
[warn] Unable to reparse org.platanios#tensorflow-api_2.12;0.1.0-SNAPSHOT from sonatype-snapshots, using Mon Nov 06 05:13:55 IST 2017