Closed evanthomas closed 6 years ago
evanthomas
commented
6 years ago Accessing a Tensor is enough
def main(args: Array[String]): Unit = {
val shape = Shape(1, 60, 45, 4)
val N = shape.numElements.asInstanceOf[Int]*FLOAT32.byteSize
val buf = ByteBuffer.allocate(N)
val t = Tensor.fromBuffer(FLOAT32, shape, N, buf)
println(t(0,0,0,0).scalar)
// val u = t(::, ::, ::, 0::3)
}I built locally but it didn't help.
eaplatanios
commented
6 years ago @evanthomas I can't reproduce this on my MacBook. What version of MacOS are you using?
evanthomas
commented
6 years ago 10.13.1
$ uname -a
Darwin mac2949 17.2.0 Darwin Kernel Version 17.2.0: Fri Sep 29 18:27:05 PDT 2017; root:xnu-4570.20.62~3/RELEASE_X86_64 x86_64I am using exactly the same version. When you built locally did you remove the classifier ... part from the dependencies in the build.sbt file?
evanthomas
commented
6 years ago No I didn't and it seemed to be still pulling from sonatype. I've removed the classifier, cleaned org.platanios from my ivy cache, rebuilt, removed sonatype as a resolver in my build.sbt. Now I get:
/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/bin/java "-javaagent:/Users/thomas.e/Applications/IntelliJ IDEA CE.app/Contents/lib/idea_rt.jar=62764:/Users/thomas.e/Applications/IntelliJ IDEA CE.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/lib/tools.jar:/Users/thomas.e/neurosim/ImageAI-server/target/scala-2.12/classes:/Users/thomas.e/.ivy2/cache/ar.com.hjg/pngj/jars/pngj-2.1.0.jar:/Users/thomas.e/.ivy2/local/org.platanios/tensorflow_2.12/0.1.0-SNAPSHOT/jars/tensorflow_2.12.jar:/Users/thomas.e/.ivy2/cache/org.typelevel/spire_2.12/jars/spire_2.12-0.14.1.jar:/Users/thomas.e/.ivy2/cache/org.typelevel/spire-macros_2.12/jars/spire-macros_2.12-0.14.1.jar:/Users/thomas.e/.ivy2/cache/org.typelevel/macro-compat_2.12/jars/macro-compat_2.12-1.1.1.jar:/Users/thomas.e/.ivy2/cache/org.typelevel/machinist_2.12/jars/machinist_2.12-0.6.1.jar:/Users/thomas.e/.ivy2/cache/org.typelevel/cats-macros_2.12/jars/cats-macros_2.12-0.9.0.jar:/Users/thomas.e/.ivy2/cache/org.typelevel/cats-kernel_2.12/jars/cats-kernel_2.12-0.9.0.jar:/Users/thomas.e/.ivy2/cache/org.typelevel/cats-core_2.12/jars/cats-core_2.12-0.9.0.jar:/Users/thomas.e/.ivy2/cache/org.typelevel/algebra_2.12/jars/algebra_2.12-0.7.0.jar:/Users/thomas.e/.ivy2/cache/org.tensorflow/tensorflow/jars/tensorflow-1.4.0.jar:/Users/thomas.e/.ivy2/cache/org.tensorflow/proto/jars/proto-1.4.0.jar:/Users/thomas.e/.ivy2/cache/org.tensorflow/libtensorflow_jni/jars/libtensorflow_jni-1.4.0.jar:/Users/thomas.e/.ivy2/cache/org.tensorflow/libtensorflow/jars/libtensorflow-1.4.0.jar:/Users/thomas.e/.ivy2/cache/org.spire-math/jawn-parser_2.12/jars/jawn-parser_2.12-0.10.4.jar:/Users/thomas.e/.ivy2/cache/org.slf4j/slf4j-api/jars/slf4j-api-1.7.25.jar:/Users/thomas.e/.ivy2/cache/org.scalactic/scalactic_2.12/bundles/scalactic_2.12-3.0.4.jar:/Users/thomas.e/.ivy2/cache/org.scala-lang/scala-reflect/jars/scala-reflect-2.12.4.jar:/Users/thomas.e/.ivy2/local/org.platanios/tensorflow-jni_2.12/0.1.0-SNAPSHOT/jars/tensorflow-jni_2.12.jar:/Users/thomas.e/.ivy2/local/org.platanios/tensorflow-examples_2.12/0.1.0-SNAPSHOT/jars/tensorflow-examples_2.12.jar:/Users/thomas.e/.ivy2/local/org.platanios/tensorflow-data_2.12/0.1.0-SNAPSHOT/jars/tensorflow-data_2.12.jar:/Users/thomas.e/.ivy2/local/org.platanios/tensorflow-api_2.12/0.1.0-SNAPSHOT/jars/tensorflow-api_2.12.jar:/Users/thomas.e/.ivy2/cache/org.objenesis/objenesis/jars/objenesis-2.6.jar:/Users/thomas.e/.ivy2/cache/org.hamcrest/hamcrest-core/jars/hamcrest-core-1.3.jar:/Users/thomas.e/.ivy2/cache/org.apache.commons/commons-compress/jars/commons-compress-1.15.jar:/Users/thomas.e/.ivy2/cache/junit/junit/jars/junit-4.12.jar:/Users/thomas.e/.ivy2/cache/io.circe/circe-parser_2.12/jars/circe-parser_2.12-0.8.0.jar:/Users/thomas.e/.ivy2/cache/io.circe/circe-numbers_2.12/jars/circe-numbers_2.12-0.8.0.jar:/Users/thomas.e/.ivy2/cache/io.circe/circe-jawn_2.12/jars/circe-jawn_2.12-0.8.0.jar:/Users/thomas.e/.ivy2/cache/io.circe/circe-generic_2.12/jars/circe-generic_2.12-0.8.0.jar:/Users/thomas.e/.ivy2/cache/io.circe/circe-core_2.12/jars/circe-core_2.12-0.8.0.jar:/Users/thomas.e/.ivy2/cache/com.typesafe.scala-logging/scala-logging_2.12/bundles/scala-logging_2.12-3.7.2.jar:/Users/thomas.e/.ivy2/cache/com.sksamuel.scrimage/scrimage-io-extra_2.12/jars/scrimage-io-extra_2.12-2.1.8.jar:/Users/thomas.e/.ivy2/cache/com.sksamuel.scrimage/scrimage-filters_2.12/jars/scrimage-filters_2.12-2.1.8.jar:/Users/thomas.e/.ivy2/cache/com.sksamuel.scrimage/scrimage-core_2.12/jars/scrimage-core_2.12-2.1.8.jar:/Users/thomas.e/.ivy2/cache/com.google.protobuf/protobuf-java/bundles/protobuf-java-3.4.0.jar:/Users/thomas.e/.ivy2/cache/com.github.scopt/scopt_2.12/jars/scopt_2.12-3.6.0.jar:/Users/thomas.e/.ivy2/cache/com.github.mpilquist/simulacrum_2.12/jars/simulacrum_2.12-0.10.0.jar:/Users/thomas.e/.ivy2/cache/com.diffplug.matsim/matfilerw/jars/matfilerw-3.0.1.jar:/Users/thomas.e/.ivy2/cache/com.chuusai/shapeless_2.12/bundles/shapeless_2.12-2.3.2.jar:/Users/thomas.e/.ivy2/cache/ch.qos.logback/logback-core/jars/logback-core-1.2.3.jar:/Users/thomas.e/.ivy2/cache/ch.qos.logback/logback-classic/jars/logback-classic-1.2.3.jar:/Users/thomas.e/.ivy2/cache/org.sameersingh.scalaplot/scalaplot/jars/scalaplot-0.1.jar:/Users/thomas.e/.ivy2/cache/jfree/jfreechart/jars/jfreechart-1.0.13.jar:/Users/thomas.e/.ivy2/cache/jfree/jcommon/jars/jcommon-1.0.16.jar:/Users/thomas.e/.ivy2/cache/commons-io/commons-io/jars/commons-io-2.4.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-tiff/jars/imageio-tiff-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-thumbsdb/jars/imageio-thumbsdb-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-tga/jars/imageio-tga-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-sgi/jars/imageio-sgi-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-psd/jars/imageio-psd-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-pnm/jars/imageio-pnm-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-pict/jars/imageio-pict-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-pdf/jars/imageio-pdf-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-pcx/jars/imageio-pcx-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-metadata/jars/imageio-metadata-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-jpeg/jars/imageio-jpeg-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-iff/jars/imageio-iff-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-icns/jars/imageio-icns-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-core/jars/imageio-core-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.imageio/imageio-bmp/jars/imageio-bmp-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.common/common-lang/jars/common-lang-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.common/common-io/jars/common-io-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.twelvemonkeys.common/common-image/jars/common-image-3.2.1.jar:/Users/thomas.e/.ivy2/cache/com.itextpdf.tool/xmlworker/jars/xmlworker-1.1.0.jar:/Users/thomas.e/.ivy2/cache/com.itextpdf/itextpdf/jars/itextpdf-5.1.2.jar:/Users/thomas.e/.ivy2/cache/com.drewnoakes/metadata-extractor/jars/metadata-extractor-2.8.1.jar:/Users/thomas.e/.ivy2/cache/com.adobe.xmp/xmpcore/jars/xmpcore-5.1.2.jar:/Users/thomas.e/.ivy2/cache/args4j/args4j/bundles/args4j-2.33.jar:/Users/thomas.e/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.12.4.jar:/Users/thomas.e/.ivy2/cache/org.scala-lang/scala-reflect/srcs/scala-reflect-2.12.4-sources.jar:/Users/thomas.e/.ivy2/cache/org.scala-lang/scala-library/srcs/scala-library-2.12.4-sources.jar co.mumbler.imageai.server.launch.TensorTest
objc[89923]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/bin/java (0x1009db4c0) and /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/libinstrument.dylib (0x100a674e0). One of the two will be used. Which one is undefined.
2017-11-29 13:06:49.559 [main] INFO TensorFlow Native - Extracting the 'tensorflow_jni' native library to /var/folders/dv/lmmz58g94ys2tvns_r3gym880002w9/T/tensorflow_scala_native_libraries2851696607792942615/libtensorflow_jni.so.
2017-11-29 13:06:49.584 [main] INFO TensorFlow Native - Copied 609908 bytes to /var/folders/dv/lmmz58g94ys2tvns_r3gym880002w9/T/tensorflow_scala_native_libraries2851696607792942615/libtensorflow_jni.so.
Exception in thread "main" java.lang.UnsatisfiedLinkError: /private/var/folders/dv/lmmz58g94ys2tvns_r3gym880002w9/T/tensorflow_scala_native_libraries2851696607792942615/libtensorflow_jni.so: dlopen(/private/var/folders/dv/lmmz58g94ys2tvns_r3gym880002w9/T/tensorflow_scala_native_libraries2851696607792942615/libtensorflow_jni.so, 1): Library not loaded: @rpath/libtensorflow.so
Referenced from: /private/var/folders/dv/lmmz58g94ys2tvns_r3gym880002w9/T/tensorflow_scala_native_libraries2851696607792942615/libtensorflow_jni.so
Reason: image not found
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at org.platanios.tensorflow.jni.TensorFlow$.$anonfun$load$4(TensorFlow.scala:95)
at org.platanios.tensorflow.jni.TensorFlow$.$anonfun$load$4$adapted(TensorFlow.scala:93)
at scala.Option.foreach(Option.scala:257)
at org.platanios.tensorflow.jni.TensorFlow$.load(TensorFlow.scala:93)
at org.platanios.tensorflow.jni.TensorFlow$.<init>(TensorFlow.scala:155)
at org.platanios.tensorflow.jni.TensorFlow$.<clinit>(TensorFlow.scala)
at org.platanios.tensorflow.jni.Tensor$.<init>(Tensor.scala:24)
at org.platanios.tensorflow.jni.Tensor$.<clinit>(Tensor.scala)
at org.platanios.tensorflow.api.tensors.Context$.apply(Context.scala:50)
at org.platanios.tensorflow.api.package$.<init>(package.scala:89)
at org.platanios.tensorflow.api.package$.<clinit>(package.scala)
at co.mumbler.imageai.server.launch.TensorTest$.main(TensorTest.scala:10)
at co.mumbler.imageai.server.launch.TensorTest.main(TensorTest.scala)So looks like I haven't built it properly.
eaplatanios
commented
6 years ago Have you added the directory that contains libtensorflow.so and libtensorflow_framework.so to LD_LIBRARY_PATH?
eaplatanios
commented
6 years ago You could also try copying these two files to /usr/local/lib or /usr/lib and see if it loads successfully then.
evanthomas
commented
6 years ago LD_LIBRARY_PATH was not set. Once it was set it works!
Thanks again!
eaplatanios
commented
6 years ago @evanthomas No problem. I’ll look into updating the published artifacts too. :)
This code reproduces the problem. I haven't tested exactly this code on linux GPU but a similar slice on linux does not throw the exception.
I'm using the latest snapshot from Maven, not my own build. Here is my build.sbt:
Running the code gives:
Here is the JVM diagnostics: