Closed kgoderis closed 6 months ago
Self-note. It seems that the above error is due to me building against a 1.8 JDK, instead of something more recent
Don't try and compile TF-Java using Rosetta, you'll pull in a TF binary which has AVX instructions which will cause a SIGILL and take down the JVM.
I've not tried to compile it on an M1 since we bumped to TF 2.6.0 and made some build changes, I can take a look at doing that. Theoretically you should be able to run mvn package
and have it build everything, but I think you'll need to be in a venv which has a version of numpy installed, and be running bazel natively rather than via Rosetta. After that it comes down to a bunch of weird configuration things in bazel which we might not be patching appropriately.
As for other ML frameworks, I've personally got XGBoost and ONNX Runtime working in Java on an M1 Mac and contributed any fixes back upstream. We had ONNX Runtime working a month or two after the M1 came out. Anything that's in pure Java will work just fine on an M1, but I've not looked at dl4j or djl which both have large native libraries inside.
@Craigacp Building under Rosetta but using a TF build config file without the AVX instructions is not an option then ? I was not aware that it is pulling a TF binary, I was under the impression that it pulls the TF repo and does compile TF as part of the TF-J build process.
@Craigacp Any pointer on how to get ONNX going, because the this what I get on the home page? LOL
[Edit : I presume you did some cross-compiling to get it work . Going through the docs right now...]
@Craigacp Building under Rosetta but using a TF build config file without the AVX instructions is not an option then ? I was not aware that it is pulling a TF binary, I was under the impression that it pulls the TF repo and does compile TF as part of the TF-J build process.
Java is slow under Rosetta as it messes with the JIT. You could compile TF without AVX support under Rosetta, but it would probably be fairly slow, and at that point I'm not sure what the utility of it is.
@Craigacp Any pointer on how to get ONNX going, because the this what I get on the home page? LOL
[Edit : I presume you did some cross-compiling to get it work . Going through the docs right now...]
I've not tried cross-compiling. Checkout the ONNX Runtime repo on a M1 Mac and then compile it as normal for java ./build.sh --update --build --config Release --parallel --build_java --test
.
Java is slow under Rosetta as it messes with the JIT. You could compile TF without AVX support under Rosetta, but it would probably be fairly slow, and at that point I'm not sure what the utility of it is.
Well, my main dev machine is now an M1, obviously. So, the utility lies in developing TF models locally, but consequently then train them on a TPU/x86 cloud-based machine. I just want to avoid any pain in my development process.
In order to compile it under Rosetta I presume that I need an x86 JVM installed on top of other x86 tools like Bazel, right ?
Yes, you'll need a full x86 development stack, including Python, probably including compilers as well, and then you might need to change how it finds the compilers to make sure it picks the x86 ones.
Some people seem to be able to get arm64 binaries for TF 2.6.0, for example, see https://github.com/tensorflow/tensorflow/issues/52160#issuecomment-933987607. If that is true, running the build for TF Java with a command line this should work:
BUILD_FLAGS="--cpu=darwin_arm64 --host-cpu=darwin_arm64" mvn clean install
@saudet That did not work unfortunately.
I am able to compile the Tensorflow repo (https://github.com/tensorflow/tensorflow/issues/52160#issuecomment-968173580), but then Tf-J fails with
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project tensorflow-core-generator: Compilation failure
[ERROR] /Users/kgoderis/Development/tensorflow-java/tensorflow-core/tensorflow-core-generator/src/main/java/org/tensorflow/proto/framework/OpListOrBuilder.java:[23,7] error: An unhandled exception was thrown by the Error Prone static analysis plugin.
[ERROR] Please report this at https://github.com/google/error-prone/issues/new and include the following:
[ERROR]
[ERROR] error-prone version: 2.6.0
[ERROR] BugPattern: JavaLangClash
[ERROR] Stack Trace:
[ERROR] java.lang.NoSuchMethodError: 'java.lang.Iterable com.sun.tools.javac.code.Scope$WriteableScope.getSymbolsByName(com.sun.tools.javac.util.Name, com.sun.tools.javac.util.Filter)'
[ERROR] at com.google.errorprone.bugpatterns.JavaLangClash.check(JavaLangClash.java:66)
[ERROR] at com.google.errorprone.bugpatterns.JavaLangClash.matchClass(JavaLangClash.java:53)
[ERROR] at com.google.errorprone.scanner.ErrorProneScanner.processMatchers(ErrorProneScanner.java:450)
[ERROR] at com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:548)
[ERROR] at com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:151)
[ERROR] at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:860)
[ERROR] at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
[ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:74)
[ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:48)
[ERROR] at jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:111)
[ERROR] at jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:119)
[ERROR] at jdk.compiler/com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:152)
[ERROR] at com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:561)
[ERROR] at com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:151)
[ERROR] at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:614)
[ERROR] at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:60)
[ERROR] at com.google.errorprone.scanner.Scanner.scan(Scanner.java:58)
[ERROR] at com.google.errorprone.scanner.ErrorProneScannerTransformer.apply(ErrorProneScannerTransformer.java:43)
[ERROR] at com.google.errorprone.ErrorProneAnalyzer.finished(ErrorProneAnalyzer.java:152)
[ERROR] at jdk.compiler/com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:132)
[ERROR] at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1394)
[ERROR] at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1341)
[ERROR] at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:933)
[ERROR] at jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:317)
[ERROR] at jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:176)
[ERROR] at jdk.compiler/com.sun.tools.javac.Main.compile(Main.java:64)
[ERROR] at jdk.compiler/com.sun.tools.javac.Main.main(Main.java:50)
[ERROR]
On the other hand, diving into the ./tensorflow-core/tensorflow-core-api where the TF core should be built, I was able to start the compilation (sudo bazel build --config opt --cpu=darwin_arm64 --host_cpu=darwin_arm64 --incompatible_restrict_string_escapes=false --experimental_repo_remote_exec --define=ABSOLUTE_JAVABASE=/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home --host_javabase=@bazel_tools//tools/jdk:absolute_javabase //:all
), but is soon exited with errors like these:
external/org_tensorflow/tensorflow/core/platform/default/port.cc:360:14: error: no matching constructor for initialization of 'tensorflow::port::MemoryInfo'
MemoryInfo mem_info = {INT64_MAX, INT64_MAX};
^ ~~~~~~~~~~~~~~~~~~~~~~
external/org_tensorflow/tensorflow/core/platform/mem.h:62:8: note: candidate constructor (the implicit copy constructor) not viable: requires 1 argument, but 2 were provided
struct MemoryInfo {
^
external/org_tensorflow/tensorflow/core/platform/mem.h:62:8: note: candidate constructor (the implicit move constructor) not viable: requires 1 argument, but 2 were provided
external/org_tensorflow/tensorflow/core/platform/mem.h:62:8: note: candidate constructor (the implicit default constructor) not viable: requires 0 arguments, but 2 were provided
external/org_tensorflow/tensorflow/core/platform/default/port.cc:373:23: error: no matching constructor for initialization of 'tensorflow::port::MemoryBandwidthInfo'
MemoryBandwidthInfo membw_info = {INT64_MAX};
^ ~~~~~~~~~~~
external/org_tensorflow/tensorflow/core/platform/mem.h:67:8: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'long long' to 'const tensorflow::port::MemoryBandwidthInfo' for 1st argument
struct MemoryBandwidthInfo {
^
external/org_tensorflow/tensorflow/core/platform/mem.h:67:8: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'long long' to 'tensorflow::port::MemoryBandwidthInfo' for 1st argument
external/org_tensorflow/tensorflow/core/platform/mem.h:67:8: note: candidate constructor (the implicit default constructor) not viable: requires 0 arguments, but 1 was provided
Update : Bumping Google's errorprone to
Update: Changing .bazelrc in tensorflow-core/tensorflow-core-api to
build --remote_cache=https://storage.googleapis.com/tensorflow-sigs-jvm
build --remote_upload_local_results=false
build --action_env PYTHON_BIN_PATH="/Users/kgoderis/miniforge3/bin/python3"
build --action_env PYTHON_LIB_PATH="/Users/kgoderis/miniforge3/lib/python3.9/site-packages"
build --python_path="/Users/kgoderis/miniforge3/bin/python3"
build:opt --copt=-Wno-sign-compare
build:opt --host_copt=-Wno-sign-compare
test --flaky_test_attempts=3
test --test_size_filters=small,medium
test:v1 --test_tag_filters=-benchmark-test,-no_oss,-gpu,-nomac,-no_mac,-oss_serial
test:v1 --build_tag_filters=-benchmark-test,-no_oss,-gpu,-nomac,-no_mac
test:v2 --test_tag_filters=-benchmark-test,-no_oss,-gpu,-nomac,-no_mac,-oss_serial,-v1only
test:v2 --build_tag_filters=-benchmark-test,-no_oss,-gpu,-nomac,-no_mac,-v1only
build --incompatible_restrict_string_escapes=false
build --experimental_repo_remote_exec
build --define=ABSOLUTE_JAVABASE=/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home
build --host_javabase=@bazel_tools//tools/jdk:absolute_javabase
In addition I bumped .bazelversion to 4.2.1 and changed build.sh to make bazel run under sudo
gets the compilation of that maven compile unit going. Still contains references to the setup on my dev machine, but we are advancing ;-)
Some of you will be happy. I got the whole thing compiled, however I had to skip tests as it was failing on that part, and there were some warnings on TARGET_OS_IPHONE. Apart from that, it kinda looks good:
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for TensorFlow Java Parent 0.4.0-SNAPSHOT:
[INFO]
[INFO] TensorFlow Java Parent ............................. SUCCESS [ 0.476 s]
[INFO] TensorFlow Core Parent ............................. SUCCESS [ 0.009 s]
[INFO] TensorFlow Core Generators ......................... SUCCESS [ 0.277 s]
[INFO] TensorFlow Core API Library ........................ SUCCESS [ 48.684 s]
[INFO] TensorFlow Core API Library Platform ............... SUCCESS [ 0.020 s]
[INFO] TensorFlow Framework Library ....................... SUCCESS [ 0.069 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 49.599 s
[INFO] Finished at: 2021-11-14T13:39:12+01:00
[INFO] ------------------------------------------------------------------------
(base) kgoderis@Karels-M1-MacBook-Pro target % pwd
/Users/kgoderis/Development/tensorflow-java/tensorflow-core/tensorflow-core-api/target
(base) kgoderis@Karels-M1-MacBook-Pro target % ls -la
total 180432
drwxr-xr-x 12 root staff 384 Nov 14 13:19 .
drwxr-xr-x 17 kgoderis staff 544 Nov 14 13:16 ..
drwxr-xr-x 3 root staff 96 Nov 14 13:16 classes
drwxr-xr-x 3 root staff 96 Nov 14 13:16 generated-sources
drwxr-xr-x 3 root staff 96 Nov 14 13:16 generated-test-sources
drwxr-xr-x 3 root staff 96 Nov 14 13:17 maven-archiver
drwxr-xr-x 3 root staff 96 Nov 14 13:16 maven-status
drwxr-xr-x 3 root staff 96 Nov 14 13:16 native
drwxr-xr-x 82 root staff 2624 Nov 14 13:19 surefire-reports
-rw-r--r-- 1 root staff 78551487 Nov 14 13:32 tensorflow-core-api-0.4.0-SNAPSHOT-macosx-arm64.jar
-rw-r--r-- 1 root staff 8245523 Nov 14 13:32 tensorflow-core-api-0.4.0-SNAPSHOT.jar
drwxr-xr-x 5 root staff 160 Nov 14 13:17 test-classes
(base) kgoderis@Karels-M1-MacBook-Pro tensorflow % pwd
/Users/kgoderis/Development/tensorflow-java/tensorflow-core/tensorflow-core-api/bazel-bin/external/org_tensorflow/tensorflow
(base) kgoderis@Karels-M1-MacBook-Pro tensorflow % file libtensorflow_framework.2.6.0.dylib
libtensorflow_framework.2.6.0.dylib: Mach-O 64-bit dynamically linked shared library arm64
The build fails with Java 8 (arm64)
ERROR: /private/var/tmp/_bazel_root/6712ec151cb8fc337cc5082ff0f496e3/external/bazel_tools/tools/jdk/BUILD:346:14: Action external/bazel_tools/tools/jdk/platformclasspath.jar failed: (Exit 1): java failed: error executing command
(cd /private/var/tmp/_bazel_root/6712ec151cb8fc337cc5082ff0f496e3/execroot/tensorflow_core_api && \
exec env - \
/Library/Java/JavaVirtualMachines/zulu-8-arm64.jdk/Contents/Home/bin/java -XX:+IgnoreUnrecognizedVMOptions '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.platform=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED' -cp bazel-out/darwin_arm64-opt/bin/external/bazel_tools/tools/jdk/platformclasspath_classes:/Library/Java/JavaVirtualMachines/zulu-8-arm64.jdk/Contents/Home/lib/tools.jar DumpPlatformClassPath bazel-out/darwin_arm64-opt/bin/external/bazel_tools/tools/jdk/platformclasspath.jar external/local_jdk)
Execution platform: @local_execution_config_platform//:platform
Exception in thread "main" java.lang.AssertionError:
Could not find java.lang.Object on bootclasspath; something has gone terribly wrong.
Please file a bug: https://github.com/bazelbuild/bazel/issues
at DumpPlatformClassPath.writeEntries(DumpPlatformClassPath.java:136)
at DumpPlatformClassPath.writeClassPathJars(DumpPlatformClassPath.java:174)
at DumpPlatformClassPath.dumpJDK8BootClassPath(DumpPlatformClassPath.java:77)
at DumpPlatformClassPath.main(DumpPlatformClassPath.java:65)
ERROR: /private/var/tmp/_bazel_root/6712ec151cb8fc337cc5082ff0f496e3/external/com_google_protobuf/BUILD:290:15 Building external/com_google_protobuf/libany_proto-speed.jar (1 source jar) failed: (Exit 1): java failed: error executing command
[Update] It seems one has to be explicit about the toolchain in .bazelrc. I added
build --host_javabase=@bazel_tools//tools/jdk:absolute_javabase
build --javabase=@bazel_tools//tools/jdk:absolute_javabase
build --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8
build --java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8
but unfortunately it fails because in pom.xml we use --add-exports flags for the JVM, which is not supported by the 1.8 JDK. I am not sure why we need these flags in the first place ( I know what the flag is supposed to do), and therefore, what could be a workaround solution. Anyone?
If it builds with 11 why do you need to build it with 8? It should produce Java 8 compatible jar files even when compiled on 11.
If it builds with 11 why do you need to build it with 8? It should produce Java 8 compatible jar files even when compiled on 11.
Because I want to integrate this in a project which uses Spark NLP, and that only runs on a Java 8 VM. As far as I understand, Java 11 compiled jars do not run ok older JVMs
TF-Java is compiled on 11 but targets 8, and so will produce class files which are compatible with Java 8.
TF-Java is compiled on 11 but targets 8, and so will produce class files which are compatible with Java 8.
Ah... I was not aware of this. That means we are good to go. Will you pick up what we did and get the jars onto sonatype?
I'm trying to replicate what you have on my M1 Mac so I can figure out what the test failures are, but I'm getting issues compiling protobuf.
We can't easily deploy to Maven Central as our builds are done through Github Actions and they don't have any Apple Silicon runners.
@Craigacp I think I solved that by altering .bazelrc cfr https://github.com/tensorflow/java/issues/394#issuecomment-968668060
or this
build --define=ABSOLUTE_JAVABASE=/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home build --host_javabase=@bazel_tools//tools/jdk:absolute_javabase
Not sure in fact, I did many things and document only half of it
I had to make some modifications to the pom files to get it to build the appropriate jars, and having to run the build as the superuser is worrying to me. I also had to add my JVM as a bazel build flag too. However I didn't need to do anything else to my .bazelrc
other than build --incompatible_restrict_string_escapes=false
. My build does now pass all the tests with those modifications.
We're upgrading to TF 2.7.0 at the moment, I'll rerun the build after that has merged in as it might fix some of the issues.
If I remember well the tests failed on a mismatch of dimension on the input matrix on a NN layer. Mind you that I tried to compile against Java 8 cfr my misunderstanding.
Could you try and build a clean checkout of this branch - https://github.com/Craigacp/tensorflow-java/tree/apple-silicon ? It'll require sudo, and I don't want that in an actual build, but it would be a useful check if someone else can build it.
@Craigacp Trying to do so. However, need Google Error Prone bumped to 2.10.0, and what about Bazel? 3.7.2 or 4.2.1 ?
In fact, I remember I went for Bazel 4.2.1 because there are no pre-compiled Bazel binaries For MacOS arm64, and I wanted to avoid to Compile Bazel from source
Error prone should work on Java 11. That build is hard coded to expect a Azul Zulu 11 installed in the system. The bazel version is set to 4.2.1 and the whole thing should build with mvn clean package
without other modifications, the same way the x86 builds do.
Are you still trying to use Bazel 4.2.1
? Because that's what the error is complaining about. We require 3.7.2
because tensorflow does.
I set the bazelversion to 4.2.1. Maybe it's not cleaned the build properly?
It does not work
Are you still trying to use Bazel
4.2.1
? Because that's what the error is complaining about. We require3.7.2
because tensorflow does.
Yes, but I got the whole thing compiled with 4.2.1 last weekend
I had to make some modifications to the pom files to get it to build the appropriate jars,
Ah, yes, we'll need to update the profiles in the pom.xml files a bit like pull https://github.com/bytedeco/javacpp-presets/pull/1092 for this to work. Are you saying you've already done this? Or should I do it?
I had to make some modifications to the pom files to get it to build the appropriate jars,
Ah, yes, we'll need to update the profiles in the pom.xml files a bit like pull bytedeco/javacpp-presets#1092 for this to work. Are you saying you've already done this? Or should I do it?
I have already done it, but it'll need a little more work before it gets merged to make it conditional as we won't have builds on Maven Central.
It does not work
- Zulu 11 + Error prone 2.6.0 -> error
- still needs sudo
- Bazel 4.2.1 needed as no arm binaries for 3.7.2 are available
Please check your environment. Error prone 2.6.0 works with Java 11 but fails with Java 17. I know it still needs sudo, but the sudo is now inside the build script so it's a little minimized. The bazelversion is set to 4.2.1 so it should pull that in automatically.
@Craigacp I am not sure what I am missing here:
(base) kgoderis@Karels-M1-MacBook-Pro craig % java -version
openjdk version **"11.0.13"** 2021-10-19 LTS
OpenJDK Runtime Environment Zulu11.52+13-CA (build 11.0.13+8-LTS)
OpenJDK 64-Bit Server VM Zulu11.52+13-CA (build 11.0.13+8-LTS, mixed mode)
(base) kgoderis@Karels-M1-MacBook-Pro craig % sudo mvn clean package
Password:
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] TensorFlow Java Parent [pom]
[INFO] TensorFlow Core Parent [pom]
[INFO] TensorFlow Core Generators [jar]
[INFO] TensorFlow Core API Library [jar]
[INFO] TensorFlow Core API Library Platform [jar]
[INFO] TensorFlow Framework Library [jar]
[INFO]
[INFO] -------------------< org.tensorflow:tensorflow-java >-------------------
[INFO] Building TensorFlow Java Parent 0.4.0-SNAPSHOT [1/6]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tensorflow-java ---
[INFO]
[INFO] --- maven-source-plugin:3.2.1:jar-no-fork (attach-sources) @ tensorflow-java ---
[INFO] Skipping source per configuration.
[INFO]
[INFO] --- maven-javadoc-plugin:3.2.0:jar (attach-javadocs) @ tensorflow-java ---
[INFO] Skipping javadoc generation
[INFO]
[INFO] -------------------< org.tensorflow:tensorflow-core >-------------------
[INFO] Building TensorFlow Core Parent 0.4.0-SNAPSHOT [2/6]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tensorflow-core ---
[INFO]
[INFO] --- maven-source-plugin:3.2.1:jar-no-fork (attach-sources) @ tensorflow-core ---
[INFO] Skipping source per configuration.
[INFO]
[INFO] --- maven-javadoc-plugin:3.2.0:jar (attach-javadocs) @ tensorflow-core ---
[INFO] Skipping javadoc generation
[INFO]
[INFO] --------------< org.tensorflow:tensorflow-core-generator >--------------
[INFO] Building TensorFlow Core Generators 0.4.0-SNAPSHOT [3/6]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tensorflow-core-generator ---
[INFO] Deleting /Users/kgoderis/Development/craig/tensorflow-core/tensorflow-core-generator/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ tensorflow-core-generator ---
[INFO] Using 'UTF8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/kgoderis/Development/craig/tensorflow-core/tensorflow-core-generator/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ tensorflow-core-generator ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 43 source files to /Users/kgoderis/Development/craig/tensorflow-core/tensorflow-core-generator/target/classes
[INFO] -------------------------------------------------------------
[WARNING] COMPILATION WARNING :
[INFO] -------------------------------------------------------------
[WARNING] [options] bootstrap class path not set in conjunction with -source 8
[INFO] 1 warning
[INFO] -------------------------------------------------------------
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/kgoderis/Development/craig/tensorflow-core/tensorflow-core-generator/src/main/java/org/tensorflow/op/generator/javadoc/JavaDocRenderer.java:[47,7] error: An unhandled exception was thrown by the Error Prone static analysis plugin.
Please report this at https://github.com/google/error-prone/issues/new and include the following:
error-prone version: 2.6.0
BugPattern: JavaLangClash
Stack Trace:
java.lang.NoSuchMethodError: 'java.lang.Iterable com.sun.tools.javac.code.Scope$WriteableScope.getSymbolsByName(com.sun.tools.javac.util.Name, com.sun.tools.javac.util.Filter)'
at com.google.errorprone.bugpatterns.JavaLangClash.check(JavaLangClash.java:66)
at com.google.errorprone.bugpatterns.JavaLangClash.matchClass(JavaLangClash.java:53)
at com.google.errorprone.scanner.ErrorProneScanner.processMatchers(ErrorProneScanner.java:450)
at com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:548)
at com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:151)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:860)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86)
at com.google.errorprone.scanner.Scanner.scan(Scanner.java:74)
at com.google.errorprone.scanner.Scanner.scan(Scanner.java:48)
at jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:111)
at jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:119)
at jdk.compiler/com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:152)
at com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:561)
at com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:151)
at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:614)
at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:60)
at com.google.errorprone.scanner.Scanner.scan(Scanner.java:58)
at com.google.errorprone.scanner.ErrorProneScannerTransformer.apply(ErrorProneScannerTransformer.java:43)
at com.google.errorprone.ErrorProneAnalyzer.finished(ErrorProneAnalyzer.java:152)
at jdk.compiler/com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:132)
at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1394)
at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1341)
at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:933)
at jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:317)
at jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:176)
at jdk.compiler/com.sun.tools.javac.Main.compile(Main.java:64)
at jdk.compiler/com.sun.tools.javac.Main.main(Main.java:50)
[INFO] 1 error
Hi everyone,
I am also currently trying to build the module on my Apple silicon machine (0.4.0-SNAPSHOT with Zulu SDK 11 on macOS Monterey).
I was able to compile it (however quite messily and independently from @Craigacp's changes) and for me it passes the tests.
Checking out Adam's fork and compiling it works for me too. @kgoderis I'm sorry that I can't help you, I had completely different issues when trying to build it.
It has many warnings though. The following types of warnings were logged:
FloatingPointLiteralPrecision
ImmutableEnumChecker
InvalidParam
MissingOverride
MissingSummary
UnusedVariable
cast
options
rawtypes
serial
unchecked
When inspecting the jars, it was missing some dependencies like org.tensorflow.ndarray and com.google.protobuf (potentially more). This applies to the jars resulting from Adams fork as well.
My question is: do you think that this will be a problem going forward? Or might this be another setup environment issue?
For the missing org.tensorflow.ndarray I had to compile it on my machine and add it manually. I believe com.google.protobuf might not have jars for apple silicon either for version < 3.18, so might need to be compiled locally as well. I haven't checked which other dependencies are missing.
The output log of my build (after using pre-compiled caches due to a maven issue):
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[1347,77] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[1393,31] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[1414,74] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[1524,47] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[1541,47] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[1583,83] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[1880,86] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[1915,88] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[1930,74] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[1950,78] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2112,83] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2125,95] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2127,94] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2129,90] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2142,37] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2142,78] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2145,37] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2148,37] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2151,37] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2221,81] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2418,39] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2419,71] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2474,39] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2475,71] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2476,61] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2477,31] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2595,80] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2631,76] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2699,55] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2700,51] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2701,39] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2726,39] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2727,26] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2755,55] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2756,51] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2757,39] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2787,58] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2809,56] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2809,105] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2810,69] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2811,55] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2812,56] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2838,62] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2839,62] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2840,62] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2841,76] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2864,57] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2864,106] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2865,70] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2866,56] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[2867,57] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[3265,60] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[3608,93] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[4388,74] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[4538,78] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[4648,84] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[4670,77] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable: P extends Pointer declared in class PointerPointer [WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[4715,77] [rawtypes] found raw type: PointerPointer missing type arguments for generic class PointerPointer
where P is a type-variable:
P extends Pointer declared in class PointerPointer
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/RawTensor.java:[132,30] [unchecked] unchecked generic array creation for varargs parameter of type Class extends Pointer>[]
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/RawTensor.java:[149,30] [unchecked] unchecked generic array creation for varargs parameter of type Class extends Pointer>[]
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/EagerSession.java:[291,30] [unchecked] unchecked generic array creation for varargs parameter of type Class extends Pointer>[]
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/EagerSession.java:[424,30] [unchecked] unchecked generic array creation for varargs parameter of type Class extends Pointer>[]
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/Graph.java:[394,29] [unchecked] unchecked generic array creation for varargs parameter of type Class extends Pointer>[]
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/Graph.java:[417,29] [unchecked] unchecked generic array creation for varargs parameter of type Class extends Pointer>[]
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/Graph.java:[451,29] [unchecked] unchecked generic array creation for varargs parameter of type Class extends Pointer>[]
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/Graph.java:[471,29] [unchecked] unchecked generic array creation for varargs parameter of type Class extends Pointer>[]
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/Graph.java:[815,39] [cast] redundant cast to int
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/Graph.java:[1046,30] [unchecked] unchecked generic array creation for varargs parameter of type Class extends Pointer>[]
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/Graph.java:[1065,30] [unchecked] unchecked generic array creation for varargs parameter of type Class extends Pointer>[]
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/Graph.java:[1081,30] [unchecked] unchecked generic array creation for varargs parameter of type Class extends Pointer>[]
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/Graph.java:[1130,30] [unchecked] unchecked generic array creation for varargs parameter of type Class extends Pointer>[]
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/Graph.java:[1173,30] [unchecked] unchecked generic array creation for varargs parameter of type Class extends Pointer>[]
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/Graph.java:[1303,33] [unchecked] unchecked method invocation: method assign in class Ops is applied to given types
required: Operand Reduces {@code input} along the dimensions given in {@code axes}. Unless {@code keepDims} is'?
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/src/main/java/org/tensorflow/framework/op/MathOps.java:[439,24] [InvalidParam] `keepdims` is very close to the parameter `keepDims`. Did you mean to reference the parameter?
(see https://errorprone.info/bugpattern/InvalidParam)
Did you mean '* must be unique. If {@code keepDims} is true, the reduced dimensions are retained with length 1.'?
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/src/main/java/org/tensorflow/framework/op/LinalgOps.java:[26,29] [UnusedVariable] The field 'frameworkOps' is never read.
(see https://errorprone.info/bugpattern/UnusedVariable)
Did you mean to remove this line?
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/src/main/java/org/tensorflow/framework/metrics/impl/MeanBaseMetricWrapper.java:[90,18] [MissingOverride] updateStateList overrides method in BaseMetric; expected @Override
(see https://errorprone.info/bugpattern/MissingOverride)
Did you mean '@Override public List Reduces {@code input} along the dimensions given in {@code axes}. Unless {@code keepDims} is'?
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/src/main/java/org/tensorflow/framework/op/math/ReduceLogSumExp.java:[46,24] [InvalidParam] `keepdims` is very close to the parameter `keepDims`. Did you mean to reference the parameter?
(see https://errorprone.info/bugpattern/InvalidParam)
Did you mean '* must be unique. If {@code keepDims} is true, the reduced dimensions are retained with length 1.'?
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/src/main/java/org/tensorflow/framework/metrics/impl/MetricsHelper.java:[72,67] [InvalidParam] `sampleWeight` is very close to the parameter `sampleWeights`. Did you mean to reference the parameter?
(see https://errorprone.info/bugpattern/InvalidParam)
Did you mean '* @return {@code Operation} with control dependencies to ensure {@code sampleWeights} can be'?
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/src/main/java/org/tensorflow/framework/metrics/impl/MetricsHelper.java:[323,64] [InvalidParam] `multiLabels` is very close to the parameter `multiLabel`. Did you mean to reference the parameter?
(see https://errorprone.info/bugpattern/InvalidParam)
Did you mean '* (after slicing by {@code classIndex}, if provided). If {@code multiLabel}, then the'?
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/src/main/java/org/tensorflow/framework/metrics/impl/MetricsHelper.java:[718,73] [InvalidParam] `keepdims` is very close to the parameter `keepDims`. Did you mean to reference the parameter?
(see https://errorprone.info/bugpattern/InvalidParam)
Did you mean '* @param keepDims Indicates whether to keep the dimensions or not. If {@code keepDims} is {@code'?
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/src/main/java/org/tensorflow/framework/metrics/impl/MetricsHelper.java:[734,73] [InvalidParam] `keepdims` is very close to the parameter `keepDims`. Did you mean to reference the parameter?
(see https://errorprone.info/bugpattern/InvalidParam)
Did you mean '* @param keepDims Indicates whether to keep the dimensions or not. If {@code keepDims} is {@code'?
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/src/main/java/org/tensorflow/framework/metrics/impl/MetricsHelper.java:[903,73] [InvalidParam] `keepdims` is very close to the parameter `keepDims`. Did you mean to reference the parameter?
(see https://errorprone.info/bugpattern/InvalidParam)
Did you mean '* @param keepDims Indicates whether to keep the dimensions or not. If {@code keepDims} is {@code'?
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/src/main/java/org/tensorflow/framework/metrics/impl/MetricsHelper.java:[918,73] [InvalidParam] `keepdims` is very close to the parameter `keepDims`. Did you mean to reference the parameter?
(see https://errorprone.info/bugpattern/InvalidParam)
Did you mean '* @param keepDims Indicates whether to keep the dimensions or not. If {@code keepDims} is {@code'?
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/src/main/java/org/tensorflow/framework/metrics/MeanTensor.java:[166,6] [MissingSummary] A summary fragment is required; consider using the value of the @return block as a summary fragment instead.
(see https://google.github.io/styleguide/javaguide.html#s7.2-summary-fragment)
Did you mean '/** Returns the total.'?
[WARNING] /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/src/main/java/org/tensorflow/framework/metrics/MeanTensor.java:[171,6] [MissingSummary] A summary fragment is required; consider using the value of the @return block as a summary fragment instead.
(see https://google.github.io/styleguide/javaguide.html#s7.2-summary-fragment)
Did you mean '/** Returns the count.'?
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ tensorflow-framework ---
[INFO] Using 'UTF8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ tensorflow-framework ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ tensorflow-framework ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.tensorflow.framework.activations.SoftmaxTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.741 s - in org.tensorflow.framework.activations.SoftmaxTest
[INFO] Running org.tensorflow.framework.activations.ELUTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.264 s - in org.tensorflow.framework.activations.ELUTest
[INFO] Running org.tensorflow.framework.activations.ExponentialTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.266 s - in org.tensorflow.framework.activations.ExponentialTest
[INFO] Running org.tensorflow.framework.activations.SoftplusTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.295 s - in org.tensorflow.framework.activations.SoftplusTest
[INFO] Running org.tensorflow.framework.activations.SwishTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.264 s - in org.tensorflow.framework.activations.SwishTest
[INFO] Running org.tensorflow.framework.activations.SigmoidTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.289 s - in org.tensorflow.framework.activations.SigmoidTest
[INFO] Running org.tensorflow.framework.activations.ReLUTest
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.352 s - in org.tensorflow.framework.activations.ReLUTest
[INFO] Running org.tensorflow.framework.activations.TanhTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.289 s - in org.tensorflow.framework.activations.TanhTest
[INFO] Running org.tensorflow.framework.activations.SELUTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.255 s - in org.tensorflow.framework.activations.SELUTest
[INFO] Running org.tensorflow.framework.activations.HardSigmoidTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.289 s - in org.tensorflow.framework.activations.HardSigmoidTest
[INFO] Running org.tensorflow.framework.activations.LinearTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.285 s - in org.tensorflow.framework.activations.LinearTest
[INFO] Running org.tensorflow.framework.activations.SoftsignTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.267 s - in org.tensorflow.framework.activations.SoftsignTest
[INFO] Running org.tensorflow.framework.metrics.KLDivergenceTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.307 s - in org.tensorflow.framework.metrics.KLDivergenceTest
[INFO] Running org.tensorflow.framework.metrics.impl.BroadcastWeightsTest
[INFO] Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.395 s - in org.tensorflow.framework.metrics.impl.BroadcastWeightsTest
[INFO] Running org.tensorflow.framework.metrics.impl.AssertBroadcastableTest
[INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.376 s - in org.tensorflow.framework.metrics.impl.AssertBroadcastableTest
[INFO] Running org.tensorflow.framework.metrics.AUCTest
[INFO] Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.756 s - in org.tensorflow.framework.metrics.AUCTest
[INFO] Running org.tensorflow.framework.metrics.RecallTest
[INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.635 s - in org.tensorflow.framework.metrics.RecallTest
[INFO] Running org.tensorflow.framework.metrics.BinaryCrossentropyTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.326 s - in org.tensorflow.framework.metrics.BinaryCrossentropyTest
[INFO] Running org.tensorflow.framework.metrics.SparseCategoricalCrossentropyTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.329 s - in org.tensorflow.framework.metrics.SparseCategoricalCrossentropyTest
[INFO] Running org.tensorflow.framework.metrics.SpecificityAtSensitivityTest
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.471 s - in org.tensorflow.framework.metrics.SpecificityAtSensitivityTest
[INFO] Running org.tensorflow.framework.metrics.MeanAbsolutePercentageErrorTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.298 s - in org.tensorflow.framework.metrics.MeanAbsolutePercentageErrorTest
[INFO] Running org.tensorflow.framework.metrics.MeanAbsoluteErrorTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.334 s - in org.tensorflow.framework.metrics.MeanAbsoluteErrorTest
[INFO] Running org.tensorflow.framework.metrics.MeanSquaredLogarithmicErrorTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.345 s - in org.tensorflow.framework.metrics.MeanSquaredLogarithmicErrorTest
[INFO] Running org.tensorflow.framework.metrics.SumTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.342 s - in org.tensorflow.framework.metrics.SumTest
[INFO] Running org.tensorflow.framework.metrics.RootMeanSquaredErrorTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.313 s - in org.tensorflow.framework.metrics.RootMeanSquaredErrorTest
[INFO] Running org.tensorflow.framework.metrics.CategoricalCrossentropyTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.375 s - in org.tensorflow.framework.metrics.CategoricalCrossentropyTest
[INFO] Running org.tensorflow.framework.metrics.TrueNegativesTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.365 s - in org.tensorflow.framework.metrics.TrueNegativesTest
[INFO] Running org.tensorflow.framework.metrics.TruePositivesTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.384 s - in org.tensorflow.framework.metrics.TruePositivesTest
[INFO] Running org.tensorflow.framework.metrics.BinaryAccuracyTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.394 s - in org.tensorflow.framework.metrics.BinaryAccuracyTest
[INFO] Running org.tensorflow.framework.metrics.MeanIoUTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.388 s - in org.tensorflow.framework.metrics.MeanIoUTest
[INFO] Running org.tensorflow.framework.metrics.CategoricalHingeTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.307 s - in org.tensorflow.framework.metrics.CategoricalHingeTest
[INFO] Running org.tensorflow.framework.metrics.HingeTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.308 s - in org.tensorflow.framework.metrics.HingeTest
[INFO] Running org.tensorflow.framework.metrics.MeanRelativeErrorTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.295 s - in org.tensorflow.framework.metrics.MeanRelativeErrorTest
[INFO] Running org.tensorflow.framework.metrics.PoissonTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.342 s - in org.tensorflow.framework.metrics.PoissonTest
[INFO] Running org.tensorflow.framework.metrics.PrecisionAtRecallTest
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.472 s - in org.tensorflow.framework.metrics.PrecisionAtRecallTest
[INFO] Running org.tensorflow.framework.metrics.MeanSquaredErrorTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.32 s - in org.tensorflow.framework.metrics.MeanSquaredErrorTest
[INFO] Running org.tensorflow.framework.metrics.CosineSimilarityTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.323 s - in org.tensorflow.framework.metrics.CosineSimilarityTest
[INFO] Running org.tensorflow.framework.metrics.LogCoshErrorTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.315 s - in org.tensorflow.framework.metrics.LogCoshErrorTest
[INFO] Running org.tensorflow.framework.metrics.FalsePositivesTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.39 s - in org.tensorflow.framework.metrics.FalsePositivesTest
[INFO] Running org.tensorflow.framework.metrics.TopKCategoricalAccuracyTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.348 s - in org.tensorflow.framework.metrics.TopKCategoricalAccuracyTest
[INFO] Running org.tensorflow.framework.metrics.CategoricalAccuracyTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.38 s - in org.tensorflow.framework.metrics.CategoricalAccuracyTest
[INFO] Running org.tensorflow.framework.metrics.PrecisionTest
[INFO] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.678 s - in org.tensorflow.framework.metrics.PrecisionTest
[INFO] Running org.tensorflow.framework.metrics.SensitivityAtSpecificityTest
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.485 s - in org.tensorflow.framework.metrics.SensitivityAtSpecificityTest
[INFO] Running org.tensorflow.framework.metrics.AccuracyTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.352 s - in org.tensorflow.framework.metrics.AccuracyTest
[INFO] Running org.tensorflow.framework.metrics.MeanTensorTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.362 s - in org.tensorflow.framework.metrics.MeanTensorTest
[INFO] Running org.tensorflow.framework.metrics.RecallAtPrecisionTest
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.481 s - in org.tensorflow.framework.metrics.RecallAtPrecisionTest
[INFO] Running org.tensorflow.framework.metrics.FalseNegativesTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.417 s - in org.tensorflow.framework.metrics.FalseNegativesTest
[INFO] Running org.tensorflow.framework.metrics.SquaredHingeTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.33 s - in org.tensorflow.framework.metrics.SquaredHingeTest
[INFO] Running org.tensorflow.framework.losses.KLDivergenceTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.354 s - in org.tensorflow.framework.losses.KLDivergenceTest
[INFO] Running org.tensorflow.framework.losses.BinaryCrossentropyTest
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.394 s - in org.tensorflow.framework.losses.BinaryCrossentropyTest
[INFO] Running org.tensorflow.framework.losses.SparseCategoricalCrossentropyTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.398 s - in org.tensorflow.framework.losses.SparseCategoricalCrossentropyTest
[INFO] Running org.tensorflow.framework.losses.LogCoshTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.344 s - in org.tensorflow.framework.losses.LogCoshTest
[INFO] Running org.tensorflow.framework.losses.MeanAbsolutePercentageErrorTest
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.371 s - in org.tensorflow.framework.losses.MeanAbsolutePercentageErrorTest
[INFO] Running org.tensorflow.framework.losses.MeanAbsoluteErrorTest
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.351 s - in org.tensorflow.framework.losses.MeanAbsoluteErrorTest
[INFO] Running org.tensorflow.framework.losses.MeanSquaredLogarithmicErrorTest
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.351 s - in org.tensorflow.framework.losses.MeanSquaredLogarithmicErrorTest
[INFO] Running org.tensorflow.framework.losses.CategoricalCrossentropyTest
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.406 s - in org.tensorflow.framework.losses.CategoricalCrossentropyTest
[INFO] Running org.tensorflow.framework.losses.CategoricalHingeTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.365 s - in org.tensorflow.framework.losses.CategoricalHingeTest
[INFO] Running org.tensorflow.framework.losses.HingeTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.391 s - in org.tensorflow.framework.losses.HingeTest
[INFO] Running org.tensorflow.framework.losses.PoissonTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.346 s - in org.tensorflow.framework.losses.PoissonTest
[INFO] Running org.tensorflow.framework.losses.MeanSquaredErrorTest
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.35 s - in org.tensorflow.framework.losses.MeanSquaredErrorTest
[INFO] Running org.tensorflow.framework.losses.CosineSimilarityTest
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.382 s - in org.tensorflow.framework.losses.CosineSimilarityTest
[INFO] Running org.tensorflow.framework.losses.HuberTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.357 s - in org.tensorflow.framework.losses.HuberTest
[INFO] Running org.tensorflow.framework.losses.SquaredHingeTest
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.392 s - in org.tensorflow.framework.losses.SquaredHingeTest
[INFO] Running org.tensorflow.framework.constraints.NonNegTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.293 s - in org.tensorflow.framework.constraints.NonNegTest
[INFO] Running org.tensorflow.framework.constraints.MinMaxNormTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.334 s - in org.tensorflow.framework.constraints.MinMaxNormTest
[INFO] Running org.tensorflow.framework.constraints.MaxNormTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.364 s - in org.tensorflow.framework.constraints.MaxNormTest
[INFO] Running org.tensorflow.framework.constraints.UnitNormTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.333 s - in org.tensorflow.framework.constraints.UnitNormTest
[INFO] Running org.tensorflow.framework.op.SetOpsTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.379 s - in org.tensorflow.framework.op.SetOpsTest
[INFO] Running org.tensorflow.framework.op.NnOpsTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.317 s - in org.tensorflow.framework.op.NnOpsTest
[INFO] Running org.tensorflow.framework.op.MathOpsTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.422 s - in org.tensorflow.framework.op.MathOpsTest
[INFO] Running org.tensorflow.framework.op.LinalgOpsTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.315 s - in org.tensorflow.framework.op.LinalgOpsTest
[INFO] Running org.tensorflow.framework.optimizers.AdaDeltaTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.469 s - in org.tensorflow.framework.optimizers.AdaDeltaTest
[INFO] Running org.tensorflow.framework.optimizers.OptimizersTest
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.263 s - in org.tensorflow.framework.optimizers.OptimizersTest
[INFO] Running org.tensorflow.framework.optimizers.FtrlTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.348 s - in org.tensorflow.framework.optimizers.FtrlTest
[INFO] Running org.tensorflow.framework.optimizers.AdamTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.307 s - in org.tensorflow.framework.optimizers.AdamTest
[INFO] Running org.tensorflow.framework.optimizers.GradientDescentTest
[WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 1.307 s - in org.tensorflow.framework.optimizers.GradientDescentTest
[INFO] Running org.tensorflow.framework.optimizers.NadamTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.37 s - in org.tensorflow.framework.optimizers.NadamTest
[INFO] Running org.tensorflow.framework.optimizers.AdamaxTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.324 s - in org.tensorflow.framework.optimizers.AdamaxTest
[INFO] Running org.tensorflow.framework.optimizers.AdaGradTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.323 s - in org.tensorflow.framework.optimizers.AdaGradTest
[INFO] Running org.tensorflow.framework.optimizers.MomentumTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.327 s - in org.tensorflow.framework.optimizers.MomentumTest
[INFO] Running org.tensorflow.framework.optimizers.AdaGradDATest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.317 s - in org.tensorflow.framework.optimizers.AdaGradDATest
[INFO] Running org.tensorflow.framework.optimizers.RMSPropTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.432 s - in org.tensorflow.framework.optimizers.RMSPropTest
[INFO] Running org.tensorflow.framework.regularizers.L2Test
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.315 s - in org.tensorflow.framework.regularizers.L2Test
[INFO] Running org.tensorflow.framework.regularizers.L1L2Test
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.337 s - in org.tensorflow.framework.regularizers.L1L2Test
[INFO] Running org.tensorflow.framework.regularizers.RegularizerLossTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.299 s - in org.tensorflow.framework.regularizers.RegularizerLossTest
[INFO] Running org.tensorflow.framework.regularizers.L1Test
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.337 s - in org.tensorflow.framework.regularizers.L1Test
[INFO] Running org.tensorflow.framework.data.SkipDatasetTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.292 s - in org.tensorflow.framework.data.SkipDatasetTest
[INFO] Running org.tensorflow.framework.data.MapDatasetTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.286 s - in org.tensorflow.framework.data.MapDatasetTest
[INFO] Running org.tensorflow.framework.data.DatasetIteratorTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.316 s - in org.tensorflow.framework.data.DatasetIteratorTest
[INFO] Running org.tensorflow.framework.data.TakeDatasetTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.269 s - in org.tensorflow.framework.data.TakeDatasetTest
[INFO] Running org.tensorflow.framework.data.BatchDatasetTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.312 s - in org.tensorflow.framework.data.BatchDatasetTest
[INFO] Running org.tensorflow.framework.initializers.OnesTest
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.347 s - in org.tensorflow.framework.initializers.OnesTest
[INFO] Running org.tensorflow.framework.initializers.RandomUniformTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.298 s - in org.tensorflow.framework.initializers.RandomUniformTest
[INFO] Running org.tensorflow.framework.initializers.GlorotTest
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.318 s - in org.tensorflow.framework.initializers.GlorotTest
[INFO] Running org.tensorflow.framework.initializers.IdentityTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.329 s - in org.tensorflow.framework.initializers.IdentityTest
[INFO] Running org.tensorflow.framework.initializers.ConstantTest
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.337 s - in org.tensorflow.framework.initializers.ConstantTest
[INFO] Running org.tensorflow.framework.initializers.ZerosTest
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.342 s - in org.tensorflow.framework.initializers.ZerosTest
[INFO] Running org.tensorflow.framework.initializers.RandomNormalTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.309 s - in org.tensorflow.framework.initializers.RandomNormalTest
[INFO] Running org.tensorflow.framework.initializers.TruncatedNormalTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.346 s - in org.tensorflow.framework.initializers.TruncatedNormalTest
[INFO] Running org.tensorflow.framework.initializers.HeTest
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.344 s - in org.tensorflow.framework.initializers.HeTest
[INFO] Running org.tensorflow.framework.initializers.VarianceScalingTest
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.331 s - in org.tensorflow.framework.initializers.VarianceScalingTest
[INFO] Running org.tensorflow.framework.initializers.LeCunTest
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.342 s - in org.tensorflow.framework.initializers.LeCunTest
[INFO] Running org.tensorflow.framework.initializers.OrthogonalTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.336 s - in org.tensorflow.framework.initializers.OrthogonalTest
[INFO]
[INFO] Results:
[INFO]
[WARNING] Tests run: 498, Failures: 0, Errors: 0, Skipped: 1
[INFO]
[INFO]
[INFO] --- maven-jar-plugin:3.2.0:jar (default-jar) @ tensorflow-framework ---
[INFO] Building jar: /Users/ducha/Workspace/tf_build/tensorflow-java/tensorflow-framework/target/tensorflow-framework-0.4.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-source-plugin:3.2.1:jar-no-fork (attach-sources) @ tensorflow-framework ---
[INFO] Skipping source per configuration.
[INFO]
[INFO] --- maven-javadoc-plugin:3.2.0:jar (attach-javadocs) @ tensorflow-framework ---
[INFO] Skipping javadoc generation
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for TensorFlow Java Parent 0.4.0-SNAPSHOT:
[INFO]
[INFO] TensorFlow Java Parent ............................. SUCCESS [ 0.644 s]
[INFO] TensorFlow Core Parent ............................. SUCCESS [ 0.005 s]
[INFO] TensorFlow Core Generators ......................... SUCCESS [ 0.461 s]
[INFO] TensorFlow Core API Library ........................ SUCCESS [01:13 min]
[INFO] TensorFlow Core API Library Platform ............... SUCCESS [01:22 min]
[INFO] TensorFlow Framework Library ....................... SUCCESS [03:36 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 06:13 min
[INFO] Finished at: 2021-11-16T17:12:51+01:00
[INFO] ------------------------------------------------------------------------
```
@Craigacp I am not sure what I am missing here:
(base) kgoderis@Karels-M1-MacBook-Pro craig % java -version openjdk version **"11.0.13"** 2021-10-19 LTS OpenJDK Runtime Environment Zulu11.52+13-CA (build 11.0.13+8-LTS) OpenJDK 64-Bit Server VM Zulu11.52+13-CA (build 11.0.13+8-LTS, mixed mode) (base) kgoderis@Karels-M1-MacBook-Pro craig % sudo mvn clean package Password: [INFO] Scanning for projects... [INFO] ------------------------------------------------------------------------ [INFO] Reactor Build Order: [INFO] [INFO] TensorFlow Java Parent [pom] [INFO] TensorFlow Core Parent [pom] [INFO] TensorFlow Core Generators [jar] [INFO] TensorFlow Core API Library [jar] [INFO] TensorFlow Core API Library Platform [jar] [INFO] TensorFlow Framework Library [jar] [INFO] [INFO] -------------------< org.tensorflow:tensorflow-java >------------------- [INFO] Building TensorFlow Java Parent 0.4.0-SNAPSHOT [1/6] [INFO] --------------------------------[ pom ]--------------------------------- [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tensorflow-java --- [INFO] [INFO] --- maven-source-plugin:3.2.1:jar-no-fork (attach-sources) @ tensorflow-java --- [INFO] Skipping source per configuration. [INFO] [INFO] --- maven-javadoc-plugin:3.2.0:jar (attach-javadocs) @ tensorflow-java --- [INFO] Skipping javadoc generation [INFO] [INFO] -------------------< org.tensorflow:tensorflow-core >------------------- [INFO] Building TensorFlow Core Parent 0.4.0-SNAPSHOT [2/6] [INFO] --------------------------------[ pom ]--------------------------------- [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tensorflow-core --- [INFO] [INFO] --- maven-source-plugin:3.2.1:jar-no-fork (attach-sources) @ tensorflow-core --- [INFO] Skipping source per configuration. [INFO] [INFO] --- maven-javadoc-plugin:3.2.0:jar (attach-javadocs) @ tensorflow-core --- [INFO] Skipping javadoc generation [INFO] [INFO] --------------< org.tensorflow:tensorflow-core-generator >-------------- [INFO] Building TensorFlow Core Generators 0.4.0-SNAPSHOT [3/6] [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tensorflow-core-generator --- [INFO] Deleting /Users/kgoderis/Development/craig/tensorflow-core/tensorflow-core-generator/target [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ tensorflow-core-generator --- [INFO] Using 'UTF8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /Users/kgoderis/Development/craig/tensorflow-core/tensorflow-core-generator/src/main/resources [INFO] [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ tensorflow-core-generator --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 43 source files to /Users/kgoderis/Development/craig/tensorflow-core/tensorflow-core-generator/target/classes [INFO] ------------------------------------------------------------- [WARNING] COMPILATION WARNING : [INFO] ------------------------------------------------------------- [WARNING] [options] bootstrap class path not set in conjunction with -source 8 [INFO] 1 warning [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /Users/kgoderis/Development/craig/tensorflow-core/tensorflow-core-generator/src/main/java/org/tensorflow/op/generator/javadoc/JavaDocRenderer.java:[47,7] error: An unhandled exception was thrown by the Error Prone static analysis plugin. Please report this at https://github.com/google/error-prone/issues/new and include the following: error-prone version: 2.6.0 BugPattern: JavaLangClash Stack Trace: java.lang.NoSuchMethodError: 'java.lang.Iterable com.sun.tools.javac.code.Scope$WriteableScope.getSymbolsByName(com.sun.tools.javac.util.Name, com.sun.tools.javac.util.Filter)' at com.google.errorprone.bugpatterns.JavaLangClash.check(JavaLangClash.java:66) at com.google.errorprone.bugpatterns.JavaLangClash.matchClass(JavaLangClash.java:53) at com.google.errorprone.scanner.ErrorProneScanner.processMatchers(ErrorProneScanner.java:450) at com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:548) at com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:151) at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:860) at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:86) at com.google.errorprone.scanner.Scanner.scan(Scanner.java:74) at com.google.errorprone.scanner.Scanner.scan(Scanner.java:48) at jdk.compiler/com.sun.source.util.TreeScanner.scan(TreeScanner.java:111) at jdk.compiler/com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:119) at jdk.compiler/com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:152) at com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:561) at com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:151) at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:614) at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:60) at com.google.errorprone.scanner.Scanner.scan(Scanner.java:58) at com.google.errorprone.scanner.ErrorProneScannerTransformer.apply(ErrorProneScannerTransformer.java:43) at com.google.errorprone.ErrorProneAnalyzer.finished(ErrorProneAnalyzer.java:152) at jdk.compiler/com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:132) at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1394) at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1341) at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:933) at jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:317) at jdk.compiler/com.sun.tools.javac.main.Main.compile(Main.java:176) at jdk.compiler/com.sun.tools.javac.Main.compile(Main.java:64) at jdk.compiler/com.sun.tools.javac.Main.main(Main.java:50) [INFO] 1 error
@kgoderis Running sudo mvn package
runs maven as root with a different environment, where I assume it's picking up Java 17 by default. For example on my machine where I have JAVA_HOME
set to Zulu 11, when running maven as root it pulls in Java 17 as that's my system wide default:
$ mvn --version
Apache Maven 3.8.3 (ff8e977a158738155dc465c6a97ffaf31982d739)
Maven home: /opt/homebrew/Cellar/maven/3.8.3/libexec
Java version: 11.0.10, vendor: Azul Systems, Inc., runtime: /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home
Default locale: en_GB, platform encoding: UTF-8
OS name: "mac os x", version: "12.0.1", arch: "aarch64", family: "mac"
$ sudo mvn --version
Apache Maven 3.8.3 (ff8e977a158738155dc465c6a97ffaf31982d739)
Maven home: /opt/homebrew/Cellar/maven/3.8.3/libexec
Java version: 17, vendor: Homebrew, runtime: /opt/homebrew/Cellar/openjdk/17/libexec/openjdk.jdk/Contents/Home
Default locale: en_GB, platform encoding: UTF-8
OS name: "mac os x", version: "12.0.1", arch: "aarch64", family: "mac"
Running as root can also cause Python issues as for some reason the TF native library build requires numpy. In my branch I modified build.sh
to only run bazel as root, along with a few chmods and lns to make everything line up. Before we merge this into master
we need to remove those as we shouldn't encourage people to run the build as root.
Hi everyone,
I am also currently trying to build the module on my Apple silicon machine (0.4.0-SNAPSHOT with Zulu SDK 11 on macOS Monterey).
I was able to compile it (however quite messily and independently from @Craigacp's changes) and for me it passes the tests.
Checking out Adam's fork and compiling it works for me too. @kgoderis I'm sorry that I can't help you, I had completely different issues when trying to build it.
It has many warnings though. The following types of warnings were logged:
FloatingPointLiteralPrecision ImmutableEnumChecker InvalidParam MissingOverride MissingSummary UnusedVariable cast options rawtypes serial unchecked
@DevinTDHa Error-prone does flag a bunch of issues with TF-Java, some of which are due to the generated protobuf code which we can't change, and others are things that we will fix eventually but haven't got to yet.
When inspecting the jars, it was missing some dependencies like org.tensorflow.ndarray and com.google.protobuf (potentially more). This applies to the jars resulting from Adams fork as well.
My question is: do you think that this will be a problem going forward? Or might this be another setup environment issue?
For the missing org.tensorflow.ndarray I had to compile it on my machine and add it manually. I believe com.google.protobuf might not have jars for apple silicon either for version < 3.18, so might need to be compiled locally as well. I haven't checked which other dependencies are missing.
Those jars won't be built as part of the TF-Java build process, they should be pulled in by Maven if you depend on TF-Java, so it's expected that they aren't present in our build. TF-Java doesn't build a with-dependencies
jar, but if you want one then it's fairly easy to make. Protobuf-java is purely Java, there is no native library in it, so the jar file on Maven Central will work just fine on Apple Silicon. tensorflow-ndarray
is also pure Java without a native library, so that shouldn't be an issue either.
@Craigacp Thanks for pointing out sudo java --version. Indeed, I have Java 15 in that environment
What is the current status for this? Is there anything we (community) can help to make it work on M1?
Our use case is that we need to be unblocked to develop ML Inference on M1 machines. We don't currently need to Train ML models with tensorflow-java (we use Python for that, which already works with M1).
Using the branch I mentioned above it should compile and produce macos-arm64 jars. I'll respin the branch for the 0.4.0 release at some point this week as it's currently targeting TF 2.6.0, and 0.4.0 ships with TF 2.7.
We can't provide binaries until there is some build support in Github Actions for it, or we figure out cross compiling from a macOS x86_64 environment.
If you can figure out why it requires sudo to build the TF native library, and how we can convert it into a regular user build then that will help get it merged into master
as we can't require that the build runs as root. It's something to do with how XCode is configured, but I've not had enough time to figure it out yet.
Using the branch I mentioned above it should compile and produce macos-arm64 jars
Ok, thanks, I'll try to build it from that branch and report here if I had a success or failure (to hopefully help others do the same).
Hello Everyone,
For anyone who might need it, I uploaded the compiled jars for myself here:
https://github.com/DevinTDHa/precompiled-tf-jni-m1
Hope it might be useful to someone else as well.
Hi @Craigacp,
I have a question regarding this issue. I tried to keep up to date with the releases and was able to compile it with the help of you branch so far, up to version 0.4.1.
Updating to 0.4.2 I am running into some issues and I was not able to figure it out yet. The branch I am working on is the following: https://github.com/DevinTDHa/tensorflow-java/tree/apple-silicon-0.4.2-shaded
The compilation runs fine until the Exporting Ops
stage in the build.sh
file.
The following error occurs:
[INFO] --- javacpp:1.5.6:build (javacpp-validate) @ tensorflow-core-api ---
[INFO] Detected platform "macosx-arm64"
[INFO] Building platform "macosx-arm64"
[INFO]
[INFO] --- javacpp:1.5.6:build (javacpp-build) @ tensorflow-core-api ---
[INFO] Detected platform "macosx-arm64"
[INFO] Building platform "macosx-arm64"
[INFO] bash /Users/ducha/Workspace/Building/tensorflow-java/tensorflow-core/tensorflow-core-api/build.sh
Password:
INFO: Invocation ID: 627f0e51-b286-4d5c-b950-ac564b5c7318
INFO: Options provided by the client:
Inherited 'common' options: --isatty=1 --terminal_columns=156
INFO: Reading rc options for 'build' from /Users/ducha/Workspace/Building/tensorflow-java/tensorflow-core/tensorflow-core-api/tensorflow.bazelrc:
Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /Users/ducha/Workspace/Building/tensorflow-java/tensorflow-core/tensorflow-core-api/.bazelrc:
'build' options: --remote_cache=https://storage.googleapis.com/tensorflow-sigs-jvm --remote_upload_local_results=false --incompatible_restrict_string_escapes=false
INFO: Reading rc options for 'build' from /Users/ducha/Workspace/Building/tensorflow-java/tensorflow-core/tensorflow-core-api/tensorflow.bazelrc:
'build' options: --define framework_shared_object=true --java_toolchain=@tf_toolchains//toolchains/java:tf_java_toolchain --host_java_toolchain=@tf_toolchains//toolchains/java:tf_java_toolchain --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/common,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/fallback,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils
INFO: Found applicable config definition build:short_logs in file /Users/ducha/Workspace/Building/tensorflow-java/tensorflow-core/tensorflow-core-api/tensorflow.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /Users/ducha/Workspace/Building/tensorflow-java/tensorflow-core/tensorflow-core-api/tensorflow.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:opt in file /Users/ducha/Workspace/Building/tensorflow-java/tensorflow-core/tensorflow-core-api/tensorflow.bazelrc: --copt=-Wno-sign-compare --host_copt=-Wno-sign-compare
INFO: Found applicable config definition build:macos_arm64 in file /Users/ducha/Workspace/Building/tensorflow-java/tensorflow-core/tensorflow-core-api/tensorflow.bazelrc: --cpu=darwin_arm64 --host_cpu=darwin_arm64
INFO: Found applicable config definition build:macos in file /Users/ducha/Workspace/Building/tensorflow-java/tensorflow-core/tensorflow-core-api/tensorflow.bazelrc: --apple_platform_type=macos --copt=-DGRPC_BAZEL_BUILD --copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++14 --host_cxxopt=-std=c++14
INFO: Analyzed 6 targets (0 packages loaded, 0 targets configured).
INFO: Found 6 targets...
INFO: Elapsed time: 0.477s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
Listing /Users/ducha/Workspace/Building/tensorflow-java/tensorflow-core/tensorflow-core-api/bazel-bin/external/org_tensorflow/tensorflow:
total 837800
drwxr-xr-x 28 root wheel 896 Nov 24 19:14 c
drwxr-xr-x 76 root wheel 2432 Nov 24 18:55 cc
drwxr-xr-x 6 root wheel 192 Nov 24 11:55 compiler
drwxr-xr-x 28 root wheel 896 Nov 24 18:44 core
-r-xr-xr-x 1 ducha wheel 402496848 Nov 24 19:18 libtensorflow_cc.2.7.4.dylib
-r-xr-xr-x 1 root wheel 179888 Nov 24 13:19 libtensorflow_cc.2.7.4.dylib-2.params
lrwxr-xr-x 1 root wheel 28 Nov 25 14:47 libtensorflow_cc.2.dylib -> libtensorflow_cc.2.7.4.dylib
lrwxr-xr-x 1 root wheel 28 Nov 25 14:47 libtensorflow_cc.dylib -> libtensorflow_cc.2.7.4.dylib
-r-xr-xr-x 1 root wheel 26219836 Nov 24 19:10 libtensorflow_framework.2.7.4.dylib
-r-xr-xr-x 1 root wheel 51689 Nov 24 13:51 libtensorflow_framework.2.7.4.dylib-2.params
lrwxr-xr-x 1 root wheel 35 Nov 24 19:10 libtensorflow_framework.2.dylib -> libtensorflow_framework.2.7.4.dylib
lrwxr-xr-x 1 root wheel 31 Nov 24 19:10 libtensorflow_framework.dylib -> libtensorflow_framework.2.dylib
drwxr-xr-x 3 root wheel 96 Nov 24 12:37 lite
drwxr-xr-x 53 root wheel 1696 Nov 24 18:52 stream_executor
drwxr-xr-x 4 root wheel 128 Nov 24 12:37 tools
Exporting Ops
2022-11-25 14:47:09.892464: F src/bazel/op_generator/op_export_main.cc:121] Non-OK-status: env->LoadDynamicLibrary(argv[1], &ops_libs_handles[0]) status: NOT_FOUND: dlopen(/Users/ducha/Workspace/Building/tensorflow-java/tensorflow-core/tensorflow-core-api/bazel-bin/external/org_tensorflow/tensorflow/libtensorflow_cc.2.7.4.dylib, 0x0006): malformed trie child, cycle to nodeOffset=0x3
weak-def symbol not found (__ZGVZN10tensorflow13Divide16x8PowILi7EEE17__simd128_int16_tS1_E9FIRST_BIT)
/Users/ducha/Workspace/Building/tensorflow-java/tensorflow-core/tensorflow-core-api/build.sh: line 106: 94731 Abort trap: 6 $BAZEL_BIN/java_op_exporter $TENSORFLOW_LIB $GEN_RESOURCE_DIR/ops.pb $GEN_RESOURCE_DIR/ops.pbtxt $BAZEL_SRCS/external/org_tensorflow/tensorflow/core/api_def/base_api src/bazel/api_def
I tried to skip the op-export but this also resulted in an error, that TARGET_OS_IPHONE
was not defined (and indeed it isn't in the code) in tensorflow-java/tensorflow-core/tensorflow-core-api/target/native/org/tensorflow/internal/c_api/macosx-arm64/jnijavacpp.cpp
.
I am not entirely sure what would cause this issue. I am using llvm@11 from homebrew and bazelisk for this process.
I am not entirely sure what would cause this issue. I am using llvm@11 from homebrew and bazelisk for this process.
Please update to JavaCPP 1.5.8 in that case, see https://github.com/bytedeco/javacpp/issues/609
The compilation runs fine until the Exporting Ops stage in the build.sh file.
I just found out that the Export Ops
step in general doesn't work on my system anymore. Seems like something broke after one of the auto-updates for Xcode?
I just found out that the Export Ops step in general doesn't work on my system anymore. Seems like something broke after one of the auto-updates for Xcode?
We're also hitting this in our local builds, not figured out what's going on yet.
@Craigacp
I was able to resolve it by downgrading command line tools and Xcode to version 13.1.6! Goes through like before.
@DevinTDHa , you just saved my life. I've been trying to build TF 2.10 on my M1 for awhile and was blocked by this malformed trie
error as well. Looks like downgrading Xcode from 14.x to 13.x did the job!
Other than that, building 2.10 is pretty straightforward and I've fixed also how the op exporter is linking to TF. I would like to update TF-Java repo so that the latest snapshot can be easily build on M1 machines. @saudet , I've also updated JavaCPP to 1.5.8. Still, I'm facing some new problems that don't seem related to M1 this time but maybe more on 2.10, I'll take a look later but any advice from you would be more than welcome:
/Users/klessard/Documents/Projects/ML/Sources/TensorFlow/tensorflow-java/tensorflow-core/tensorflow-core-api/target/native/org/tensorflow/internal/c_api/macosx-arm64/jnitensorflow.cpp:3380:47: error: no matching constructor for initialization of 'SpanAdapter<tensorflow::SourceLocation>'
SpanAdapter< tensorflow::SourceLocation > radapter(ptr->GetSourceLocations());
^ ~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/klessard/Documents/Projects/ML/Sources/TensorFlow/tensorflow-java/tensorflow-core/tensorflow-core-api/target/classes/org/tensorflow/internal/c_api/include/tensorflow_adapters.h:19:28: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'absl::Span<const SourceLocation>' to 'const SpanAdapter<tensorflow::SourceLocation>' for 1st argument
template<typename T> class SpanAdapter {
^
/Users/klessard/Documents/Projects/ML/Sources/TensorFlow/tensorflow-java/tensorflow-core/tensorflow-core-api/target/classes/org/tensorflow/internal/c_api/include/tensorflow_adapters.h:19:28: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'absl::Span<const SourceLocation>' to 'SpanAdapter<tensorflow::SourceLocation>' for 1st argument
/Users/klessard/Documents/Projects/ML/Sources/TensorFlow/tensorflow-java/tensorflow-core/tensorflow-core-api/target/classes/org/tensorflow/internal/c_api/include/tensorflow_adapters.h:23:5: note: candidate constructor not viable: no known conversion from 'Span<const tensorflow::SourceLocation>' to 'const Span<tensorflow::SourceLocation>' for 1st argument
SpanAdapter(const Span<T>& arr) : ptr(0), size(0), owner(0), arr2(arr), arr(arr2) { }
^
/Users/klessard/Documents/Projects/ML/Sources/TensorFlow/tensorflow-java/tensorflow-core/tensorflow-core-api/target/classes/org/tensorflow/internal/c_api/include/tensorflow_adapters.h:24:5: note: candidate constructor not viable: no known conversion from 'absl::Span<const SourceLocation>' to 'Span<tensorflow::SourceLocation> &' for 1st argument
SpanAdapter( Span<T>& arr) : ptr(0), size(0), owner(0), arr(arr) { }
^
/Users/klessard/Documents/Projects/ML/Sources/TensorFlow/tensorflow-java/tensorflow-core/tensorflow-core-api/target/classes/org/tensorflow/internal/c_api/include/tensorflow_adapters.h:25:5: note: candidate constructor not viable: no known conversion from 'absl::Span<const SourceLocation>' to 'const Span<tensorflow::SourceLocation> *' for 1st argument
SpanAdapter(const Span<T>* arr) : ptr(0), size(0), owner(0), arr(*(Span<T>*)arr) { }
^
/Users/klessard/Documents/Projects/ML/Sources/TensorFlow/tensorflow-java/tensorflow-core/tensorflow-core-api/target/classes/org/tensorflow/internal/c_api/include/tensorflow_adapters.h:21:5: note: candidate constructor not viable: requires 3 arguments, but 1 was provided
SpanAdapter(T const * ptr, typename Span<T>::size_type size, void* owner) : ptr((T*)ptr), size(size), owner(owner),
Yeah, we should do version upgrades for TF Core separately. Put that in a branch and I'll take a look at it.
Based on that error message, something like this should fix that one though:
.put(new Info("absl::Span<const tensorflow::SourceLocation>").annotations("@Span")
.valueTypes("@Cast(\"const tensorflow::SourceLocation*\") SourceLocation")
.pointerTypes("SourceLocation"))
Based on that error message, something like this should fix that one though:
.put(new Info("absl::Span<const tensorflow::SourceLocation>").annotations("@Span") .valueTypes("@Cast(\"const tensorflow::SourceLocation*\") SourceLocation") .pointerTypes("SourceLocation"))
Yep, that did the trick, thanks! Still, hitting issues now when JavaCPP tries to load the libjnitensorflow
library. JavaCPP fails with the following debug trace.
Debug: Locking /Users/klessard/.javacpp/cache before extracting
Debug: Extracting jar:file:/Users/klessard/Documents/Projects/ML/Sources/TensorFlow/tensorflow-java/tensorflow-core/tensorflow-core-api/target/tensorflow-core-api-0.5.0-SNAPSHOT-macosx-arm64.jar!/org/tensorflow/internal/c_api/macosx-arm64/libtensorflow_framework.2.dylib
Debug: Loading /Users/klessard/.javacpp/cache/tensorflow-core-api-0.5.0-SNAPSHOT-macosx-arm64.jar/org/tensorflow/internal/c_api/macosx-arm64/libtensorflow_framework.2.dylib
Debug: Locking /Users/klessard/.javacpp/cache before extracting
Debug: Extracting jar:file:/Users/klessard/Documents/Projects/ML/Sources/TensorFlow/tensorflow-java/tensorflow-core/tensorflow-core-api/target/tensorflow-core-api-0.5.0-SNAPSHOT-macosx-arm64.jar!/org/tensorflow/internal/c_api/macosx-arm64/libtensorflow_cc.2.dylib
Debug: Loading /Users/klessard/.javacpp/cache/tensorflow-core-api-0.5.0-SNAPSHOT-macosx-arm64.jar/org/tensorflow/internal/c_api/macosx-arm64/libtensorflow_cc.2.dylib
Debug: Locking /Users/klessard/.javacpp/cache before extracting
Debug: Extracting jar:file:/Users/klessard/Documents/Projects/ML/Sources/TensorFlow/tensorflow-java/tensorflow-core/tensorflow-core-api/target/tensorflow-core-api-0.5.0-SNAPSHOT-macosx-arm64.jar!/org/tensorflow/internal/c_api/macosx-arm64/libjnitensorflow.dylib
Debug: Loading /Users/klessard/.javacpp/cache/tensorflow-core-api-0.5.0-SNAPSHOT-macosx-arm64.jar/org/tensorflow/internal/c_api/macosx-arm64/libjnitensorflow.dylib
Debug: Failed to load /Users/klessard/.javacpp/cache/tensorflow-core-api-0.5.0-SNAPSHOT-macosx-arm64.jar/org/tensorflow/internal/c_api/macosx-arm64/libjnitensorflow.dylib: java.lang.UnsatisfiedLinkError: Can't load library: /Users/klessard/.javacpp/cache/tensorflow-core-api-0.5.0-SNAPSHOT-macosx-arm64.jar/org/tensorflow/internal/c_api/macosx-arm64/libjnitensorflow.dylib
Debug: Loading library jnitensorflow
Debug: Failed to load for jnitensorflow: java.lang.UnsatisfiedLinkError: no jnitensorflow in java.library.path: [/Users/klessard/Library/Java/Extensions, /Library/Java/Extensions, /Network/Library/Java/Extensions, /System/Library/Java/Extensions, /usr/lib/java, .]
[ERROR] Tests run: 17, Failures: 0, Errors: 15, Skipped: 0, Time elapsed: 4.74 s <<< FAILURE! - in org.tensorflow.TensorTest
[ERROR] org.tensorflow.TensorTest.createFromBufferWithNonNativeByteOrder Time elapsed: 4.698 s <<< ERROR!
java.lang.UnsatisfiedLinkError: no jnitensorflow in java.library.path: [/Users/klessard/Library/Java/Extensions, /Library/Java/Extensions, /Network/Library/Java/Extensions, /System/Library/Java/Extensions, /usr/lib/java, .]
at org.tensorflow.TensorTest.createFromBufferWithNonNativeByteOrder(TensorTest.java:134)
Caused by: java.lang.UnsatisfiedLinkError: Can't load library: /Users/klessard/.javacpp/cache/tensorflow-core-api-0.5.0-SNAPSHOT-macosx-arm64.jar/org/tensorflow/internal/c_api/macosx-arm64/libjnitensorflow.dylib
at org.tensorflow.TensorTest.createFromBufferWithNonNativeByteOrder(TensorTest.java:134)
The JNI library is there and looks ok, here's the output of its otool -L
:
/Users/klessard/.javacpp/cache/tensorflow-core-api-0.5.0-SNAPSHOT-macosx-arm64.jar/org/tensorflow/internal/c_api/macosx-arm64/libjnitensorflow.dylib:
libjnitensorflow.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libtensorflow_cc.2.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.23.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)
I'll try to debug that but I've pushed too a temporary branch with my actual code, so if you are interested to give it a try @saudet . Don't forget that only works apparently on Xcode CL Tools 13.x, which might require to downgrade
Like some others I am in need to get Tensorflow-java running on an M1 based machine, certainly now that Apple has released a Tensorflow distribution for M1
[I know there is https://github.com/tensorflow/java/issues/252 but I want to revive the discussion after Apple's recent efforts]
Before even to attempt doing this I was wondering of any of the underlying strategies do make sense, or alternatively, do work
[This one fails based on the current HEAD. (java.lang.NoSuchMethodError: 'java.lang.Iterable com.sun.tools.javac.code.Scope$WriteableScope.getSymbolsByName(com.sun.tools.javac.util.Name, com.sun.tools.javac.util.Filter)'). It does not even gets to the TF native build phase]
Compile from source using x86 tools (e.g. in a "arch -x86_64 zsh" shell), taking into account specific guidelines e.g. remove usage of specific instruction sets. Consequently, run the java.jar using a x86 JVM, e.g. thus under Rossetta
Any other angle to look at the problem ?
[For that matter, how to leverage other ML frameworks on M1, e.g deeplearning4j ?]