search

tensorflow / java

Java bindings for TensorFlow
Apache License 2.0
831 stars 202 forks source link

r0.5 - tensorflow-core-api compilation - "cannot find symbol" for enums defined in C code of TensorFlow #584

Open lvsubbu opened 6 hours ago

lvsubbu commented 6 hours ago

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

System information

Describe the current behavior I successfully compiled and installed Javacpp package and Tensorflow 2.14.1 too. But when compiling tensorflow-core-api in java (r0.5), by mvn clean install -DskipTests -X, I am getting "cannot find symbol" for ALL those enums defined in Tensorflow C code.

For example TF_TString_Type is enum defined in tensorflow/tsl/platform/ctstring_internal.h

typedef enum TF_TString_Type {
  TF_TSTR_SMALL = 0,

The corresponding Java-generated code is as below

/** enum TF_TString_Type */
public static final int

public static native @ByVal TF_TString_Type TF_StringGetType(@Const TF_TString str);

All the struct types in C, I see Java class got generated, and no issues. But the enum is treated as int, that is fine. But the enum symbol is not resolved during the compilation

But I am getting error for it, [ERROR] symbol: class TSL_Code [ERROR] location: class tensorflow [ERROR] /tf_install/tensorflow/java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[565,28] error: cannot find symbol [ERROR] symbol: class TF_TString_Type [ERROR] location: class tensorflow [ERROR] /tf_install/tensorflow/java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[4957,105] error: cannot find symbol [ERROR] symbol: class Status [ERROR] location: class tensorflow [ERROR] /tf_install/tensorflow/java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[4960,46] error: cannot find symbol [ERROR] symbol: class Status [ERROR] location: class tensorflow [ERROR] /tf_install/tensorflow/java/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/internal/c_api/AbstractTF_Tensor.java:[24] error: cannot find symbol [ERROR] symbol: static TF_TString_Dealloc [ERROR] location: class tensorflow [ERROR] symbol: class TF_TString_Type [ERROR] location: class tensorflow [ERROR] /tf_install/tensorflow/java/tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/internal/c_api/global/tensorflow.java:[4957,105] error: cannot find symbol

Describe the expected behavior No error

Code to reproduce the issue compile the tensorflow-core-api using "mvn clean install"

Other info / logs

Craigacp commented 6 hours ago

Can you try 1.0.0-rc.2? JavaCPP is clearly generating something odd, but to get to TF 2.14.2 we had to modify the JavaCPP generation code and those modifications are not in v0.5.0.

saudet commented 4 hours ago

@Craigacp What is this bug exactly?

Craigacp commented 4 hours ago

I think they are trying to take TF-Java v0.5.0 and upgrade it from TF 2.10 to TF 2.14 on an IBM mainframe? We have commits which did the 2.10->2.14 upgrade to ignore additional things from absl that got pulled in which JavaCPP failed to parse correctly, plus that was the start of Google moving a bunch of things from TF into TSL which caused all the header files to move. I don't know if equivalent changes have been applied to their codebase, but they need to do something like this https://github.com/tensorflow/java/commit/ac5c5601082c01a9b9f7cd696bf6575eca8d259b#diff-7e8b03ae7258b7fc83f09b902064f92b2388e09185675f985c66c0a857a71979.

It would be much better to move to TF-Java 1.0.0-rc.2, but I have no idea if TF 2.16.2 would run on an S390x. We did substantially change the build system which will likely make it harder to build TF-Java on that platform due to a poor interaction between bazel and the C++ shim we use to hide some TF C++ files which are difficult to call via the extracted interface.

saudet commented 3 hours ago

@Craigacp I see, things like absl are pretty hard to parse, and currently you're just skipping what you're not using anyway, but if you do need to start using some of that stuff, let me know.

@lvsubbu There's been some bugs fixed with enum in JavaCPP since 1.5.8, so please try again with >= 1.5.10