Issues related to integrating syntaxnet/parsey mcparseface in Java + tf.reduce_sum

[marhlder]

Summary/tldr: It would be nice with more String support in the Java API and a tf.reduce_sum which supports string concatenation.

Body: I have successfully hacked in the custom ops of syntaxnet into the Tensorflow master as I wish to run syntaxnet from my Java NLP pipeline.

I do this by saving a SavedModelBundle and load it again from Java.

My project is based on a saved model of parsey_mcparseface from this branch: https://github.com/dmansfield/models/tree/documents-from-tensor By user @dmansfield.

And a tensorflow build based on the custom ops from: https://github.com/tensorflow/models/tree/master/syntaxnet Hacked into master of tensorflow.

Hacking in the ops, as build in ops, was necessary as the Java api through the JNI does not support user loaded ops yet (AFAIK).

The code with the syntaxnet ops hacked in can be found here: https://github.com/marhlder/syntaxnet-ops-hack

I have everything running except that the strings in the output tensor from my saved model includes junk (non-printable chars) which I think is due to the offset table described in tensor_jni.cc file.

I can run the same model from Python without any issues.

The Java API does currently not seem to support anything but scalar string tensors. It looks like using a scalar string tensor will solve my problem as that part of the codes seems to handle the offset table.

I would therefore like to reduce the string tensor produced by my parsey mcparseface model. I then found that the tf.reduce_sum does not yet support string concatenation.

There should already be an op for it: https://github.com/tensorflow/tensorflow/issues/701

User @mrry commented that he was looking for a use case for this here: https://stackoverflow.com/questions/34247374/merge-string-tensors-in-tensorflow

[mrry]

I can't speak to the Java API issues—@ashankar would know the answers here—but the tf.reduce_join() operation will concatenate a tf.string tensor along one or more dimensions (separated by an optional separator string), and I think it has the functionality you're looking for.

[marhlder]

Thx a lot. tf.reduce_join() worked :)

It seems like I still get junk chars in my output from my Java tensor even though i use the tensor.bytesValue() method to retrieve my scalar string.

The important characters seems to be ok.. My output looks like this: -:0I love grapes .
 
PRP2PRON:nsubj ove
VBP2VERB:ROOT grapes
NNS2NOUN:dobj

.2
.:punct -:1Do you even lift bro ? Do 
VBP2VERB:aux you PRP2PRON:nsubj even RB2ADV:advmod liftVB2VERB:ROOT bro NNP2NOUN:dobj
? 
.2
.:punct

[quaeler]

Can you provide a hex dump of the content including the garbage characters (their true values are getting obfuscated above - looks like they just got replaced with U+FFFD)?

[marhlder]

Sure:

0A032D3A30120F49206C6F766520677261706573202E1A1B0A01491000180020012A03505250320450524F4E3A056E7375626A1A1B0A046C6F7665100118042A035642503204564552423A04524F4F541A1F0A066772617065731006180B20012A034E4E5332044E4F554E3A04646F626A1A160A012E100D180D20012A012E32012E3A0570756E63740A032D3A311216446F20796F75206576656E206C6966742062726F203F1A1A0A02446F1000180120032A035642503204564552423A036175781A1D0A03796F751002180420032A03505250320450524F4E3A056E7375626A1A1D0A046576656E1006180920032A02524232034144563A066164766D6F641A1A0A046C696674100B180E2A0256423204564552423A04524F4F541A1C0A0362726F1010181220032A034E4E5032044E4F554E3A04646F626A1A160A013F1014181420032A012E32012E3A0570756E6374

Thx for taking a look at it 👍

[quaeler]

@marhlder let's put the thanks in the fridge until i actually do something of merit :- )

there are robust unit tests for Tensor.java, which pass on your syntaxnet-ops-hack repo (which i had a little time tonight to fork/clone/build/poke around) so i don't think what you're seeing is due to something deficient at the JNI implementation boundary.

I couldn't find your Java code running the parser model; in lieu of me implementing something like parser_eval.py in Java to debug the situation, might you check that code into your repo so that i could use it as a base to debug from?

[marhlder]

Hehe thx again ;)

I have pushed my java code :) The code is very proof of concept, but feel free to use it :)

My javac command line string looks like this: javac -cp .:/usr/share/java/junit4.jar:/usr/share/java/hamcrest-core.jar:/usr/share/java/libtensorflow.jar TensorflowModelWrapper.java TensorFlowModelWrapperTest.java

And running it looks like this: java -Dfile.encoding=utf-8 -cp .:/usr/share/java/junit4.jar:/usr/share/java/hamcrest-core.jar:/usr/share/java/libtensorflow.jar org.junit.runner.JUnitCore TensorFlowModelWrapperTest

I have included my compiled jni.so and .jar files which originates from my https://github.com/marhlder/syntaxnet-ops-hack repo :) (But you probly wanna compile them your self)

The SavedModelBundle is also included :)

The code used to build the model is a merge between https://github.com/dmansfield/models/tree/documents-from-tensor and https://github.com/tensorflow/models/tree/master/syntaxnet

Using a modified parsey_mcparseface.py source file from https://github.com/dmansfield/models/tree/documents-from-tensor

[marhlder]

I just tried running my program with a larger input and that makes it crash with the following message:

Error in `java': malloc(): memory corruption: 0x00007f3db800f0a0 ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f3e650577e5] /lib/x86_64-linux-gnu/libc.so.6(+0x8213e)[0x7f3e6506213e] /lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7f3e65064184] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(_Znwm+0x18)[0x7f3e461bde78] /lib/libtensorflow_jni.so(+0x1c8d49e)[0x7f3e2316d49e] /lib/libtensorflow_jni.so(+0x1c8f1d2)[0x7f3e2316f1d2] /lib/libtensorflow_jni.so(+0x1dc6391)[0x7f3e232a6391] /lib/libtensorflow_jni.so(+0x1dc3b8f)[0x7f3e232a3b8f] /lib/libtensorflow_jni.so(+0x1dc3c30)[0x7f3e232a3c30] /lib/libtensorflow_jni.so(+0x1dc3f63)[0x7f3e232a3f63] /lib/libtensorflow_jni.so(+0x1c9a6b4)[0x7f3e2317a6b4] /lib/libtensorflow_jni.so(+0x1c8fb74)[0x7f3e2316fb74] /lib/libtensorflow_jni.so(+0x1b922ec)[0x7f3e230722ec] /lib/libtensorflow_jni.so(+0x1b642df)[0x7f3e230442df] /lib/libtensorflow_jni.so(+0x1b515c5)[0x7f3e230315c5] /lib/libtensorflow_jni.so(+0x1dbf6c1)[0x7f3e2329f6c1] /lib/libtensorflow_jni.so(+0x1dbd7a7)[0x7f3e2329d7a7] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb8c80)[0x7f3e461e8c80] /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f3e657e76ba] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f3e650e73dd] ....... (Continues for ages) Aborted (core dumped)

[marhlder]

Might there be a memory management issue in the JNI code? Java_org_tensorflow_Tensor_scalarBytes? Looks like it might be releasing the byte array before the java code gets a chance it use it?

I'm not much into JNI but other snippets seems to parse the byte array to a method before they release it (i.e. giving the java program a chance to make a copy or something) E.g. https://stackoverflow.com/questions/24076624/how-to-pass-char-array-from-c-jni-function-to-java-method-as-byte

Hmm i guess release means something else in this context..

I just printed the string in the JNI code and it seems like there is problems already before the JNI code gets its hands on it.

I inserted a tf.Print op at the end of my graph in python before i export it to a SavedModelBundle and got the following output (I'm still running the model through Java):

[\n\003-:0\022$I love grapes.Do you even lift bro ?\032\033\n\001I\020\000\030\000 \001\003PRP2\004PRON:\005nsubj\032\033\n\004love\020\001\030\004\003VBP2\004VERB:\004ROOT\032!\n\tgrapes.Do\020\006\030\016 \001\002NN2\004NOUN:\004dobj\032\035\n\003you\020\020\030\022 \005\003PRP2\004PRON:\005nsubj\032\035\n\004even\020\024\030\027 \005\002RB2\003ADV:\006advmod\032\035\n\004lift\020\031\030\034 \002\002VB2\004VERB:\005ccomp\032\034\n\003bro\020\036\030 \005\003NNP2\004NOUN:\004dobj\032\026\n\001?\020\"\030\" \001\001.2\001.:\005punct]

[marhlder]

I'm just cleaning up the code i use to export the model. Will post it soon :)

[quaeler]

Ok - no rush; i'm just starting to spin up for the day now and will probably have a busy morning.

The tf.Print output in Python seems to jive with the hex dump you provided above as far as the 'junk' characters - though the placement of the whole of the second sentence is different. What happens between the steps of where you placed the tf.Print and where you were reporting seeing junk-less output from Python?

[marhlder]

The given output through tf.Print is the saved model running through the Java API. (I've edited my previous post to clarify). There still does not seem to be any issues when i run the model purely through Python. (That is run my modified parsey_mcparseface.py from https://github.com/dmansfield/models/tree/documents-from-tensor)

I think the difference between the two outputs i've posted (hex and tf.Print) is my own fault :)

The code I use to create and export my model to a SavedModelBundle can now be found here: https://github.com/marhlder/parsey_mcparseface_export

I run the script ./syntaxnet/export_model.sh from models/syntaxnet/

[quaeler]

Hmmm... i couldn't find export_model.sh, parsey_mcparseface.py nor either of TensorflowModelWrapper.java & TensorFlowModelWrapperTest.java in your https://github.com/marhlder/parsey_mcparseface_export repo. :- ?

(and could find no parsey_mcparseface.py in https://github.com/dmansfield/models/tree/documents-from-tensor )

((and i cannot find a models/syntaxnet directory pair - there is a .../syntaxnet/syntaxnet/models in Mansfield's repo - is this what you mean for this directory?))

i feel like i've totally misread what you've written after so many strikeouts. ?

[marhlder]

Sorry my bad :) Seems i did not push to the newly created remote. Its on a branch called documents-from-tensor. parsey_mcparseface.py is in syntaxnet/syntaxnet on https://github.com/marhlder/parsey_mcparseface_export/tree/documents-from-tensor/syntaxnet/syntaxnet.

The java files are in https://github.com/marhlder/syntaxnet-ops-hack/tree/syntaxnet-ops-hack in a folder called run_saved_model_java

I think my issues might be related to how parsey_mcparseface.py is calling gen_parser_ops.document_sink() in the case where i'm not exporting the model. Will investigate

[quaeler]

Now we're cooking with grease... was able to fork/clone/build locally and get the junk characters you see. Will start anatomizing.

[marhlder]

Ok, i got it working by making a new custom op instead of the syntaxnet DocumentSink op i've added my own DocumentStringSink op which makes it possible to output a string tensor instead of a file. Will push my code soon.

Lets close this issue and continue on the linked repositories :)

[quaeler]

Very glad you got a working solution - i don't see a problem with how you expected your original implementation to work though, so i'd still like to determine why it didn't work as written, if you don't mind.

I'll look for the code pushes; i noticed when i tried to build the target //tensorflow/tools/pip_package:build_pip_package in your syntaxnet-ops-hack branch (which i merged down to my master fork), there are dependencies in there somewhere which end up building both protobuf_archive/libprotobuf.a and com_google_protobuf/libprotobuf.a At link time for the subtarget //tensorflow/python:_pywrap_tensorflow_internal.so, the linker isn't at all happy about finding the duplicate symbols. If this is a 'forgot to push something' type of situation, please do push -- but if this is a 'works for me' situation, please don't waste time on it. :- )

[marhlder]

I have not tried to build the pip package for some time as it is not necessary for my objective of getting it to run through Java. I will have a look at it 👍

I think the extra "junk" bytes are something used internally in the syntaxnet code and that the code from https://github.com/dmansfield/models/tree/documents-from-tensor, merged with the newest code from the syntaxnet repo, did not take this into account.

[quaeler]

I agree with that conclusion - that the operations being done in the DocumentSink and DocumentStringSink operators are excluding those junk characters in a way that tf.reduce_join was not cognizant of.

As far as not building the pip package, were this 1690s Salem they might look at you askance -- by what black magic were you able to add the DocumentStringSink operator to document_filters.cc / parser_ops.cc the other day, and then successfully reference it in your parsey_mcparseface.py?

[gbolcer]

Ubuntu 17.10, Python 3.6.2, Syntaxnet-ops-hack main branch, cpu only, java 9.0.1, Eclipse 4.7.1a.

Just for the record I can build both the Tensorflow 1.4 cpu and gpu and pip packages in the normal tensorflow branch, but I need and want to use Syntaxnet in Java as part of my toolchain integration with other software. Those libraries will obviously complain about not having the Syntaxnet Ops (FeatureSize, etc).

Issues:

http://www.bolcer.org/java-syntaxnet.png

1) Large inputs will crash Java (not a big deal as I can segment/page my input to get around this).

2) Output character encoding and/or bytes. In my Eclipse Run Configuration-->Common console I tried several different encodings. It defaults to inherited-->utf-8.

I'm going to try to export my own local model as the next step.

Tried the default one that comes w/ the 1.4 Tensorflow branch.

tensorflow/tensorflow/contrib/syntaxnet/models/parsey_mcparseface/context.pbtxt https://github.com/tensorflow/tensorflow r1.4

No difference. So, DocumentStringSink? Also looking at TensorTest.java.

[sarda014]

@gbolcer I also see some junk characters when I use the saved_model from https://github.com/unsiloai/syntaxnet-ops-hack/tree/syntaxnet-ops-hack/run_saved_model_java/SAVED_MODEL

Were you able to export your own model and get this working?

@marhlder Will it be possible for you to share the modified parsey_mcparseface.py source file from https://github.com/dmansfield/models/tree/documents-from-tensor which you used to generate the model?

[gbolcer]

It's been a while and I have been using a docker image as a temporary solution. Let me try to recreate it.

[marhlder]

For any one still interested in this, I currently use the syntaxnet ops as a seperate lib of custom ops which I then include into tensorflow. The code works with the prebuild java binaries distributed by TensorFlow. It probably also works for the python binaries installed with pip.

The code can be found in this branch: https://github.com/unsiloai/syntaxnet-ops-hack/tree/separate_lib.

See: https://www.tensorflow.org/extend/adding_an_op, https://www.tensorflow.org/extend/adding_an_op#compile_the_op_using_bazel_tensorflow_source_installation.

The library can be build using: ./configure bazel build --config opt \ //tensorflow/java:tensorflow \ //tensorflow/java:libtensorflow_jni bazel build --config opt //tensorflow/user_ops:syntaxnet_dragnn_ops.so

For exporting the model see: https://github.com/unsiloai/parsey_mcparseface_export/tree/documents-from-tensor/syntaxnet/syntaxnet https://github.com/unsiloai/parsey_mcparseface_export/tree/documents-from-tensor/syntaxnet/syntaxnet/export_model.sh

[gbolcer]

Thanks, will give it a try. Still very interested in getting it working the way I need it.

Okay, first try I ran into this error here. https://github.com/bazelbuild/bazel/issues/4652

I deleted all mentions of the armeabi-v7a text wherever it complained about it, and it looks like it's generating the .so and .jar files. Will test the sample java program next.....

Plain old Tensorflow (https://github.com/tensorflow/tensorflow.git) compiles out of the box on my platform w/ the configuration below, including the java:tensorflow java:libtensorflow_jni, and //tensorflow/tools/pip_package:build_pip_package

Okay, bazel version 0.15.0

Ubuntu (lsb_release -a) No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04 LTS Release: 18.04 Codename: bionic

gcc --version gcc (Ubuntu 7.3.0-16ubuntu3) 7.3.0

cd tensorflow git describe --tags v1.8.0-4356-g2f7609d95d

[marhlder]

I have re-enabled notifications for this issue/thread, but you are also welcome to contact me more directly if I can be of help.

[gbolcer]

Okay, new problems on this command.

bazel build --config opt //tensorflow/user_ops:syntaxnet_dragnn_ops.so greg@salt:~/code/syntaxnet-ops-hack-separate_lib$ bazel build --config opt //tensorflow/user_ops:syntaxnet_dragnn_ops.so WARNING: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/core/BUILD:1816:1: in includes attribute of cc_library rule //tensorflow/core:framework_headers_lib: '../../external/nsync/public' resolves to 'external/nsync/public' not below the relative path of its package 'tensorflow/core'. This will be an error in the future. Since this rule was created by the macro 'cc_header_only_library', the error might have been caused by the macro implementation in /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/tensorflow.bzl:1143:30 ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/syntaxnet/BUILD:13:1: First argument of 'load' must be a label and start with either '//', ':', or '@'. ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/syntaxnet/BUILD:13:1: file 'syntaxnet' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/syntaxnet/BUILD:35:1: name 'tf_proto_library' is not defined (did you mean 'cc_proto_library'?) ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/syntaxnet/BUILD:40:1: name 'tf_proto_library' is not defined (did you mean 'cc_proto_library'?) ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/syntaxnet/BUILD:45:1: name 'tf_proto_library' is not defined (did you mean 'cc_proto_library'?) ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/syntaxnet/BUILD:56:1: name 'tf_proto_library' is not defined (did you mean 'cc_proto_library'?) ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/syntaxnet/BUILD:66:1: name 'tf_proto_library' is not defined (did you mean 'cc_proto_library'?) ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/syntaxnet/BUILD:76:1: name 'tf_proto_library' is not defined (did you mean 'cc_proto_library'?) ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/dragnn/tensorflow_ops.bzl:537:12: name 'set' is not defined ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/BUILD:16:1: Target '//tensorflow/user_ops/syntaxnet:document_filters' contains an error and its package is in error and referenced by '//tensorflow/user_ops:syntaxnet_dragnn_ops.so' ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/BUILD:16:1: Target '//tensorflow/user_ops/syntaxnet:lexicon_builder' contains an error and its package is in error and referenced by '//tensorflow/user_ops:syntaxnet_dragnn_ops.so' ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/BUILD:16:1: Target '//tensorflow/user_ops/syntaxnet:reader_ops' contains an error and its package is in error and referenced by '//tensorflow/user_ops:syntaxnet_dragnn_ops.so' ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/BUILD:16:1: Target '//tensorflow/user_ops/syntaxnet:unpack_sparse_features' contains an error and its package is in error and referenced by '//tensorflow/user_ops:syntaxnet_dragnn_ops.so' ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/BUILD:16:1: Target '//tensorflow/user_ops/syntaxnet:registry' contains an error and its package is in error and referenced by '//tensorflow/user_ops:syntaxnet_dragnn_ops.so' ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/BUILD:16:1: error loading package 'tensorflow/user_ops/dragnn': Extension 'tensorflow/user_ops/dragnn/tensorflow_ops.bzl' has errors and referenced by '//tensorflow/user_ops:syntaxnet_dragnn_ops.so' ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/BUILD:16:1: error loading package 'tensorflow/user_ops/dragnn': Extension 'tensorflow/user_ops/dragnn/tensorflow_ops.bzl' has errors and referenced by '//tensorflow/user_ops:syntaxnet_dragnn_ops.so' ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/BUILD:16:1: error loading package 'tensorflow/user_ops/dragnn': Extension 'tensorflow/user_ops/dragnn/tensorflow_ops.bzl' has errors and referenced by '//tensorflow/user_ops:syntaxnet_dragnn_ops.so' ERROR: /home/greg/code/syntaxnet-ops-hack-separate_lib/tensorflow/user_ops/BUILD:16:1: error loading package 'tensorflow/user_ops/dragnn': Extension 'tensorflow/user_ops/dragnn/tensorflow_ops.bzl' has errors and referenced by '//tensorflow/user_ops:syntaxnet_dragnn_ops.so' ERROR: Analysis of target '//tensorflow/user_ops:syntaxnet_dragnn_ops.so' failed; build aborted: error loading package 'tensorflow/user_ops/dragnn': Extension 'tensorflow/user_ops/dragnn/tensorflow_ops.bzl' has errors INFO: Elapsed time: 0.614s INFO: 0 processes. FAILED: Build did NOT complete successfully (3 packages loaded) currently loading: tensorflow/user_ops/dragnn greg@salt:~/code/syntaxnet-ops-hack-separate_lib$

[marhlder]

I do not think the bazel scripts are compatible with 0.15.0

You are likely to have more luck with 0.8.0

[gbolcer]

Thanks! I just changed all the set= to depset=, I also commented out the syntaxnet reference in the load command as it seemed to already have it accessible.

It seems to have generated the file.

tensorflow/user_ops/syntaxnet/workspace.h: In instantiation of 'int syntaxnet::WorkspaceRegistry::Request(const string&) [with W = syntaxnet::VectorVectorIntWorkspace; std::cxx11::string = std::cxx11::basic_string]': tensorflow/user_ops/syntaxnet/sentence_features.h:139:77: required from here tensorflow/user_ops/syntaxnet/workspace.h:61:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] Target //tensorflow/user_ops:syntaxnet_dragnn_ops.so up-to-date: bazel-bin/tensorflow/user_ops/syntaxnet_dragnn_ops.so INFO: Elapsed time: 27.009s, Critical Path: 11.15s INFO: 86 processes: 86 local. INFO: Build completed successfully, 91 total actions greg@salt:~/code/syntaxnet-ops-hack-separate_lib$

On Mon, Jul 2, 2018 at 2:27 PM, Martin Damgaard Nielsen < notifications@github.com> wrote:

I do not think the bazel scripts are compatible with 0.15.0

You are likely to have more luck with 0.8.0

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/12328#issuecomment-401941075, or mute the thread https://github.com/notifications/unsubscribe-auth/ASZzSGG_ZV09q2K7_JIyNG50KBQVN6KVks5uCpBIgaJpZM4O47VJ .

-- greg@bolcer.org, http://bolcer.org, c: +1.714.928.5476

[gbolcer]

Little further...now I just need to export the model.

Loading model.... 2018-07-02 15:57:17.117391: I tensorflow/core/platform/s3/aws_logging.cc:53] Initializing Curl library 2018-07-02 15:57:17.133986: I tensorflow/cc/saved_model/loader.cc:240] Loading SavedModel with tags: { serve }; from: /home/greg/Insync/Working/tensorlibs/run_saved_model_java/SAVED_MODEL/ 2018-07-02 15:57:17.188982: I tensorflow/cc/saved_model/loader.cc:289] SavedModel load for tags { serve }; Status: fail. Took 54988 microseconds. Exception in thread "main" org.tensorflow.TensorFlowException: Op type not registered 'FeatureSize' in binary running on salt. Make sure the Op and Kernel are registered in the binary running in this process. at org.tensorflow.SavedModelBundle.load(Native Method) at org.tensorflow.SavedModelBundle.load(SavedModelBundle.java:39) at crafty.bear.tensorbuilder.syntaxnet.TensorflowModelWrapper.(TensorflowModelWrapper.java:15) at crafty.bear.tensorbuilder.syntaxnet.ParseyMcParsefaceWrapper.(ParseyMcParsefaceWrapper.java:14) at crafty.bear.tensorbuilder.syntaxnet.TensorFlowModelWrapperTest.testSyntaxnet(TensorFlowModelWrapperTest.java:54) at crafty.bear.tensorbuilder.parsley.Parsely.main(Parsely.java:9)

[sarda014]

Thanks @marhlder , @gbolcer. I was able to build the syntaxnet_dragnn_ops binary, and use the trained models from https://github.com/unsiloai/parsey_mcparseface_export/tree/documents-from-tensor/syntaxnet/SAVED_MODEL to run inference.

Btw, I am using bazel 0.5.4.

[gbolcer]

I'm almost there too. I didn't build it right the first time, but it looks like it'll build correctly this time. Thanks for the link to the model. I ran into some java9 issues too. I had to add the following:

--java_toolchain=@bazel_tools//tools/jdk:toolchain_java9 --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_java9 --jobs 1

[gbolcer]

Okay, I got as far as I could. I had to do it on Windows at home. This is the final error message. I think it could be that it's not linking correctly on windows. I'll try it on my Ubuntu box when I get back into the office on Monday.

Greg

org.tensorflow.TensorFlowException: Op type not registered 'FeatureSize' in binary running on FX11. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) tf.contrib.resampler should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed. at org.tensorflow.SavedModelBundle.load(Native Method) at org.tensorflow.SavedModelBundle.load(SavedModelBundle.java:39) at crafty.bear.tensorbuilder.syntaxnet.TensorflowModelWrapper.(TensorflowModelWrapper.java:15) at crafty.bear.tensorbuilder.syntaxnet.ParseyMcParsefaceWrapper.(ParseyMcParsefaceWrapper.java:14) at crafty.bear.tensorbuilder.syntaxnet.TensorFlowModelWrapperTest.testSyntaxnet(TensorFlowModelWrapperTest.java:54) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:564) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:538) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:760) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:460) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:206)

[marhlder]

Did you build and dynamicly load the op library before loading the saved model?

On Fri, Jul 6, 2018, 22:47 gbolcer notifications@github.com wrote:

Okay, I got as far as I could. I had to do it on Windows at home. This is the final error message. I think it could be that it's not linking correctly on windows. I'll try it on my Ubuntu box when I get back into the office on Monday.

Greg

org.tensorflow.TensorFlowException: Op type not registered 'FeatureSize' in binary running on FX11. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) tf.contrib.resampler should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed. at org.tensorflow.SavedModelBundle.load(Native Method) at org.tensorflow.SavedModelBundle.load(SavedModelBundle.java:39) at crafty.bear.tensorbuilder.syntaxnet.TensorflowModelWrapper.(TensorflowModelWrapper.java:15) at crafty.bear.tensorbuilder.syntaxnet.ParseyMcParsefaceWrapper.(ParseyMcParsefaceWrapper.java:14) at crafty.bear.tensorbuilder.syntaxnet.TensorFlowModelWrapperTest.testSyntaxnet(TensorFlowModelWrapperTest.java:54) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:564) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:538) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:760) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:460) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:206)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/12328#issuecomment-403141309, or mute the thread https://github.com/notifications/unsubscribe-auth/ACkL71DQicYkDsV39RevFZBHuW0Xx8Zlks5uD8zZgaJpZM4O47VJ .

[gbolcer]

I think the issue is that I had to build the whole stack statically with --config=monolithic and some of my changes to get it to compile didn't link correctly even w/ --linkopts.

I did also find a difference in REGISTER_OP

REGISTER_OP("Fact").Output("fact: string");

versus

REGISTER_OP("Fact") .Output("fact: string") .SetShapeFn(tensorflow::shape_inference::UnknownShape);

Looking in to see if that is related.

Thanks,

Greg

On 7/6/2018 3:45 PM, Martin Damgaard Nielsen wrote:

Did you build and dynamicly load the op library before loading the saved model?

[gbolcer]

Reverted to Java8 and Python27. That eliminated all the cruft from the alternate configurations. I also had to specify --config=monolithic (on windows), but it all compiled and linked. I'm compiling the OP now. This is farther than I've gotten w/o errors, so it's looking promising.

bazel build --config opt --config=monolithic //tensorflow/java:tensorflow //tensorflow/java:libtensorflow_jni

Creating library bazel-out/x64_windows-opt/bin/tensorflow/java/libtensorflow_jni.so.if.lib and object bazel-out/x64_windows-opt/bin/tensorflow/java/libtensorflow_jni.so.if.exp batch_kernels.lo.lib(batch_kernels.obj) : warning LNK4217: locally defined symbol ?DEVICE_CPU@tensorflow@@3QEBDEB (char const const tensorflow::DEVICE_CPU) imported in function "void cdecl tensorflow::`dynamic initializer for 'registrarbody0object''(void)" (??Eregistrarbody0object@tensorflow@@YAXXZ) arithmetic_optimizer.lib(arithmetic_optimizer.obj) : warning LNK4049: locally defined symbol ?DEVICE_CPU@tensorflow@@3QEBDEB (char const const tensorflow::DEVICE_CPU) imported arithmetic_optimizer.lib(arithmetic_optimizer.obj) : warning LNK4217: locally defined symbol ?DEVICE_GPU@tensorflow@@3QEBDEB (char const const tensorflow::DEVICE_GPU) imported in function "private: bool __cdecl tensorflow::grappler::`anonymous namespace'::ReorderCastAndTranspose::NodeIsOnCpuOrGpu(class tensorflow::NodeDef const )const " (?NodeIsOnCpuOrGpu@ReorderCastAndTranspose@?A0xa1632fa2@grappler@tensorflow@@AEBA_NPEBVNodeDef@4@@Z) layout_optimizer.lib(layout_optimizer.obj) : warning LNK4049: locally defined symbol ?DEVICE_GPU@tensorflow@@3QEBDEB (char const * const tensorflow::DEVICE_GPU) imported INFO: Elapsed time: 2293.212s, Critical Path: 470.55s INFO: 1893 processes: 1892 local, 1 worker. INFO: Build completed successfully, 1962 total actions

[gbolcer]

Hmmm...BUILD file is on the build path, so not sure why.

S C:\code\syntaxnet-ops-hack> bazel build --config opt //tensorflow/user_ops:syntaxnet_dragnn_ops.so ERROR: Skipping '//tensorflow/user_ops:syntaxnet_dragnn_ops.so': no such package 'tensorflow/user_ops': BUILD file not found on package path WARNING: Target pattern parsing failed. ERROR: no such package 'tensorflow/user_ops': BUILD file not found on package path INFO: Elapsed time: 0.959s INFO: 0 processes. FAILED: Build did NOT complete successfully (0 packages loaded) PS C:\code\syntaxnet-ops-hack>

[gbolcer]

Windows issue with Variadic Macros. Added a comma before the ellipsis in char_properties.h:127 and that seems to have fixed it.

ERROR: C:/code/syntaxnet-ops-hack/tensorflow/core/user_ops/syntaxnet/BUILD:268:1: C++ compilation of rule '//tensorflow/core/user_ops/syntaxnet:sentence_features' failed (Exit 2) c:\users\greg_bazel_greg\mzruyaxg\execroot\org_tensorflow\tensorflow\core\user_ops\syntaxnet\char_properties.h(127): error C2010: '.': unexpected in macro formal parameter list cl : Command line warning D9002 : ignoring unknown option '-march=native' cl : Command line warning D9002 : ignoring unknown option '-march=native' Target //tensorflow/core/user_ops:syntaxnet_dragnn_ops.so failed to build Use --verbose_failures to see the command lines of failed build steps. INFO: Elapsed time: 6.184s, Critical Path: 2.82s INFO: 3 processes: 3 local. FAILED: Build did NOT complete successfully PS C:\code\syntaxnet-ops-hack\tensorflow>

[gbolcer]

File filter "readable" issue.

ERROR: C:/code/syntaxnet-ops-hack/tensorflow/python/BUILD:3698:1: Executing genrule //tensorflow/python:pywrap_tensorflow_filtered_def_file failed (Exit 1): bash.exe failed: error executing command cd C:/users/greg/_bazel_greg/mzruyaxg/execroot/org_tensorflow SET PATH=C:\git\usr\bin;C:\git\bin;C:\cuda\bin;C:\cuda\libnvvp;C:\Program Files (x86)\Razer Chroma SDK\bin;C:\Program Files\Razer Chroma SDK\bin;C:\alt\java8\bin;C:\alt\p2\Scripts\;C:\gcc\bin;C:\alt\p2\;C:\git\bin;C:\git\usr\bin;C:\Windows;C:\Windows\System32;C:\Windows\System32\Wbem;C:\Users\greg\AppData\Local\Microsoft\WindowsApps;C:\Program Files\ImageMagick-7.0.2-Q16;C:\Program Files\TortoiseHg\;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Windows Live\Shared;C:\Program Files (x86)\EaseUS\Todo Backup\bin;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\Skype\Phone;C:\Program Files (x86)\QuickTime\QTSystem\;C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\Hostx64\x64;C:\Program Files\PuTTY\;C:\Program Files (x86)\Skype\Phone\;C:\Program Files\Gource\cmd;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Users\greg\AppData\Local\Microsoft\WindowsApps; SET PYTHON_BIN_PATH=C:/alt/p2/python.exe SET PYTHON_LIB_PATH=C:/alt/p2/lib/site-packages SET TF_NEED_CUDA=0 SET TF_NEED_OPENCL=0 C:/git/usr/bin/bash.exe -c source external/bazel_tools/tools/genrule/genrule-setup.sh; bazel-out/host/bin/external/local_config_def_file_filter/def_file_filter.exe \ --input bazel-out/x64_windows-opt/bin/tensorflow/python/tf_custom_op_library_additional_deps.so.def \ --output bazel-out/x64_windows-opt/genfiles/tensorflow/python/pywrap_tensorflow_filtered_def_file.def \ --target _pywrap_tensorflow_internal.pyd

Traceback (most recent call last): File "c:\users\greg\appdata\local\temp\Bazel.runfiles_7__hrg\runfiles\local_config_def_file_filter\def_file_filter.py", line 168, in sys.exit(main()) File "c:\users\greg\appdata\local\temp\Bazel.runfiles_7__hrg\runfiles\local_config_def_file_filter\def_file_filter.py", line 129, in main for idx, line in enumerate(io.TextIOWrapper(proc.stdout, encoding="utf-8")): AttributeError: 'file' object has no attribute 'readable' Target //tensorflow/core/user_ops:syntaxnet_dragnn_ops.so failed to build INFO: Elapsed time: 6.765s, Critical Path: 2.20s INFO: 1 process: 1 local. FAILED: Build did NOT complete successfully PS C:\code\syntaxnet-ops-hack\tensorflow>

[gbolcer]

Substituting _getpid() for getpid() on windows....and success! Testing out the wrapper code in the morning....

Creating library bazel-out/x64_windows-opt/bin/tensorflow/core/user_ops/syntaxnet_dragnn_ops.so.if.lib and object bazel-out/x64_windows-opt/bin/tensorflow/core/user_ops/syntaxnet_dragnn_ops.so.if.exp Target //tensorflow/core/user_ops:syntaxnet_dragnn_ops.so up-to-date: C:/users/greg/_bazel_greg/mzruyaxg/execroot/org_tensorflow/bazel-out/x64_windows-opt/bin/tensorflow/core/user_ops/syntaxnet_dragnn_ops.so INFO: Elapsed time: 111.389s, Critical Path: 52.46s INFO: 162 processes: 162 local. INFO: Build completed successfully, 165 total actions PS C:\code\syntaxnet-ops-hack\tensorflow>

[marhlder]

I am glad you got something working. I sorta got it to work with r1.9 of TensorFlow. Except i keep getting a symbolic error: "undefined symbol: _ZN10tensorflow7strings6StrCatB5cxx11ERKNS0_8AlphaNumE"

[gbolcer]

I've seen that error before. What version of gcc are you using? I think I ended up installing gcc-4, gcc-5, gcc-6 (and later gcc-7), but I'm not sure which one fixed it.

Just for grins, try adding --copts=" -D_GLIBCXX_USE_CXX11_ABI=0" to your bazel command. (Or it might be --cxxopts and --linkopts)

[gbolcer]

I'm stuck at the point that I either can't load the model or if I use System.load explicitly, it complains about undefined symbols.

java.lang.UnsatisfiedLinkError: /mnt/hbase/Insync/Working/tensorlibs/linux/syntaxnet-ops/user_ops/syntaxnet_dragnn_ops.so: /mnt/hbase/Insync/Working/tensorlibs/linux/syntaxnet-ops/user_ops/syntaxnet_dragnn_ops.so: undefined symbol: _ZN10tensorflow10DEVICE_CPUE

I can't seem to get it to load parser_ops for some reason. I can't find a dep in the BUILD files.

[gbolcer]

Now I'm really confused. bazel clean and then re-build w/o the monolithic tag. The java appears to have compiled and loaded the model. Now to run some data through it.

2018-07-12 14:07:16.220721: E tensorflow/core/framework/op_kernel.cc:1242] OpKernel ('op: "Fact" device_type: "CPU" label: "sergey"') for unknown op: Fact 2018-07-12 14:07:16.220817: E tensorflow/core/framework/op_kernel.cc:1242] OpKernel ('op: "Fact" device_type: "CPU" label: "Sergey"') for unknown op: Fact 2018-07-12 14:07:16.220832: E tensorflow/core/framework/op_kernel.cc:1242] OpKernel ('op: "Fact" device_type: "GPU" host_memory_arg: "fact"') for unknown op: Fact 2018-07-12 14:07:16.244364: I tensorflow/cc/saved_model/loader.cc:242] Loading SavedModel with tags: { serve }; from: /home/greg/Insync/Working/tensorlibs/run_saved_model_java/SAVED_MODEL/ 2018-07-12 14:07:16.346255: I tensorflow/cc/saved_model/loader.cc:161] Restoring SavedModel bundle. 2018-07-12 14:07:17.012545: I tensorflow/cc/saved_model/loader.cc:196] Running LegacyInitOp on SavedModel bundle. 2018-07-12 14:07:17.015533: I tensorflow/cc/saved_model/loader.cc:291] SavedModel load for tags { serve }; Status: success. Took 771200 microseconds.

[gbolcer]

Getting very close. Looking at an input check error coming from somewhere. It seems to be embedded in the model I'm using. I'll have to dump a new one.

Model loaded... Fetching session... Input is: 683 About to run on 682 chars 2018-07-12 14:38:49.776439: F tensorflow/core/user_ops/syntaxnet/proto_io.h:277] Check failed: input.record_format_size() == 1 (0 vs. 1)TextWriter only supports files with one record format: name: "wsj-data-parsed"

[gbolcer]

Bummer, now I'm back where I started. But at least I know the software seems to be working, I just need to go back and experiment with the models and output again.

University NNP2NOUN:nn Faculty! NNP2NOUN:nn Union#' NNP2NOUN:nn Wants)- NNPS2NOUN:nsubj Trustees/6VBZ2VERB:ROOT To89 TO2PRT:prep! Resign\n;B NNP2NOUN:pobj ''DE ''2
.:punct

[gbolcer]

I hope you guys don't mind me posting my work-in-progress here. I got to the point where there seems to be some output file type when I invoke the commands with my stored model. I think I can tweak my stored context.pbtxt file with the right values related to this:

https://github.com/tensorflow/models/issues/257

Loading model.... 2018-07-16 11:42:30.316949: E tensorflow/core/framework/op_kernel.cc:1242] OpKernel ('op: "Fact" device_type: "CPU" label: "sergey"') for unknown op: Fact 2018-07-16 11:42:30.317013: E tensorflow/core/framework/op_kernel.cc:1242] OpKernel ('op: "Fact" device_type: "CPU" label: "Sergey"') for unknown op: Fact 2018-07-16 11:42:30.317026: E tensorflow/core/framework/op_kernel.cc:1242] OpKernel ('op: "Fact" device_type: "GPU" host_memory_arg: "fact"') for unknown op: Fact 2018-07-16 11:42:30.319997: I tensorflow/cc/saved_model/loader.cc:242] Loading SavedModel with tags: { serve }; from: /home/greg/Desktop/platform/tensorbuilder/models/SAVED_MODEL/ 2018-07-16 11:42:30.366281: I tensorflow/cc/saved_model/loader.cc:161] Restoring SavedModel bundle. 2018-07-16 11:42:30.522776: I tensorflow/cc/saved_model/loader.cc:196] Running LegacyInitOp on SavedModel bundle. 2018-07-16 11:42:30.526127: I tensorflow/cc/saved_model/loader.cc:291] SavedModel load for tags { serve }; Status: success. Took 206136 microseconds. Model loaded... Fetching session... Input is: 68 Sentence is: "Chicago State University Faculty Union Wants Trustees To Resign\n" Running final parse. inputBuilder: "Chicago State University Faculty Union Wants Trustees To Resign\n"

About to run on 68 chars After Tensor After runner Output op names: output 2018-07-16 11:42:32.088690: F tensorflow/core/user_ops/syntaxnet/proto_io.h:277] Check failed: input.record_format_size() == 1 (0 vs. 1)TextWriter only supports files with one record format: name: "wsj-data-parsed"

[sarda014]

@marhlder I am trying to export the model using https://github.com/unsiloai/parsey_mcparseface_export/tree/documents-from-tensor/ by doing the following -

1. Build syntaxnet bazel test --linkopt=-headerpad_max_install_names dragnn/... syntaxnet/... util/utf8/...

2. Build pip package bazel-bin/dragnn/tools/build_pip_package --include-tensorflow --output-dir=/tmp/syntaxnet_pkg

3. Install pip package pip install /tmp/syntaxnet_pkg/syntaxnet*.whl

4. Run export_model.sh

But, I keep getting the following error - "Op type not registered 'FeatureSize'". Do these steps look alright to you, or am I missing something? Thanks in advance :)

[marhlder]

@sarda014 I think there are multiple issues: It does not seem like you are building TensorFlow with "bazel-bin/dragnn/tools/build_pip_package --include-tensorflow --output-dir=/tmp/syntaxnet_pkg" but rather a separate package which includes tensorflow. "export_model.sh" relies on the currently installed TensorFlow package on not on a dragnn package.

[thai2019]

@marhlder I want to install this package, parsey_mcparseface_export, and then export a modified syntaxnet graph to protobuf. Do you have a complete installation procedure I can follow? Thanks.