When this project will implement data_flow_grad data_flow_ops resource_variable_ops sparse_ops ragged_functional_ops

[mullerhai]

HI : these api in java not find ,but it is necessary for the user, I think if we take thses api to real world in tensorflow -java ,it is will friendly for user thx

[Craigacp]

We're working on building out Java implementations of the ops written in Python but there are a lot and it's not finished yet. If you've built a model using those ops in Python then that model will work in Java.

Is there something specific from those ops that you need? They are usually composed out of more primitive ops that are already available in TF-Java so the functionality is available but harder to access.

[mullerhai]

as you know these Ops has some import method for use ,like DataFlow dynamicPartition, dynamicStitch transformIds colocateWith clipByNorm,without these method ,I can not do some model like deepfm

[Craigacp]

DeepFM should be implementable entirely within TF-Java without using any of those operations, it's just a pairwise feature interaction using an embedding vector and an MLP. Is there a specific implementation that you'd like to port into Java? Some of the ops you mention would be used for some scale out variants, but we don't support models across multiple devices in TF-Java, so there isn't a good notion of placement for us to expose. Norm clipping is a composite op of things that we already have so at least that is straightforward. As I said, we're slowly building out the functionality, but there's lots to build. If there are composite ops you need then we'd welcome contributions to add them.

[YujieW0201]

Hi! Are you planning to add custom ops for TensorFlow Decision Forests (TF-DF)? https://github.com/tensorflow/decision-forests I want to load my from from TF-DF and got error message: Op type not registered 'SimpleMLCreateModelResource' in binary. Make sure the Op and Kernel are registered in the binary running in this process. Is it possible for me to register the Op in the binary? Thanks!

[Craigacp]

In many cases (e.g. tensorflow-text) you can load the native library that contains the ops with TensorFlow.loadLibrary(String). That should allow SavedModelBundle to load models that contain those ops. I've not tested it with decision forests, but if it doesn't work we can open an issue on their github.

[mattdornfeld]

@Craigacp I tried the solution describe here, where you pass the compiled .so file for the tf-df library to TensorFlow.loadLibrary before calling SavedModelBundle.load. I ran into this exception

        Caused by:
        org.tensorflow.exceptions.TFInvalidArgumentException: No shape inference function exists for op 'SimpleMLLoadModelFromPathWithHandle', did you forget to define it?
            at org.tensorflow.internal.c_api.AbstractTF_Status.throwExceptionIfNotOK(AbstractTF_Status.java:87)
            at org.tensorflow.SavedModelBundle.load(SavedModelBundle.java:623)
            at org.tensorflow.SavedModelBundle.access$000(SavedModelBundle.java:67)
            at org.tensorflow.SavedModelBundle$Loader.load(SavedModelBundle.java:97)
            at org.tensorflow.SavedModelBundle.load(SavedModelBundle.java:357)

Any advice on how to get around this?

[Craigacp]

As you've found this seems to be an issue with how TF-DF defines its operations (https://discuss.tensorflow.org/t/decision-forests-issue-with-c-api/7434/5), and is outside our control. Until they fix the op definition I don't think we can do anything. The TF Python API disables this shape check (https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/framework/ops.py#L3211), but the ShapeRefiner API isn't part of the C API so we can't use it in TF Java.

[achoum]

Hi @Craigacp,

The op in question is defined here. Would simply calling "SetShapeFn" (like here) solve this error?

Cheers, M

[Craigacp]

I expect so. I don't think it's actually going to do anything with the inferred shape as I doubt there are many operations dowstream of it, but the C API requires that all the ops have a defined output shape function.

[achoum]

I did the change mentioned above (adding SetShapeFn to the op definitions) and it seems the situation was resolved. See this issue.