Research passing Scala functions to TensorFlow Python APIs

[PawelLipski]

See how Java bindings to Tensorflow addressed this problem (if they have at all)

[PawelLipski]

The official Java bindings for TF seem to rely on DSL that de facto allows for constructing a restricted set of Python ASTs from Java code (see e.g. https://www.tensorflow.org/api_docs/java/reference/org/tensorflow/Operand)

[PawelLipski]

We can mimic this approach in our implementation as well; a more generic but also much more advanced approach for Scala 3 would be with quote/splice metaprogramming: https://dotty.epfl.ch/docs/reference/metaprogramming/macros.html

[PawelLipski]

This might provide the hint on how to do it with low-ish engineering overhead: https://github.com/ninia/jep/wiki/How-Jep-Works#objects

A PyJObject wraps the reference to the original Java object 
and presents the Python interpreter with an interface for understanding the object as a Python object.
From the point-of-view of the Python interpreter,
a PyJObject is just another Python object with a select set of attributes (fields and methods) on it.

TBD exact mechanics and its limitations (e.g. note that in JEP Python interpreter instances are managed by JEP within the JVM).

[PawelLipski]

Also, Swift<->TF bindings might be interesting

[Gryfit]

https://github.com/tensorflow/swift/issues/211 https://www.tensorflow.org/swift/tutorials/model_training_walkthrough

[PawelLipski]

Ok @Gryfit could you drop a 1-2 sentence summary of what's possible/not possible in the mentioned Swift binding?

[Gryfit]

JAVA:

Awkward builders, that require creating operations from scratch. https://www.tensorflow.org/api_docs/java/reference/org/tensorflow/OperationBuilder https://www.baeldung.com/tensorflow-java#3-defining-functions

    Operation ax = graph.opBuilder("Mul", "ax")
  .addInput(a.output(0))
  .addInput(x.output(0))
  .build(); 

Quoting documentation:

The Java API doesn't yet include convenience functions for adding operations.

SWIFT:

As far as I understand the idea is to calculate loss function (one of those we would normally pass to model.compile) on swift side. That way passing function is not necessary.

Either way as suggested in this PR https://github.com/tensorflow/swift/issues/211 a way to pass functions has not been implemented although there is a suggestion it could be done relatively easy https://github.com/tensorflow/swift/issues/211#issuecomment-594852844.

print(test.f(PythonObject({ z in pow(z, 2) }), 5))

It would however mean writing those function in python and then wrapping them into swift.

JS:

Also no support.

Conclusion

In general even though the new tf.function remains heavily advocated feature in python library as of writing no other language API included this.