wala / ML

Eclipse Public License 2.0
24 stars 17 forks source link

Adding initial support for decorator `tf.function` #86

Closed tatianacv closed 11 months ago

tatianacv commented 11 months ago

Currently, we are using jython2 in com.ibm.wala.cast.python.ml.test, where we don't encounter decorator problems when running the tests. But when we use jython3, we do encounter problems with decorator tf.function. Therefore we are adding initial support for the decorator tf.function in tensorflow.xml. Initial support for tf.function because after this change, in jython3, can now process the decorator that has parenthesis e.g. tf.function() or tf.function(...).

Refer to https://github.com/wala/ML/issues/33 for the issue with inconsistent jython versions.

After this change we see: Test code with parenthesis:

import tensorflow as tf

@tf.function()
def returned(a):
  return a

a = tf.range(5)
b = returned(a)

The analysis of the test with parenthesis:

answer:
[Node: <Code body of function Lscript tf2_testing_decorator.py> Context: CallStringContext: [ com.ibm.wala.FakeRootClass.fakeRootMethod()V@2 ], v251][{[D:Symbolic,n, D:Compound,[D:Constant,28, D:Constant,28]] of pixel}]
[Ret-V:Node: synthetic < PythonLoader, Ltensorflow/functions/range, do()LRoot; > Context: CallStringContext: [ script tf2_testing_decorator.py.do()LRoot;@96 ]][{[D:Symbolic,n, D:Compound,[D:Constant,28, D:Constant,28]] of pixel}]
[Node: synthetic < PythonLoader, Ltensorflow/functions/range, do()LRoot; > Context: CallStringContext: [ script tf2_testing_decorator.py.do()LRoot;@96 ], v6][{[D:Symbolic,n, D:Compound,[D:Constant,28, D:Constant,28]] of pixel}]
[Node: <Code body of function Lscript tf2_testing_decorator.py/returned> Context: CallStringContext: [ script tf2_testing_decorator.py.do()LRoot;@98 ], v2][{[D:Symbolic,n, D:Compound,[D:Constant,28, D:Constant,28]] of pixel}]
[Node: <Code body of function Lscript tf2_testing_decorator.py> Context: CallStringContext: [ com.ibm.wala.FakeRootClass.fakeRootMethod()V@2 ], v257][{[D:Symbolic,n, D:Compound,[D:Constant,28, D:Constant,28]] of pixel}]
[Ret-V:Node: <Code body of function Lscript tf2_testing_decorator.py/returned> Context: CallStringContext: [ script tf2_testing_decorator.py.do()LRoot;@98 ]][{[D:Symbolic,n, D:Compound,[D:Constant,28, D:Constant,28]] of pixel}]

Test code with parenthesis:

import tensorflow as tf

@tf.function
def returned(a):
  return a

a = tf.range(5)
b = returned(a)

The analysis of the test without parenthesis:

answer:
[Node: <Code body of function Lscript tf2_testing_decorator.py> Context: CallStringContext: [ com.ibm.wala.FakeRootClass.fakeRootMethod()V@2 ], v249][{[D:Symbolic,n, D:Compound,[D:Constant,28, D:Constant,28]] of pixel}]
[Ret-V:Node: synthetic < PythonLoader, Ltensorflow/functions/range, do()LRoot; > Context: CallStringContext: [ script tf2_testing_decorator.py.do()LRoot;@95 ]][{[D:Symbolic,n, D:Compound,[D:Constant,28, D:Constant,28]] of pixel}]
[Node: synthetic < PythonLoader, Ltensorflow/functions/range, do()LRoot; > Context: CallStringContext: [ script tf2_testing_decorator.py.do()LRoot;@95 ], v6][{[D:Symbolic,n, D:Compound,[D:Constant,28, D:Constant,28]] of pixel}]
[Node: synthetic < PythonLoader, Ltensorflow/class/Function, do()LRoot; > Context: CallStringContext: [ script tf2_testing_decorator.py.do()LRoot;@97 ], v2][{[D:Symbolic,n, D:Compound,[D:Constant,28, D:Constant,28]] of pixel}]
[Node: <Code body of function Lscript tf2_testing_decorator.py> Context: CallStringContext: [ com.ibm.wala.FakeRootClass.fakeRootMethod()V@2 ], v255][{[D:Symbolic,n, D:Compound,[D:Constant,28, D:Constant,28]] of pixel}]
[Ret-V:Node: synthetic < PythonLoader, Ltensorflow/class/Function, do()LRoot; > Context: CallStringContext: [ script tf2_testing_decorator.py.do()LRoot;@97 ]][{[D:Symbolic,n, D:Compound,[D:Constant,28, D:Constant,28]] of pixel}]

We see that with the decorator, expr is classified as a CALL, and without it is classified as an Attribute.