ponder-lab / Hybridize-Functions-Refactoring

Refactorings for optimizing imperative TensorFlow clients for greater efficiency.
Eclipse Public License 2.0
0 stars 0 forks source link

Missing nested function in call graph #282

Open khatchad opened 1 year ago

khatchad commented 1 year ago

There's a missing call to _distributed_train_step() in the call graph.

The function is defined here:

https://github.com/ponder-lab/mead-baseline/blob/4411029050d5549b34c75fe205451d6a1e80d335/mead/api_examples/pretrain_paired_tf.py#L290-L298

And called here:

https://github.com/ponder-lab/mead-baseline/blob/4411029050d5549b34c75fe205451d6a1e80d335/mead/api_examples/pretrain_paired_tf.py#L329

But I am seeing this is the logs:

!ENTRY edu.cuny.hunter.hybridize.core 4 0 2023-11-07 21:56:34.658
!MESSAGE Can't get call graph nodes for: < PythonLoader, Lscript pretrain_paired_tf.py/main/_distributed_train_step, do()LRoot; >.

!ENTRY edu.cuny.hunter.hybridize.core 1 0 2023-11-07 21:56:34.659
!MESSAGE Found 0 node(s) corresponding to: < PythonLoader, Lscript pretrain_paired_tf.py/main/_distributed_train_step, do()LRoot; >.

!ENTRY edu.cuny.hunter.hybridize.core 2 0 2023-11-07 21:56:34.659
!MESSAGE Unable to infer side-effects of: main._distributed_train_step().
!STACK 0
edu.cuny.hunter.hybridize.core.analysis.UndeterminablePythonSideEffectsException: Can't find: < PythonLoader, Lscript pretrain_paired_tf.py/main/_distributed_train_step, do()LRoot; > in call graph.
    at edu.cuny.hunter.hybridize.core.analysis.Function.inferPythonSideEffects(Function.java:384)
    at edu.cuny.hunter.hybridize.core.refactorings.HybridizeFunctionRefactoringProcessor.lambda$2(HybridizeFunctionRefactoringProcessor.java:230)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
    at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1707)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
    at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
    at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)

Regression

Unsure why it would be missing in the call graph. It's not a callback and thus most likely unrelated to #280. The only thing I can think of is that it is contained in a with statement.

khatchad commented 1 year ago

run.log.gz Attached logs.

khatchad commented 1 year ago

I don't think this has anything to do with the with statement. Instead, it's related to https://github.com/wala/ML/issues/91.