search

google / starlark-go

Starlark in Go: the Starlark configuration language, implemented in Go
BSD 3-Clause "New" or "Revised" License
2.3k stars 209 forks source link

How to go about tracking data accesses #293

Closed fenollp closed 4 years ago

fenollp commented 4 years ago

In my usage of Starlark I pass JSON-like objects to pure user-defined functions and I'd like to know which data paths the function reads.

So given 

def myfn(x):
    firstKey2 = x["key2"][0]
    return int(x["key1"]) + firstKey2["k"]

I'd like to obtain the following "JSON paths":

How should I go about this, in your opinion @alandonovan?

  1. Can it be done with syntax.Walk? It'd need to keep track of identifiers from function call to function call and native calls would just be black boxes (although that may turn out to be acceptable).
  2. If I make x into my own type (that implements all the interfaces that Dict, List, Int, String, Float, Bool implement) and have this implementation register the paths read, what should its Type() be? Is such a type even possible?
  3. Fork go.starlark.net/starlark and add my tracking calls to the implementations of each Value in there.

IMO (1) is the saner solution, I'm just scared that it'd be re-implementing the interpreter with its own bugs...

Do you have opinions/goals regarding tracing Starlark? If I were to modify interp.go and eval.go to add my own "profiling" calls (under a flag) would you be considering to merge these upstream?

alandonovan commented 4 years ago

None of these approaches is attractive. Approach 1 requires you to limit what computations you can do or else give up as soon as you make a function call. Approach 2 is not behavior-preserving: the magical mock value would be treated differently than a real int, bool, or whatever. Approach 3 is a maintenance nightmare; I would be unlikely to accept a PR to add the hooks you would need.

If I were you, I would rethink your bigger problem so that you don't need to solve this subproblem.