need API to enumerate names/bindings of a Function's local variables

[hungtcs]

Hi all,

I am trying to access the context variable that calls the function in a custom built-in function, can I do it?

var code = `
name = "hungtcs"
hello();
`

func main() {
  var predeclared = starlark.StringDict{
    "hello": starlark.NewBuiltin("hello", func(thread *starlark.Thread, fn *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) {
      // Question: How do I access global variables here?
      // fmt.Printf("hello %s\n", name)
      return starlark.None, nil
    }),
  }

  globals, err := starlark.ExecFile(
    &starlark.Thread{},
    "main.star",
    code,
    predeclared,
  )
  if err != nil {
    panic(err)
  }

  fmt.Printf("globals: %v\n", globals)
}

My idea was to implement code similar to the following, but using go to writing the hello method:

def hello():
  print("hello %s" % name)
name = "hungtcs"
hello();

[adonovan]

Builtins are not associated with any module, and do not have special access to any variables that were not passed to them as arguments; this is as it should be.

It is possible (from Go) to set a thread-local value that affects the behavior of certain built-ins called from that thread. In this way, built-ins can communicate with each other through Go data structures, and communicate with the host application as a whole.

It is also possible to use the debug interface to walk up the call stack to find any enclosing active calls to Starlark functions, and find the their associated modules and global variables (thread.DebugFrame(i).Callable.(*Function).Globals()["name"]). But overuse of this technique leads to confusion, surprises, and built-in functions that are hard to accurately explain.

Why not give the hello function a name parameter?

[hungtcs]

Thank you very much for your answer! I'm trying to implement a function similar to JS string interpolation like this:

let name = 'hungtcs'
let message = `hello ${ name }`

I'm aiming to implement something similar in starlark

name = 'hungtcs'
message = format("${ name }") # format is a starlark.NewBuiltin

Although starlark already provides functions such as String::format, I wanted to keep it as simple as possible when providing it to the user.

Thanks for the thread.DebugFrame(i).Callable.(*Function).Globals()["name"] solution, which looks like it will solve my needs at the moment, although it may make the code look less elegant.

[hungtcs]

Sorry I just realized that I can't get local variables inside functions this way. Should I consider it untenable to implement something like a js interpolating function?

name = "hungtcs"

def test():
    foo = "abc"
    print(format("hello {name},, {foo}"))

test()

var predeclared = starlark.StringDict{
    "format": starlark.NewBuiltin("format", func(thread *starlark.Thread, fn *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) {
        var globals = thread.DebugFrame(1).Callable().(*starlark.Function).Globals()

        var string = args[0].(starlark.String)
        var format *starlark.Builtin
        if val, err := string.Attr("format"); err != nil {
            return nil, err
        } else {
            format = val.(*starlark.Builtin)
        }

        var formatKwargs = make([]starlark.Tuple, 0)
        for key, val := range globals {
            formatKwargs = append(formatKwargs, starlark.Tuple{starlark.String(key), val})
        }

        var result, err = format.CallInternal(thread, starlark.Tuple{}, formatKwargs)
        if err != nil {
            return nil, err
        }

        return result, nil
    }),
}

[adonovan]

It's possible to access the values of local variables through the debug interface: thread.DebugFrame(i).Callable.(*Function).Local(i). The tricky thing is that the information about each local variable (its name and binding location) are available only through internal functions (Function.funcode.Locals). Only the first few local variables corresponding to parameters are currently accessible using public APIs (Function.Param(i)).

[hungtcs]

@adonovan Yes, thanks for your perspective, I added a line of code to starlark/value.go to expose funcode.Locals so that I could get the local bindings of the function.

https://github.com/google/starlark-go/blob/3f0a3703c02aaff109c1d5df37be01fee784d4fc/starlark/value.go#L730

func (fn *Function) Locals() []compile.Binding { return fn.funcode.Locals }

But how to get Value via binding ? Then I found this function, and now I can get the corresponding value:

for idx, local := range locals {
    fmt.Println(thread.DebugFrame(1).Local(idx))
}

Here's my current format function, which seems to be working just fine, Do you have an opinion on this, I'm not sure it's as correct as I think it is (thanks).

var predeclared = starlark.StringDict{
    "format": starlark.NewBuiltin("format", func(thread *starlark.Thread, fn *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) {
        var function = thread.DebugFrame(1).Callable().(*starlark.Function)

        var locals = function.Locals()
        var globals = function.Globals()

        var formatArgs = make(map[string]starlark.Value)
        for key, val := range globals {
            formatArgs[key] = val
        }

        for idx, local := range locals {
            formatArgs[local.Name] = thread.DebugFrame(1).Local(idx)
        }

        var string = args[0].(starlark.String)
        var format *starlark.Builtin
        if val, err := string.Attr("format"); err != nil {
            return nil, err
        } else {
            format = val.(*starlark.Builtin)
        }

        var formatKwargs = make([]starlark.Tuple, 0)
        for key, val := range formatArgs {
            formatKwargs = append(formatKwargs, starlark.Tuple{starlark.String(key), val})
        }

        var result, err = format.CallInternal(thread, starlark.Tuple{}, formatKwargs)
        if err != nil {
            return nil, err
        }

        return result, nil
    }),
}

[adonovan]

var function = thread.DebugFrame(1).Callable().(*starlark.Function)

You need to handle the case where the Callable is something other than a *Function.

  var string = args[0].(starlark.String)

Again here you need to handle a type error.

  if val, err := string.Attr("format"); err != nil {
      return nil, err

This can be a panic, since we know that every string has a format method.

      format = val.(*starlark.Builtin)

Not needed. val is fine.

      formatKwargs = append(formatKwargs, starlark.Tuple{starlark.String(key), val})

key is already a string.

  var result, err = format.CallInternal(thread, starlark.Tuple{}, formatKwargs)

Never call CallInternal directly. Use Call.

You can pass nil for the Tuple.

You can return Call(...) directly; there's no need for "if err".

The main problem is that Function.Locals isn't public API. We could add it (or something like it), but we'd need to be very careful not to expose anything about the representation of compiled programs.

[hungtcs]

Thanks @adonovan I'm basically sure it works now. Please expose these internal interfaces in a suitable way to make the implementation of this function possible. most people probably won't use this feature, but it's very useful for library developers or people who want to implement certain magic functions.