ext: support member overloads for map-like custom types

[AlexanderYastrebov]

If this change is accepted the same could be implemented for slice-like types.

Example usecase is to access http.Request properties like:

    expr := `
        request.Method == "GET"
        && "q" in request.URL.Query()
        && request.URL.Query().Has("q")
        && request.URL.Query().Get("q") == "t"
        && request.URL.Host == "foo.test:1234"
        && request.URL.Hostname() == "foo.test"
        && request.URL.Port() == "1234"
        && request.URL.RequestURI() == "/foo?q=t"
        && "X-Foo" in request.Header
        && "bar" in request.Header["X-Foo"]
        && "baz" in request.Header["X-Foo"]
        && request.Header.Get("x-foo") == "bar"
        && "baz" in request.Header.Values("x-foo")
    `

    urlType := cel.ObjectType("url.URL")
    urlValuesType := cel.MapType(cel.StringType, cel.ListType(cel.StringType)).WithRuntimeTypeName("url.Values")
    httpHeaderType := cel.MapType(cel.StringType, cel.ListType(cel.StringType)).WithRuntimeTypeName("http.Header")

    var env *cel.Env

    makeGetter := func(typ *cel.Type, getter string, argsAndResultTypes ...*cel.Type) cel.EnvOption {
        if len(argsAndResultTypes) == 0 {
            panic("missing return type")
        }
        argTypes := argsAndResultTypes[:len(argsAndResultTypes)-1]
        retType := argsAndResultTypes[len(argsAndResultTypes)-1]
        return cel.Function(getter,
            cel.MemberOverload(
                strings.ReplaceAll(typ.TypeName(), ".", "_")+"_"+getter,
                append([]*cel.Type{typ}, argTypes...),
                retType,
                cel.FunctionBinding(func(args ...ref.Val) ref.Val {
                    method := reflect.ValueOf(args[0].Value()).MethodByName(getter)
                    var callArgs []reflect.Value
                    for _, arg := range args[1:] {
                        callArgs = append(callArgs, reflect.ValueOf(arg.Value()))
                    }
                    result := method.Call(callArgs)[0].Interface()
                    return env.CELTypeAdapter().NativeToValue(result)
                }),
            ),
        )
    }

    env, err := cel.NewEnv(
        cel.Variable("request", cel.ObjectType("http.Request")),
        ext.NativeTypes(reflect.TypeOf(http.Request{}), reflect.TypeOf(url.URL{})),
        ext.Strings(),

        makeGetter(urlType, "Hostname", cel.StringType),
        makeGetter(urlType, "Port", cel.StringType),
        makeGetter(urlType, "RequestURI", cel.StringType),
        makeGetter(urlType, "Query", urlValuesType),
        makeGetter(urlValuesType, "Get", cel.StringType, cel.StringType),
        makeGetter(urlValuesType, "Has", cel.StringType, cel.BoolType),
        makeGetter(httpHeaderType, "Get", cel.StringType, cel.StringType),
        makeGetter(httpHeaderType, "Values", cel.StringType, cel.ListType(cel.StringType)),
    )
    ...

[TristonianJones]

@AlexanderYastrebov there's a lot going on in this PR, but let me try to break it down into two feature requests:

1. Ability to support field selection on objects using non-identifier characters, possibly using the index syntax, e.g. request.Header["X-Foo"]
2. Ability to create custom types using a base type

The first issue is easier to support, and it would simplify some other requests related to non-protobuf compatible identifier characters.

The second is challenging since you're effectively creating union types on top of a type-checker which wasn't really setup to handle them; however, I'm not sure you need it, since most of the fields and types you want would be made accessible via the native types extension. Is this an issue with hiding certain fields from use? I guess, my question would be why the accessors and not just the field accesses?

[AlexanderYastrebov]

@TristonianJones

Consider https://pkg.go.dev/net/http

// package http

type Request struct {
  Header Header
  URL *url.URL
}

type Header map[string][]string

func (h Header) Get(key string) string

// package url

type URL struct {}

func (u *URL) Query() Values

type Values map[string][]string

func (v Values) Get(key string) string

Header is just a field of request but it has map-like type. Accessing Header as a field e.g. "bar" in request.Header["X-Foo"] already works just fine. But to have request.Header.Get("x-foo") == "bar" we need to register member overload for Get.

Currently it is only possible to register such overload on the map[string][]string type but the problem is that two such map-like types can have the same method: e.g. both http.Header and url.Values have Get(string) string.

The idea of this PR is to define http.Header and url.Values as two different runtime types instead of map[string][]string and register Get member overload for each of them.

Now while I was writhing the above I realized that we can make Get polymorphic to make it work on both http.Header and url.Values:

cel.Function("Get",
    cel.MemberOverload(
        "map[string][]string Get(string) string",
        []*cel.Type{cel.MapType(cel.StringType, cel.ListType(cel.StringType)), cel.StringType},
        cel.StringType,
        cel.FunctionBinding(func(args ...ref.Val) ref.Val {
            switch receiver := args[0].Value().(type) {
            case url.Values:
                return env.CELTypeAdapter().NativeToValue(receiver.Get(args[1].Value().(string)))
            case http.Header:
                return env.CELTypeAdapter().NativeToValue(receiver.Get(args[1].Value().(string)))
            default:
                panic(fmt.Sprintf("Unsupported %T\n", receiver))
            }
        }),
    ),
),

The downside is that type check happens in runtime. Also it will not be possible to register two member overloads that differ by return type e.g. Foo() string and Foo() bool (Get for both http.Header and url.Values has the same signature Get(string) string).

[AlexanderYastrebov]

Is this an issue with hiding certain fields from use? I guess, my question would be why the accessors and not just the field accesses?

I want to use stdlib http.Request and url.Values in CEL expression without wrapping them into custom types. With polymorphic Get from https://github.com/google/cel-go/pull/875#issuecomment-1880055737 the subject expression

expr := `
        request.Method == "GET"
        && "q" in request.URL.Query()
        && request.URL.Query().Has("q")
        && request.URL.Query().Get("q") == "t"
        && request.URL.Host == "foo.test:1234"
        && request.URL.Hostname() == "foo.test"
        && request.URL.Port() == "1234"
        && request.URL.RequestURI() == "/foo?q=t"
        && "X-Foo" in request.Header
        && "bar" in request.Header["X-Foo"]
        && "baz" in request.Header["X-Foo"]
        && request.Header.Get("x-foo") == "bar"
        && "baz" in request.Header.Values("x-foo")
    `

works so I guess we can postpone this feature until another usecase e.g. that would require member overloads with the same name and arguments but different return type.

[TristonianJones]

@AlexanderYastrebov My recommendation would be to move the custom functions up to the http.Request and url.URL types since you can differentiate between those types, and how headers are packaged into a request is specific to HTTP rather than specific to maps since header names are case-insensitive.

[AlexanderYastrebov]

I.e. instead of go-like request.Header.Get("x-foo") && request.URL.Query().Has("q") use request.HeaderGet("x-foo") && request.URL.QueryHas("q")