google / cel-go

Fast, portable, non-Turing complete expression evaluation with gradual typing (Go)
https://cel.dev
Apache License 2.0
2.26k stars 220 forks source link

Creating Env from Protobuf Message #997

Closed taichimaeda closed 2 months ago

taichimaeda commented 2 months ago

Feature request checklist

Change

I'm using CEL for filtering a large collection of item structs (~500MB in total).

Each of these structs has a number of fields, and I create an activation from these values via interpreter.NewActivation and store in the struct, in order to avoid making one on every program.Eval - but this ends up copying pretty much the entire struct because I need to pass a map to interpreter.NewActivation, which is not ideal especially when the size of the collection is so large.

I do understand CEL is not really meant for performance, but I was wondering if something similar to the one proposed in this issue might help: https://github.com/google/cel-go/issues/206, which will allow using the entire protobuf message as CEL env, so that it won't involve the unnecessary conversion to a map.

The PR https://github.com/google/cel-go/pull/219 for this issue unfortunately didn't make it to getting merged, so I was curious if there's been any plan on resuming it or if it's okay for me to work on it instead?

Example

My Item struct looks roughly looks like this:

type Item struct {
    FieldA string
    FieldB int
    ...

    Activation interpreter.Activation
}

which I initialise using interpreter.NewActivation:

items := getItems()
...
for i := range items {
    item := items[i]
    item.Activation = interpreter.NewActivation(map[string]any{
        "fieldA": item.FieldA,
        "fieldB": item.FieldB,
        ...
    })
}

But this requires copying item.FieldA, item.FieldB into the map, ... which essentially doubles the size of the Item struct...

Ideally, I would like to replace this Item struct with a protobuf message, so that I can use it in the other parts of my code as well as for filtering with CEL.

Alternatives considered

taichimaeda commented 2 months ago

Never mind, actually this has nothing to do with CEL envs, nor is the map creation problem