I'm using CEL for filtering a large collection of item structs (~500MB in total).
Each of these structs has a number of fields, and I create an activation from these values via interpreter.NewActivation and store in the struct, in order to avoid making one on every program.Eval - but this ends up copying pretty much the entire struct because I need to pass a map to interpreter.NewActivation, which is not ideal especially when the size of the collection is so large.
I do understand CEL is not really meant for performance, but I was wondering if something similar to the one proposed in this issue might help: https://github.com/google/cel-go/issues/206, which will allow using the entire protobuf message as CEL env, so that it won't involve the unnecessary conversion to a map.
The PR https://github.com/google/cel-go/pull/219 for this issue unfortunately didn't make it to getting merged, so I was curious if there's been any plan on resuming it or if it's okay for me to work on it instead?
Example
My Item struct looks roughly looks like this:
type Item struct {
FieldA string
FieldB int
...
Activation interpreter.Activation
}
which I initialise using interpreter.NewActivation:
items := getItems()
...
for i := range items {
item := items[i]
item.Activation = interpreter.NewActivation(map[string]any{
"fieldA": item.FieldA,
"fieldB": item.FieldB,
...
})
}
But this requires copying item.FieldA, item.FieldB into the map, ... which essentially doubles the size of the Item struct...
Ideally, I would like to replace this Item struct with a protobuf message, so that I can use it in the other parts of my code as well as for filtering with CEL.
Feature request checklist
Change
I'm using CEL for filtering a large collection of item structs (~500MB in total).
Each of these structs has a number of fields, and I create an activation from these values via
interpreter.NewActivation
and store in the struct, in order to avoid making one on everyprogram.Eval
- but this ends up copying pretty much the entire struct because I need to pass a map tointerpreter.NewActivation
, which is not ideal especially when the size of the collection is so large.I do understand CEL is not really meant for performance, but I was wondering if something similar to the one proposed in this issue might help: https://github.com/google/cel-go/issues/206, which will allow using the entire protobuf message as CEL env, so that it won't involve the unnecessary conversion to a map.
The PR https://github.com/google/cel-go/pull/219 for this issue unfortunately didn't make it to getting merged, so I was curious if there's been any plan on resuming it or if it's okay for me to work on it instead?
Example
My
Item
struct looks roughly looks like this:which I initialise using
interpreter.NewActivation
:But this requires copying
item.FieldA
,item.FieldB
into the map, ... which essentially doubles the size of theItem
struct...Ideally, I would like to replace this
Item
struct with a protobuf message, so that I can use it in the other parts of my code as well as for filtering with CEL.Alternatives considered