aerospike / aerospike-client-go

Aerospike Client Go
Apache License 2.0
433 stars 199 forks source link

Is it possible to avoid UDF compilation on client's side? #295

Closed sm4ll-3gg closed 3 years ago

sm4ll-3gg commented 4 years ago

Hi! The documentation says:

If there is no reduce function, the client layer simply passes all the data coming from the nodes to the application.

This is my case, but, looking at the QueryAggregate function I see that the UDF compiles anyway. This is a very big overhead for me.

Is there a way to avoid it?

sm4ll-3gg commented 4 years ago

Can I replace the QueryAggregate func withQuery and manually call the Statement's SetAggregateFunction method so that the UDF is called only on the server's side?

khaf commented 4 years ago

The UDF will be compiled since without its compilation, it won't know it doesn't have the reduce function :)

You can the simple query function, but I haven't tested if it works. I suspect it will.

sm4ll-3gg commented 4 years ago

@khaf, Thank you for the answer!

I've tried to use Query function and it seems to work. I'm going to test it in a couple of hours.
Anyway I think there should be an explicit documented way to avoid this because I have a profile with 2GB allocated memory only on UDF compilation on about 4k rpm

khaf commented 4 years ago

Lua is heavy on memory allocation, and the Go lua implementation is especially heavy. Not much we can do in this regard.

sm4ll-3gg commented 4 years ago

Yep, but we can provide a way not to use lua at all on the client's side if it is not required (a UDF doesn't contain reduce, for example).
The Query seems to be solution, buy it should be properly documented and tested