Performance problems

[hudengjunai]

I am very glad to see this project. And I have some questions Q1: After reading the 5 stage optmimation in benchmark. I found the redis-vss-retrievel fetch features from feature-store consumes a lot time. Is there any techniques to speedup? I want to run the who pipeline in 10ms. but the redis-vss vector index and feature retrieve has consumed about 4-5ms

[Spartee]

Hi @hudengjunai Which set of model revisions are you using?

The most performant (that I benchmarked from what you've seen) is this revision: https://github.com/RedisVentures/Redis-Recsys/tree/master/online-multi-stage-recsys/revisions/05-vss-retrieval-opt

In general, that stage should take roughly a third of the time given network hops. Given how you stated your issue, I'm assuming that you are using the combined vector search retrieval of item features. This stage will depend heavily on the number of items retrieved. The default is 64 (which is somewhat low for typical use cases).

Would help to know more about your benchmarking setup esp what Redis you are using? OSS? Enterprise? Other?

In general, there are a couple "easy" ways to speed this up.

• Use triton model_analyzer and follow it's help
• increase number of "model" copies for that stage.
• decrease number of items retrieved.

Harder improvements (that I have not tried):

• Use the C++ triton backend and a C++ redis client.

[Spartee]

Closing as no comment recently