rueian
commented
1 year ago Try add this line
for i, sql_and_id := range sqlAndIdArr {
----> i, sql_and_id := i, sql_and_id
eg.Go(func() (err error) {Closed totorofly closed 1 year ago
rueian
commented
1 year ago Try add this line
for i, sql_and_id := range sqlAndIdArr {
----> i, sql_and_id := i, sql_and_id
eg.Go(func() (err error) {
totorofly
commented
1 year ago Try add this line
for i, sql_and_id := range sqlAndIdArr { ----> i, sql_and_id := i, sql_and_id eg.Go(func() (err error) {Thank you, it's resolved now. Both testReidsPipelineAuto and testReidsPipelineManual are working properly on standalone Redis through public network connection.
However, when I tried to connect to the Redis cluster through internal network connection, testReidsPipelineAuto worked fine, but testReidsPipelineManual threw an error: "panic: Blocking module command called from transaction". The screenshot is shown below:
Does DoMulti not support Redis Cluster?
rueian
commented
1 year ago Yes, DoMulti supports clusters. However, it currently wraps commands with MULTI EXEC block to handle retries if their slot got migrated. This behavior might be changed in future releases.
But now, you can use the auto pipelining way.
Another thing is that RediSearch by default does not support Redis Cluster. Did you get expected results with testReidsPipelineAuto on the cluster?
totorofly
commented
1 year ago Yes, DoMulti supports clusters. However, it currently wraps commands with
MULTI EXECblock to handle retries if their slot got migrated. This behavior might be changed in future releases.But now, you can use the auto pipelining way.
Another thing is that RediSearch by default does not support Redis Cluster. Did you get expected results with testReidsPipelineAuto on the cluster?
testReidsPipelineAuto seems to be able to achieve the results I expected. The open-source version of Redis Cluster actually supports RediSearch, and building the corresponding .so module for the OSS version can enable support for RediSearch.
git clone -b v2.4.15 --recursive https://github.com/RediSearch/RediSearch.git
make setup
make build COORD=oss
cd /root/RediSearch/bin/linux-x64-release/coord-oss
redisearch.so # For master-slave module-oss.so # For cluster
rueian
commented
1 year ago Hi @dwzkit, the behavior of wrapping commands with MULTI EXEC has been removed in v0.0.95. You can now use testReidsPipelineManual with a redis cluster.
Hello, I'm here to seek your help again.For each API request from the user, the server program needs to execute 10 FT.SEARCH commands, each with a different query. Currently, my program runs these commands one by one and retrieves the results for each one sequentially. However, this approach does not save round-trip time (RTT), and it also does not save on context switching between Redis server kernel space and user space.
Therefore, I plan to use pipeline to bundle the 10 FT.SEARCH commands required for each API request together and execute them via pipeline to reduce RTT and context switching.
I have written functions for both Auto Pipelining(testReidsPipelineAuto) and Manually Pipelining(testReidsPipelineManual) and performed comparative tests. I expect both functions to return the results of the 10 FT.SEARCH commands required for each request in the order they were initiated, with each result corresponding to the respective command in a one-to-one manner.
The execution speed of testReidsPipelineAuto is faster than testReidsPipelineManual, but while testReidsPipelineAuto is expected to return results for 10 FT.SEARCH commands, it only returns the result of the last FT.SEARCH command. On the other hand, testReidsPipelineManual returns results for each of the 10 FT.SEARCH commands in order.
Here is my code example. What is causing the difference between the two, and is there an error in the implementation of the testReidsPipelineAuto function?