Change Redis default port 6379 to new port flexiably

[q131172019]

This PR is to make the following changes 1) Flexibly change redis default port from 6379 to new port (default:7379) 2) Automatically redis server installation 3) singleNodeQuery due to the changes of function redis.NewRedisClient

Test1 : new port 8389

1. Installation test:

   ubuntu@ip-172-31-8-82:~/go/src/GRS$ export REDIS_NEW_PORT=8389
   ubuntu@ip-172-31-8-82:~/go/src/GRS$ /bin/bash hack/redis_install.sh

2. Local and remote connection test:

   
   ubuntu@ip-172-31-8-82:~/go/src/GRS$ redis-cli -h ec2-52-11-1-193.us-west-2.compute.amazonaws.com -p 8389 info keyspace
   db0:keys=1,expires=0,avg_ttl=0

ubuntu@ip-172-31-8-82:~/go/src/GRS$ redis-cli -h ip-172-31-8-82 -p 8389 info keyspace

Keyspace

db0:keys=1,expires=0,avg_ttl=0

ubuntu@ip-172-31-8-82:~/go/src/GRS$ redis-cli -h localhost -p 8389 info keyspace

Keyspace

db0:keys=1,expires=0,avg_ttl=0

ubuntu@ip-172-31-8-82:~/go/src/GRS$ redis-cli -p 8389 info keyspace

Keyspace

db0:keys=1,expires=0,avg_ttl=0

ubuntu@ip-172-31-4-135:~/go/src/GRS$ redis-cli -h ec2-52-11-1-193.us-west-2.compute.amazonaws.com -p 8389 info keyspace

Keyspace

db0:keys=1,expires=0,avg_ttl=0

3. UT test:

ubuntu@ip-172-31-8-82:~/go/src/GRS$ cd resource-management/pkg/store/redis/ ubuntu@ip-172-31-8-82:~/go/src/GRS/resource-management/pkg/store/redis$ go test -v === RUN TestNewRedisClient --- PASS: TestNewRedisClient (0.00s) === RUN TestSetGettString --- PASS: TestSetGettString (0.00s) === RUN TestPersistNodes --- PASS: TestPersistNodes (0.00s) === RUN TestPersistNodeStoreStatus --- PASS: TestPersistNodeStoreStatus (0.00s) === RUN TestPersistVirtualNodesAssignments --- PASS: TestPersistVirtualNodesAssignments (0.00s) === RUN TestBatchLogicalNodeInquiry redis_test.go:234: First test is starting redis_test.go:278: Scan (2) logical nodes and actually get (2) logical nodes redis_test.go:281: Index #(0): redis_test.go:282: ================= redis_test.go:283: Id: (0003) redis_test.go:284: Region: (1002) redis_test.go:285: RP: (1002) redis_test.go:281: Index #(1): redis_test.go:282: ================= redis_test.go:283: Id: (0002) redis_test.go:284: Region: (1001) redis_test.go:285: RP: (1001) redis_test.go:287: First test is OK

redis_test.go:290:
    Second test is starting
redis_test.go:335: Scan (10) logical nodes and actually get (11) logical nodes
redis_test.go:336:
    Second test is OK

redis_test.go:339:
    Third test is starting
redis_test.go:351: Scan (1000) logical nodes and actually get (1000) logical nodes
redis_test.go:352:
    Third test is OK

redis_test.go:355:
    Fourth test is starting
redis_test.go:367: Scan (2000) logical nodes and actually get (2000) logical nodes
redis_test.go:368:
    Fouth test is OK

redis_test.go:371:
    The fifth test is starting
redis_test.go:379: Scan (20000) logical nodes and actually get (10002) logical nodes
redis_test.go:380:
    Fifth test is OK

--- PASS: TestBatchLogicalNodeInquiry (3.02s) PASS ok global-resource-service/resource-management/pkg/store/redis 3.035s

4. Integration test:

ubuntu@ip-172-31-8-82:~/go/src/GRS$ kill -9 ps -ef |grep service-api |grep -v grep |awk '{print $2}';kill -9 ps -ef |grep service-api |grep -v grep |awk '{print $2}'; go run resource-management/cmds/service-api/service-api.go --master_ip=ec2-52-11-1-193.us-west-2.compute.amazonaws.com --resource_urls=ec2-54-187-22-27.us-west-2.compute.amazonaws.com:9119 --redis_port=8389 --enable_metrics=false -v=3 > ~/TMP/service.log.2022-09-20.v000183 2>&1 &

ubuntu@ip-172-31-8-205:~/go/src/GRS$ kill -9 ps -ef |grep go |grep -v grep |awk {print $2}';kill -9 ps -ef |grep go |grep -v grep |awk '{print $2}';go run resource-management/test/resourceRegionMgrSimulator/main.go --region_name=Beijing --rp_num=10 --nodes_per_rp=25000 --master_port=9119 --data_pattern=Daily -v=3 > ~/TMP/simulator1.log.2022-09-20.v000183 2>&1 &

ubuntu@ip-172-31-8-82:~/go/src/GRS/resource-management/pkg/store/redis$ redis-cli -h ec2-52-11-1-193.us-west-2.compute.amazonaws.com -p 8389 info keyspace;date

Keyspace

db0:keys=250001,expires=0,avg_ttl=0 Tue Sep 20 18:03:15 UTC 2022

Test2: use default port 7379
1. Installation test

ubuntu@ip-172-31-8-82:~/go/src/GRS/resource-management/pkg/store/redis$ unset REDIS_NEW_PORT ubuntu@ip-172-31-8-82:~/go/src/GRS/resource-management/pkg/store/redis$ echo $REDIS_NEW_PORT

ubuntu@ip-172-31-8-82:~/go/src/GRS/resource-management/pkg/store/redis$ /bin/bash ~/go/src/GRS/hack/redis_install.sh

2. Local and remote connection test

ubuntu@ip-172-31-8-82:~/go/src/GRS/resource-management/pkg/store/redis$ redis-cli -h ec2-52-11-1-193.us-west-2.compute.amazonaws.com -p 7379 info keyspace

Keyspace

db0:keys=1,expires=0,avg_ttl=0

ubuntu@ip-172-31-8-82:~/go/src/GRS/resource-management/pkg/store/redis$ redis-cli -h ip-172-31-8-82 -p 7379 info keyspace

Keyspace

db0:keys=1,expires=0,avg_ttl=0

ubuntu@ip-172-31-8-82:~/go/src/GRS/resource-management/pkg/store/redis$ redis-cli -h localhost -p 7379 info keyspace

Keyspace

db0:keys=1,expires=0,avg_ttl=0

ubuntu@ip-172-31-8-82:~/go/src/GRS/resource-management/pkg/store/redis$ redis-cli -p 7379 info keyspace

Keyspace

db0:keys=1,expires=0,avg_ttl=0

Remote: ubuntu@ip-172-31-4-135:~/go/src/GRS$ redis-cli -h ec2-52-11-1-193.us-west-2.compute.amazonaws.com -p 7379 info keyspace

Keyspace

db0:keys=1,expires=0,avg_ttl=0

3. UT test

ubuntu@ip-172-31-8-82:~/go/src/GRS/resource-management/pkg/store/redis$ go test -v === RUN TestNewRedisClient --- PASS: TestNewRedisClient (0.00s) === RUN TestSetGettString --- PASS: TestSetGettString (0.00s) === RUN TestPersistNodes --- PASS: TestPersistNodes (0.00s) === RUN TestPersistNodeStoreStatus --- PASS: TestPersistNodeStoreStatus (0.00s) === RUN TestPersistVirtualNodesAssignments --- PASS: TestPersistVirtualNodesAssignments (0.00s) === RUN TestBatchLogicalNodeInquiry redis_test.go:234: First test is starting redis_test.go:278: Scan (2) logical nodes and actually get (2) logical nodes redis_test.go:281: Index #(0): redis_test.go:282: ================= redis_test.go:283: Id: (0003) redis_test.go:284: Region: (1002) redis_test.go:285: RP: (1002) redis_test.go:281: Index #(1): redis_test.go:282: ================= redis_test.go:283: Id: (0002) redis_test.go:284: Region: (1001) redis_test.go:285: RP: (1001) redis_test.go:287: First test is OK

redis_test.go:290:
    Second test is starting
redis_test.go:335: Scan (10) logical nodes and actually get (10) logical nodes
redis_test.go:336:
    Second test is OK

redis_test.go:339:
    Third test is starting
redis_test.go:351: Scan (1000) logical nodes and actually get (1000) logical nodes
redis_test.go:352:
    Third test is OK

redis_test.go:355:
    Fourth test is starting
redis_test.go:367: Scan (2000) logical nodes and actually get (2000) logical nodes
redis_test.go:368:
    Fouth test is OK

redis_test.go:371:
    The fifth test is starting
redis_test.go:379: Scan (20000) logical nodes and actually get (10002) logical nodes
redis_test.go:380:
    Fifth test is OK

--- PASS: TestBatchLogicalNodeInquiry (2.96s) PASS ok global-resource-service/resource-management/pkg/store/redis 2.980s

4. Integration test

ubuntu@ip-172-31-8-82:~/go/src/GRS$ kill -9 ps -ef |grep service-api |grep -v grep |awk '{print $2}';kill -9 ps -ef |grep service-api |grep -v grep |awk '{print $2}'; go run resource-management/cmds/service-api/service-api.go --master_ip=ec2-52-11-1-193.us-west-2.compute.amazonaws.com --resource_urls=ec2-54-187-22-27.us-west-2.compute.amazonaws.com:9119 --redis_port=7379 --enable_metrics=false -v=3 > ~/TMP/service.log.2022-09-20.v000184 2>&1 &

ubuntu@ip-172-31-8-205:~/go/src/GRS$ kill -9 ps -ef |grep go |grep -v grep |awk {print $2}';kill -9 ps -ef |grep go |grep -v grep |awk '{print $2}';go run resource-management/test/resourceRegionMgrSimulator/main.go --region_name=Beijing --rp_num=10 --nodes_per_rp=25000 --master_port=9119 --data_pattern=Daily -v=3 > ~/TMP/simulator1.log.2022-09-20.v000184 2>&1 &

ubuntu@ip-172-31-8-82:~/go/src/GRS/resource-management/pkg/store/redis$ redis-cli -h ip-172-31-8-82 -p 7379 info keyspace;date

Keyspace

db0:keys=250001,expires=0,avg_ttl=0 Tue Sep 20 17:55:49 UTC 2022

Successful Integration Test (500K nodes/2 regions/20 schedulers in us-west-2 + singleNodeInquiry from machine in remote region us-east-2) for SingleNodeInqury after second commit is posted

--- GRS service in us-west-2

ubuntu@ip-172-31-8-82:~/go/src/GRS$ kill -9 ps -ef |grep service-api |grep -v grep |awk '{print $2}';kill -9 ps -ef |grep service-api |grep -v grep |awk '{print $2}'; go run resource-management/cmds/service-api/service-api.go --master_ip=ec2-52-11-1-193.us-west-2.compute.amazonaws.com --resource_urls=ec2-54-187-22-27.us-west-2.compute.amazonaws.com:9119,ec2-35-91-52-72.us-west-2.compute.amazonaws.com:9119 --redis_port=8389 --enable_metrics=false -v=3 > ~/TMP/service.log.2022-09-22.v000184 2>&1 &

-- 2 region simulators in us-west-2

ubuntu@ip-172-31-8-205:~/go/src/GRS$ kill -9 ps -ef |grep go |grep -v grep |awk {print $2}';kill -9 ps -ef |grep go |grep -v grep |awk '{print $2}';go run resource-management/test/resourceRegionMgrSimulator/main.go --region_name=Beijing --rp_num=10 --nodes_per_rp=25000 --master_port=9119 --data_pattern=Daily -v=3 > ~/TMP/simulator1.log.2022-09-22.v000184 2>&1 & ubuntu@ip-172-31-7-9:~/go/src/GRS$ kill -9 ps -ef |grep go |grep -v grep |awk '{print $2}';kill -9 ps -ef |grep go |grep -v grep |awk '{print $2}'; go run resource-management/test/resourceRegionMgrSimulator/main.go --region_name=Shanghai --rp_num=10 --nodes_per_rp=25000 --master_port=9119 --data_pattern=Daily -v=3 > ~/TMP/simulator2.log.2022-09-20.v000184 2>&1 &

-- 20 scheduler clients on 2 machines in us-west-2

ubuntu@ip-172-31-4-135:~/go/src/GRS$ for i in {1..10}; do sleep 1; go run resource-management/test/e2e/singleClientTest.go --service_url=ec2-52-11-1-193.us-west-2.compute.amazonaws.com:8080 --request_machines=25000 --action=watch --repeats=1 --limit=26000 --request_timeout=30m -v=3 > ~/TMP/scheduler.6.$i.log.2022-09-22.v000184 2>&1 & done ubuntu@ip-172-31-1-189:~/go/src/GRS$ for i in {1..10}; do sleep 1; go run resource-management/test/e2e/singleClientTest.go --service_url=ec2-52-11-1-193.us-west-2.compute.amazonaws.com:8080 --request_machines=25000 --action=watch --repeats=1 --limit=26000 --request_timeout=30m -v=3 > ~/TMP/scheduler.8.$i.log.2022-09-22.v000184 2>&1 & done

--- redis store on same machine of GRS in us-west-2

ubuntu@ip-172-31-8-82:~/go/src/GRS$ redis-cli -h ec2-52-11-1-193.us-west-2.compute.amazonaws.com -p 8389 info keyspace;date

Keyspace

db0:keys=500021,expires=0,avg_ttl=0 Fri Sep 23 00:06:57 UTC 2022

---singleNodeInquiry from machine in us-east-2

ubuntu@ip-172-31-23-203:~/go/src/GRS$ go run test/e2e/singleNodeQuery.go -v=3 --service_url=ec2-52-11-1-193.us-west-2.compute.amazonaws.com:8080 --single_node_num=100 --batch_node_num=100 --remote_redis_port=8389 > ~/TMP/nodeuery.log.2022-09-22.v000184 2>&1 &

I0923 01:09:56.794232 196615 stats.go:39] [Metrics][Nodes]QueryInterval: 1s, Number of nodes queried during each interval: 1, perc50: 48.800672ms, perc90: 48.962408ms, perc99: 49.536601ms. I0923 01:09:56.794634 196615 stats.go:39] [Metrics][Nodes]QueryInterval: 1m0s, Number of nodes queried during each interval: 100, perc50: 111.418732ms, perc90: 126.248211ms, perc99: 130.527159ms.

Also, the codes with branch simulator-list-watch have been tested in AWS integration test environment (5 million nodes/5 regions x 40 RPs x 25K nodes/100 scheduler clients x 25K nodes) successfully.

[yb01]

let's change all the testing scripts in the one PR to ensure test pass for each PR

[sonyafenge]

you may need merge this pr to CentaurusInfra:simulator-list-watch because all our current changing and testing are based on this branch

[yb01]

do you intend to merge this to main or wait for PR #180 merged to the LW branch first ? and then port from the LW branch to main ?

[q131172019]

do you intend to merge this to main or wait for PR https://github.com/CentaurusInfra/global-resource-service/pull/180 merged to the LW branch first ? and then port from the LW branch to main ?

Answer: This PR171 can be merged into main branch after complete test. The PR180 also can be merged into simulator-list-watch branch as well after complete test.

[sonyafenge]

there is a change for the definition of function NewRedisClient, all call should be updated per change. Please also change the call from ./test/e2e/singleNodeQuery.go