apache / pulsar-client-go

Apache Pulsar Go Client Library
https://pulsar.apache.org/
Apache License 2.0
652 stars 335 forks source link

Use pulsar-client-go-test docker to test TestBlueGreenMigrationTestSuite case #1282

Open geniusjoe opened 2 weeks ago

geniusjoe commented 2 weeks ago

Is your feature request related to a problem? Please describe. When I ran make test, every test case worked fine except one error from test_extensible_load_manager :

 ✔ Network blue-green_green-pulsar  Created                                                                                0.0s 
 ✔ Container green-zookeeper        Healthy                                                                               15.9s 
 ✔ Container green-pulsar-init      Exited                                                                                15.2s 
 ✔ Container green-bookie           Started                                                                               15.3s 
 ✔ Container green-broker-2         Started                                                                               16.1s 
 ✔ Container green-broker-1         Started                                                                               16.1s 
 ✔ Container green-proxy            Started                                                                               16.1s 
until curl http://localhost:8081/metrics > /dev/null 2>&1 ; do sleep 1; done
# run blue-green migration test (run this test from this env to access both clusters)
go test -race -coverprofile=/tmp/coverage-blue_green_topic_migration -timeout=5m -tags extensible_load_manager -v -run TestBlueGreenMigrationTestSuite ./pulsar
go: github.com/99designs/keyring@v1.2.1 requires
        github.com/dvsekhvalnov/jose2go@v1.5.0: missing go.sum entry; to add it:
        go mod download github.com/dvsekhvalnov/jose2go
make: *** [test_extensible_load_manager] Error 1

This is because TestBlueGreenMigrationTestSuite currently uses host go environment to run test cases, and my environment doesn't have related dependencies:

# Makefile
test_extensible_load_manager: container
    ...
    # run blue-green migration test (run this test from this env to access both clusters)
    go test -race -coverprofile=/tmp/coverage-blue_green_topic_migration -timeout=5m -tags extensible_load_manager -v -run TestBlueGreenMigrationTestSuite ./pulsar
    go tool cover -html=/tmp/coverage-blue_green_topic_migration -o coverage-blue_green_topic_migration.html
    ...

I think maybe we would better to use pulsar-client-go-test test docker like other Makefile test cases to avoid dependencies conflict.

Describe the solution you'd like

  1. Since TestBlueGreenMigrationTestSuite use integration-tests/extensible-load-manager/docker-compose.yml as a blue cluster and use integration-tests/blue-green/docker-compose.yml as a green cluster, two clusters are in different network environments. So we may need to make them use the same network:
    
    #integration-tests/extensible-load-manager/docker-compose.yml
    version: '3'
    networks:
    pulsar:
    driver: bridge

integration-tests/blue-green/docker-compose.yml

version: '3' networks: extensible-load-manager_pulsar: external: true

2. There are same service names in two `docker-compose.yml` files, we may need to use new unique service names in `integration-tests/blue-green/docker-compose.yml` green cluster service:

Start ZooKeeper

green-zookeeper: container_name: green-zookeeper ... networks:

After

for _, scenario := range []topicUnloadTestCase{

    {
        testCaseName:  "proxyConnection",
        blueAdminURL:  "http://proxy:8080",
        blueClientUrl: "pulsar://proxy:6650",
        greenAdminURL: "http://green-proxy:8080",
        migrationBody: `
                    {
                        "serviceUrl": "http://green-proxy:8080",
                        "brokerServiceUrl": "pulsar://green-proxy:6650",
                    }
                `,
    },
}


**Describe alternatives you've considered**
I think we can use  `docker run --rm --network=host` in Makefile so that docker `pulsar-client-go-test` can have all docker network access priorities. But `network=host` option seems only work in Linux environment and can break platform compatibility.