[CI] Testing script.

[petro-rudenko]

SparkUCX CI testing script. For now, it assumes:

• UCX is built and installed

• SparkUCX jar file also provided

All other parameters are documented. Examples to run:

• 1 physical instance 3 spark processes: $NODELIST="jazz30" PROCESSES_PER_INSTANCE=3 SPARK_LOCAL_DIRS=/scrap/hadoop_tmp/spark ./test.sh

• 3 instances 1 spark process per instance: NODELIST="jazz30 jazz31 jazz32" SPARK_LOCAL_DIRS=/scrap/hadoop_tmp/spark ./test.sh

[amaslenn]

@petro-rudenko are you going to use it for CI testing? Can we prepare a Docker image and put all configuration stuff inside? Dockerhub images can be pulled from AZP pipeline.

[amaslenn]

@petro-rudenko I've just added one agent to support SparkUCX jobs, could you please re-trigger this PR? Force-push, new changes, anything.

[yosefe]

/azp run

[amaslenn]

Currently, only IPv6 is configured. Is IPv4 a requirement?

[petro-rudenko]

@amaslenn better to have. Theoretically should be possible to run with IPv6

[amaslenn]

@petro-rudenko currently testing scripts looks for IPv4 configured devices and skips others. Will you update the script or it is better to have IPv4?

[petro-rudenko]

@amaslenn - i took this testing scripts from UCX client-server jenkins. Let's have IPv4.

[petro-rudenko]

@amaslenn ping when Ipv4 would be ready, i'll rerun test.

[amaslenn]

@amaslenn ping when Ipv4 would be ready, i'll rerun test.

IPv4 is up.

[amaslenn]

@petro-rudenko you can go to swx-jupiter04 for local testing.

[petro-rudenko]

/azp run

[petro-rudenko]

Something weird, it closed PR, had to rebase commits.

[sonarcloud[bot]]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
0 Code Smells

No Coverage information
No Duplication information