ITDE: Unify ports for database, BucketFS, and SSH

[tomuben]

Background

Currently ITDE uses custom ports for

• DB (8888)
• BucketFs (6583)

It would be better from a customer point of view to use the standard ports:

• DB (8563)
• Optional: BucketFS (2580?)
  • Compare with https://github.com/exasol/bucketfs-client/blob/main/doc/user_guide/user_guide.md which uses 2580
  • Running an AWS deployment it uses port 2581 per default
  • -> Looking at the Exasol documentation it looks like there is no default port. I believe it would be better to keep the current port (6583) as is.

Acceptance Criteria

• Change Db port to 8563
• Change bucketFS port to 2580 (please verify with default port in official documentation)
• References to literal port numbers in file pytest_itde/__init__.py are replaced by constants of class Ports:
  • config.OptionGroups BUCKETFS and EXASOL
  • _bootstrap_db(), start_db(), ssh_port_forward
• Update documentation (for example https://exasol.github.io/integration-test-docker-environment/main/user_guide/user_guide.html#accessing-the-environment-information)

This ticket also closes #341

[ckunki]

Approved from my side, effort negligible

[ckunki]

Alignment in daily:

1. remove current branch
2. re-apply required changes after PR #339 has been merged (forward SSH port)

[Nicoretti]

Approved from my side, effort negligible

Well changing the default's may be negligible, but that's probably not true for all the ITDE call sites (users, projects). Keep in mind, changing defaults is a breaking change. Don't get me wrong, changing the defaults can be legit but the ripple effects should be considered.

• All ITDE dependents potentially also need to do some work

[redcatbear]

Blocked by: #329

[ckunki]

The files EXAConf in folder docker_db_config_template also contain port specifications:

[Node : 11]
  ExposedPorts = 8888:8899, 6583:6594

[BucketFS : bfsdefault]
    HttpPort = 6583

[DB : DB1]
    Port = 8888

Should we change these as well?

[MarleneKress79789]

The files EXAConf in folder docker_db_config_template also contain port specifications:

[Node : 11]
  ExposedPorts = 8888:8899, 6583:6594

[BucketFS : bfsdefault]
    HttpPort = 6583

[DB : DB1]
    Port = 8888

Should we change these as well?

You have to change these in the files that are not sym links(7.0, 7.1, 8.XX) and then run the update_packaging.sh which will update files in exasol_integration_test_docker_environment/docker_db_config. not sure about the > ExposedPorts though

[ckunki]

Thank to @MarleneKress79789 ! And thanks to @tkilias for discussing and improving which ports to use throughout the code. Most findings have been addressed. Please feel free to review!

[redcatbear]

@ckunki neads a pair session with @tkilias to continue.

[ckunki]

Hi @tkilias on Friday finally I was successful. Could you please review PR #354 again?

[ckunki]

To ensure compatibility with multiple versions of Exasol database ITDE employs a matrix build executing its tests for each version. Lately the build failed for database version prerelease-8.17.0.

Since version 8.x the size of the Docker Container of Exasol database climbed from 6 GB to more than 10 GB. Additionally we found out that besides the version according to the build matrix the build did also use the latest (default) version of the database. These especially were the integration tests for ITDE's pytest plugin in test/integration/pytest_itde_test.py.

So when ITDE's build matrix specifies version 8.17 and the test additionally downloads the latest version 8.18 this resulted in error message "no space left on device". If the developer's local machine had more disk space the error did not show there.

To fix the problem we added @pytest.mark.skipif to the test cases of ITDE's pytest plugin in order to only run these tests for the latest version of Exasol database — verifying the plugin only for a single version of the database is sufficient.

The easiest way to skip the tests seemed to evaluate the environment variable EXASOL_VERSION. As the tests for ITDE's pytest plugin are written in pytest as well we additionally needed to forward the environment variables to all pytest integration tests in file noxfile.py, method run_tests():

session.run(
        "pytest",
        "--itde-db-version", db_version,
        "./test/integration/pytest_itde_test.py",
        env=env,
    )