Allow injection of external test container via CLI

[tomuben]

Background

Often, clients of the Integration-Test-Docker-Environment have their specific package dependencies. In order to run tests within the test-container, they cannot use the built-in test container, but need to inject a custom Docker container, with their respective dependencies.

Currently, the default test-container is not part of the release package. Thus, the CLI does not work when startet from a package.

Details

Default Test-Container

We can remove the test-container of the ITDE. There will be no default test-container.

Database starter check

We can use exaplus in the docker-db to validate if the db is up and running

Test-data

Test Data can move to exaslct and is not needed anymore in itde. It should be small enough that we can include it into the resources of the package. (Please check the last part). Or, alternative we can download it.

The test-data folder is currently also hard-coded. It will be the client's responsibility to populate test-data. However, itde needs to bind mount the local directory with the test-data. So, this path needs to be passed as parameter to itde.

itde will return a flag from the API call if the test-environment was re-used. It's the clients responsibility to evaluate the reused flag and populate the test-data if this flag is false.

For convenience, class PopulateData will remain in itde so that clients can use it.

Integrations-python-tests

These tests are bundled today together with the test-data (see above). So there are bind-mounted together with them during startup of the test-container. Also, in the current implementation the tests are copied to a new directory in the test-container, in which the client then starts the integration tests (This is done because of some artifacts which the test-runner stores in the directory).

Hence, we also need to pass the bind-mount information (source directory and directory in test-container) to the itde via API. Additionally, there will be a third parameter, indicating the final location in the test-container, where the tests are then expected.

Additional directories for test-container

Add parameter for mapping_of_build_files_and_directories. Need to discuss how to declare the parameter in CLI; in API it can be a simple dictionary: str -> str. As discussed, CLI will not support test-container at all. So there is no need to pass mapping over CLI.

Over API the mapping of files/directories must be passed. These mapping files then will be copied together with the Dockerfile into a temporary directory (this behavior is already implemented).

Upload to BucketFS

Currently two kinds of assets are uploaded to the Bucket-FS:

1. exajdbc.jar: We decided to remove this as this is not needed anymore.
2. slc container (from exaslct)

We will modify UploadFileToBucketFS so that it will run on the host-system, not within the test-container. With that, there is no need to bind-mount any upload-directories to the test-container. The clients (exaslct) can use this class to upload any specific artifacts then. For that, we need to change UploadFileToDb so that it uses project bucketfs-utils, and not curl.

Conclusion #

We need to pass the following information regarding the test-container:

• The path of the Dockerfile
• Additional mappings, and the relative paths of the source files/directories
• A list of mapping for bind-mounts (Optional). One use case is the path of the test data. Each entry of the mapping consists of:
  • The source directory on the host system
  • The bind-mounted directory in the test-container
  • The directory to which the content will be copied after the test container has been started. (if None, the copy will skip)

Acceptance Criteria

1. Allow injection of a path containing custom test-container Dockerfile
2. Encapsulate all information needed for the test-container (See section Conclusion) in a class : TestContainerContentDescription 3.. API: Add a parameter of type TestContainerContentDescription to API function _spawn_testcontainer() 4. CLI: Add a parameter --test-container-content-description-file to CLI command _spawn_testcontainer(), which should be treated as a JSON file containing all content of the test-container (See section Conclusion)
3. No new CLI parameter needed. The functionality to spawn the test-container should only be available via API.

[tkilias]

Hint: the test container is currently hard-coded in https://github.com/exasol/integration-test-docker-environment/blob/859c34bcb2f7fa9345d1175c99db581d546c4a5f/exasol_integration_test_docker_environment/lib/test_environment/analyze_test_container.py#L13

[tkilias]

We can create a AnalyzeTestContainer Task where we set Dockerfile and BuildContext from commandline parameter. For example, --path-to-dockerfile, --path-to-build-context (can be multiple)

We need pass the commandline parameter values into the SpawnTestEnvironment where they passed through to the SpawnTestContainer

[tomuben]

Code hints

Add parameter

https://github.com/exasol/integration-test-docker-environment/blob/ab78c537a8cdc87[…]cker_environment/lib/test_environment/analyze_test_container.py https://github.com/exasol/integration-test-docker-environment/blob/ab78c537a8cdc87[…]cker_environment/lib/test_environment/analyze_test_container.py

Return parameter

https://github.com/exasol/integration-test-docker-environment/blob/ab78c537a8cdc87[…]cker_environment/lib/test_environment/analyze_test_container.py

Call with parameter

https://github.com/exasol/integration-test-docker-environment/blob/ab78c537a8cdc87[…]gration_test_docker_environment/lib/api/build_test_container.py https://github.com/exasol/integration-test-docker-environment/blob/ab78c537a8cdc87[…]nvironment/lib/test_environment/create_ssl_certificates_task.py https://github.com/exasol/integration-test-docker-environment/blob/ab78c537a8cdc87[…]docker_environment/lib/test_environment/spawn_test_container.py https://github.com/exasol/integration-test-docker-environment/blob/ab78c537a8cdc87[…]egration_test_docker_environment/lib/api/push_test_container.py