search

datastax / cassandra-quarkus

An Apache Cassandra(R) extension for Quarkus
Apache License 2.0
40 stars 28 forks source link

JAVA-2736: Speed up TestContainers startup time #7

Closed emerkle826 closed 4 years ago

emerkle826 commented 4 years ago

There are a few configurations you can tweak to make Cassandra startup faster (disabling gossip and vnodes, for example). With TestContainers, the ability to control this is a little limiting. It allows you to provide a configuration override when creating the container, but the override is simply a directory that is mapped to the Cassandra config directory (/etc/cassandra inside the container).

With this approach, settings can be tweaked, but you have to provide entire files for anything you wish to override: you can't simply provide partial overrides. This PR incorporates a few tweaks borrowed from here that result in about a 25 - 30% reduction in total Integration test time (locally at least).

emerkle826 commented 4 years ago

NOTE: The cassandra.yaml and cassandra-env.sh files I've added in the PR were obtained by spinning up an instance of the Docker container the tests currently use and copying them out. Also, the default Docker container for TestContainers Cassandra is a 3.11.2 build. If the tests decide to change the Cassandra version, or introduce multiple versions, the approach in this PR may have to change as well.

tomekl007 commented 4 years ago

Wow, this is really a lot of code - I am a bit afraid about further maintenance of it. Isn't there a way to configure only differences using programmatic API? (maybe using other with* methods)

emerkle826 commented 4 years ago

Wow, this is really a lot of code - I am a bit afraid about further maintenance of it. Isn't there a way to configure only differences using programmatic API? (maybe using other with* methods)

I've looked at the API provided via Testcontainers and it doesn't currently allow for updating parts of file resources. But you have valid points as maintaining these config files is not going to be easy and makes this test approach brittle.

I'll open a new PR to investigate ways to possibly enhance Testcontainers itself to allow for partial config overrides. For now, I'm converting this PR into a Draft PR.

olim7t commented 4 years ago

But you have valid points as maintaining these config files is not going to be easy and makes this test approach brittle.

💯 I too am worried about long-term maintainability, especially as the image gets updated and/or multiple Cassandra versions are supported.

Would the changes be generally beneficial to anyone using TestContainers? We could try to get them into the base images directly, or maybe there is a way to host our own images.

adutra commented 4 years ago

Created JAVA-2781 to pursue with the idea of contributing to TestContainers. I think we close this for now.

adutra commented 4 years ago

Also create an umbrella epic link for TestContainers contributions: https://datastax-oss.atlassian.net/browse/JAVA-2787.

adutra commented 4 years ago

@emerkle826 I think we can close this one for now? We need to improve things on the TestContainers side first, but once that's done, we likely won't need the whole cassandra.yaml file here anyways.

emerkle826 commented 4 years ago

@emerkle826 I think we can close this one for now? We need to improve things on the TestContainers side first, but once that's done, we likely won't need the whole cassandra.yaml file here anyways.

Agreed. I'll revisit this after the TestContainers improvements.