Open analogrelay opened 1 week ago
After some discussion with @heaths, I'm going to change this a bit already: Instead of TestAccount::from_env()
, use a macro: get_test_account!()
(or similar). That macro will return Ok(())
if the environment variable isn't set, thus effectively skipping the test.
Ok, here's a different approach, based on some discussions in Teams.
#[cfg_attr(not(livetest), ignore]
, which applies the #[ignore]
attribute if --cfg=livetest
is not specified when rustc
is building the tests.--cfg
value, we specify a Cargo profile called livetest
, which ensures that the build of live tests doesn't interfere with the dev build, causing the build cache to be invalidated. Ideally, we'd set the --cfg
value as part of this Cargo profile, but that feature is still unstable and requires nightly cargo.cargo/config.toml
. Running cargo livetest ...
will be equivalent to running cargo test
with --profile=livetest
(to use a separate build cache) and RUSTFLAGS=--cfg=livetest
(to compile-in the live tests themselves).With this, if you run the following command: cargo test --all-features --package azure_data_cosmos --test integration_databases
(to run only "non-live" tests), you get this output:
Running tests/integration_databases.rs (target/debug/deps/integration_databases-acf8e6c1c2544f49)
running 1 test
test database_crud ... ignored
The database_crud
test has been ignored, as desired.
When you run cargo livetest --all-features --package azure_data_cosmos --test integration_databases
(to run all tests, including livetests), but without specifying mandatory env vars, you get this output:
Running tests/integration_databases.rs (target/livetest/deps/integration_databases-ba888bd60bb98b5c)
running 1 test
test database_crud ... FAILED
failures:
---- database_crud stdout ----
Error: "failed to read AZURE_COSMOS_CONNECTION_STRING environment variable: environment variable not found"
failures:
database_crud
The "live" test has correctly failed because you requested to run it but failed to provide mandatory configuration. If you then configure the AZURE_COSMOS_CONNECTION_STRING
value correctly, and re-run the test, it runs as expected:
Running tests/integration_databases.rs (target/livetest/deps/integration_databases-ba888bd60bb98b5c)
running 1 test
test database_crud ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.18s
This is on hold as we get the "official" plan for integration/live testing in place.
Here's a first bit of infrastructure for integration tests for
azure_data_cosmos
, and a single test.To run these tests:
AZSDK_COSMOS_CONNECTION_STRING
environment variable to a valid Connection String to a real Cosmos DB account (the Emulator isn't quite supported yet, I'm having TLS issues that I need to find a good solution for, the emulator has a self-signed certificate).--features integration_tests
I still need to work out how to integrate these with CI, I'll start a separate conversation to discuss that.
Also, as part of running the tests locally, I discovered some failures that were masked by the fact that we don't run
--all-features
when testing from the CI and these tests depend on thekey_auth
feature. We probably can't use--all-features
(because some tests require setup) but perhaps we should have a list of features we enable during "unit testing" 🤔 .