Using environment variables to cofigure tests

[Ilia-Kosenkov]

Is it a good practice to use environment variables to configure tests? rextendr can be used for testing extendr (and in other situations), but for this, we need to configure patch.crates_io and, possibly, toolchain. This can be achieved using env vars, but I am not sure if this is a good practice.

[clauswilke]

I'm not sure I understand the question, but I'd say the right way to do it is to write some R code that uses rextendr to run appropriate tests. The testing code then might read the status of some env vars, but rextendr itself would not.

For example, the extendrtests package in extendr could depend on rextendr and call rust_source().

[Ilia-Kosenkov]

@clauswilke, What I had in mind is using Sys.getenv within tests (or caching this value in some global variable). E.g. rust_source(..., toolchain = Sys.getenv("REXTENDR_TOOLCHAIN")), but I understand why it is not a good scenario.

[clauswilke]

I think the tests that are shipped as part of rextendr should not depend on env vars, or at least should not require env vars to run properly. You need to be able to run the tests anywhere. If you want to add tests that are only run when specific env vars are set, that's a different story and should be fine. vdiffr does that to determine whether or not to actually run the visual tests: https://www.rdocumentation.org/packages/vdiffr/versions/0.3.3/topics/expect_doppelganger

[Ilia-Kosenkov]

@clauswilke, My intention was to have tests that, when specific var is set, slightly change their behavior. For instance, on Windows it may be useful to test stable-x86_64-msvc, stable-x86_64-gnu, stable-i686-gnu to track how cross-compilation works. For that, I can set env var and run rmcdcheck without touching the tests' code. This is useful at least locally. By default, tests will be executed as-is.

Another example would be an ability to specify patch.crates_io using env var to point rextendr to a particular branch or commit of extendr -- again for debugging & testing purposes and with no test code modifications.

[clauswilke]

Yes, I think that's fine.

[Ilia-Kosenkov]

@clauswilke, Thank you for the feedback. If I find this feature useful for local tests (and easier to use than other solutions I have at hand), I will open a PR to discuss this further.