search

lalbuild / lal

A strict, language-agnostic build system and dependency manager
MIT License
4 stars 1 forks source link

Copy components during testing to a temporary directory #40

Closed bencord0 closed 4 years ago

bencord0 commented 4 years ago

This makes individual test runs a bit cleaner and stops occasions where compiled output (or intermediates) affects another test that reuses that component.

This is particularly painful if the intermediates were created in one environment, and make doesn't rebuild the object files with the new toolchain.

clux commented 4 years ago

you planning on doing a cleanup stage in the tests or is this one of those "fine to overwrite", idempotent type of tests?

bencord0 commented 4 years ago

on doing a cleanup stage in the tests

I'm hoping to always copy the test components into the temp directory which will be cleaned up at the end of the test run.

I saw this [1] and it made me a bit suspicious about what could happen if the tests themselves broke and left the components (which are really fixtures) in an odd state.

By running the tests on a copy of the components, then we can have tests manipulate the component (and fail) without risking other tests in the run. I still need to figure out how we can remove the need to set LAL_CONFIG_HOME, but if that happens, then we can remove the #[serial] annotation and run the tests in parallel without them clobbering each other.

1: https://github.com/lalbuild/lal/pull/40/files#diff-6acdec8e30dcec350df19ba9b3ec41d0L579

clux commented 4 years ago

Yeah, that's sensible.

By running the tests on a copy of the components, then we can have tests manipulate the component (and fail) without risking other tests in the run. I still need to figure out how we can remove the need to set LAL_CONFIG_HOME, but if that happens, then we can remove the #[serial] annotation and run the tests in parallel without them clobbering each other.

You probably have to make lal initialization take a path rather than read the evar itself. That way you can have main.rs pass the evar so tests can customize it.