search

architecture-building-systems / CityEnergyAnalyst

The City Energy Analyst (CEA)
https://www.cityenergyanalyst.com/
MIT License
188 stars 59 forks source link

Tests pre- and post-PR differ #3577

Closed MatNif closed 3 weeks ago

MatNif commented 1 month ago

Describe the bug It seems that the tests that are run to check if a branch is valid in a PR and the tests that are run to check if a commit to master is valid differ from one-another. The latest commit to master cd9df1a is one such example. While in the PR all tests were successful.

Expected behaviour I would typically expect the test pre- and post-PR to be identical, or for the test in the PR to be more comprehensive than the ones that are run on a specific commit after the branch is merged. Therefore I would propose that we adapt the tests.

reyery commented 1 month ago

For PRs, we are just running the "quick" version of the tests, while for pushes to master we are running the "slow" version of the tests. The only reason why this was done was to save time for merging PRs (20 min vs 1h 20 min). But I do see the point of running the same test for both situations. I am figuring out a way to make the "slow" version faster (#3580) which could help with this situation.

reyery commented 3 weeks ago

Now PR and push to master runs the same test workflow (#3599)