search

zephyrproject-rtos / zephyr

Primary Git Repository for the Zephyr Project. Zephyr is a new generation, scalable, optimized, secure RTOS for multiple hardware architectures.
https://docs.zephyrproject.org
Apache License 2.0
10.92k stars 6.65k forks source link

tests: add integration_platforms and misc optimizations #81740

Open nashif opened 15 hours ago

nashif commented 15 hours ago

Add integration_platforms to many tests that use platform_allow to manage scope of pull_request CI.

Signed-off-by: Anas Nashif anas.nashif@intel.com

nashif commented 3 hours ago

Thanks for looking into this. A couple of questions:

  • How did you pick the integration platforms?

random or by feel

  • How much would we gain from removing [testing on] native_posix altogether?

Isnt this just going to be dropped and deprecated soon? Isnt this covered by what native_sim already provides?

The answer to "How much" here is going to be "not much", but "not much" * 1000 is alot and our CI performance is suffering because of small things all over the place. If you do need that and it does give you coverage you do not get anywhere else, we can leave it, but looking at many tests/samples we have lots of worng use and duplication that all adds up.

henrikbrixandersen commented 3 hours ago

The answer to "How much" here is going to be "not much", but "not much" * 1000 is alot and our CI performance is suffering because of small things all over the place. If you do need that and it does give you coverage you do not get anywhere else, we can leave it, but looking at many tests/samples we have lots of worng use and duplication that all adds up.

My point was that we could drop native_posix from testing all over now and harvest the gain.

nashif commented 3 hours ago

My point was that we could drop native_posix from testing all over now and harvest the gain.

yeah, I think @aescolar is already looking into that :)

aescolar commented 1 hour ago

Isnt this just going to be dropped and deprecated soon? Isnt this covered by what native_sim already provides?

My point was that we could drop native_posix from testing all over now and harvest the gain.

yeah, I think @aescolar is already looking into that :)

native_posix is already deprecated and will be removed in ~3months, when we start the 4.2 cycle. Anyhow, it is not anymore a default test platform. So we should not be building for it in PRs anymore. All tests that were explicitly setting int. platform to native_posix have been switched to native_sim. And native_posix was removed from most platform_allow filters (expect where there was a significant enough difference for what was testing when built for this platform to warrant leaving it there). So at this point it is built mostly only on weekly. Meaning, the cost today is very small, and it is just a test for the platform itself while we support it/until we remove it.