tests: posix: common: separate into smaller testsuites

[cfriedt]

Is your enhancement proposal related to a problem? Please describe. The posix.common testsuite contains a lot of mostly unrelated testsuites. As a result, the code size is large, but mostly the data size is extremely large.

It would require less variation from default settings, and would reduce ram requirements to split that testsuite into smaller groups of more closely related testsuites.

Describe the solution you'd like

The obvious way to group related testsuites is by Option Group.

E.g.

• [ ] POSIX_RW_LOCKS - #80943
• [ ] POSIX_SEMAPHORES - #80964
• [ ] POSIX_SIGNALS - #80930
• [ ] POSIX_SINGLE_PROCESS - #80981
• [ ] POSIX_SPIN_LOCKS - #80965
• [ ] POSIX_THREADS_BASE
• [ ] POSIX_THREADS_EXT
• [ ] POSIX_TIMERS - #80807
• [ ] XSI_SYSTEM_LOGGING - #80986

and potentially others.

testcase.yaml permutations can be designed to enable or disable features specific to one Option Group, like TSS, TSA, TSH (see Codes).

Additionally, such a grouping would theoretically improve bug attribution capabilities; if an error was traced (either via stacktrace or bug description) to a certain posix function, there would be a 1:1 correspondance to an Option Group, so typically, only that testsuite would need to be executed.

Describe alternatives you've considered The way the testsuite has been designed historically.

Additional context This came up in the context of #79443 but it was already thought of as a nice-to-have.

[zacck]

Can I take a stab at this one please, if it's ok could it be assigned to me?

[cfriedt]

@zacck - please do!

[zacck]

@cfriedt I'm new to Zephyr and while I have looked at the links and docs supplied above I believe I need an example to start breaking this down would you be able to assist with some information that would help here?

[ycsin]

@zacck If you have just started with Zephyr, I think you will need to be familiarized with Twister and the different KV pairs in the testcase.yaml, see Test Runner (Twister)

@cfriedt do you expect the zephyr/tests/posix/common test to be broken down into multiple tests? Because while we can break the test into multiple sub-tests within the tests/posix/common/testcase.yaml, the existing testcase.yaml would be pretty big when we want to make sure that each sub-test are tested with different configurations (different C libs, spin validation, tls, ...) that we have: (<number of option group sub-test> x <number of configurations>)

[cfriedt]

@cfriedt do you expect the zephyr/tests/posix/common test to be broken down into multiple tests?

They should be broken down into separate app directories.

That's mainly because the single application is getting quite large, and the configuration requirements are not always clear.

It would be better if the prj.conf and memory requirements were minimal.

[cfriedt]

It could be good to split this into smaller commits / a task list.

[zacck]

@cfriedt yeah breaking this down would be awesome, I am going to do that once I have a solid plan ahead which should be around tomorrow 16th OCT 24.

[cfriedt]

@cfriedt yeah breaking this down would be awesome, I am going to do that once I have a solid plan ahead which should be around tomorrow 16th OCT 24.

Cool - it can be separate PRs, which should simplify things. I'll do pthreads, because it is likely the most confusing.

[zacck]

@cfriedt yeah breaking this down would be awesome, I am going to do that once I have a solid plan ahead which should be around tomorrow 16th OCT 24.

Cool - it can be separate PRs, which should simplify things. I'll do pthteads, because it is likely the most confusing.

It might be more beneficial if you teach me how to do the most confusing parts, that way I and others can grow into being able to help with meatier stuff and contributors like you can get more done since you can effectively direct us.

I do understand that is would be slower and cumbersome, but do consider it enables us to help more.