search

zephyrproject-rtos / zephyr

Primary Git Repository for the Zephyr Project. Zephyr is a new generation, scalable, optimized, secure RTOS for multiple hardware architectures.
https://docs.zephyrproject.org
Apache License 2.0
10.93k stars 6.65k forks source link

POSIX Roadmap for LTSv3 #51211

Open cfriedt opened 2 years ago

cfriedt commented 2 years ago

Introduction

POSIX libraries and applications should Just Work™️ in 90% of the cases when being ported to Zephyr. Why? Because we want to enable our users to launch products and applications to the best of our ability, and we want to lower the barrier to entry by enabling users to leverage portable APIs.

This is an umbrella issue that will outlines work to be done in the POSIX API prior to LTSv3 (Ideally complete by v3.5).

Problem description

As discussed at EOSS 2023 and EOSS 2024, Zephyr's POSIX API layer needs some improvements. These can be broken down into the following:

  1. Maintainability
  2. Application / libc / toolchain Interface
  3. Application / library Portability

Proposed change

Implement all Option Requirements of POSIX Subprofiling Option Groups required for PSE51, PSE52, and most of PSE53 compatibility.

aep

Update the POSIX areas at docs.zephyrproject.org. For reference, the top level page of Zephyr's POSIX documentation.

https://docs.zephyrproject.org/latest/services/portability/posix/index.html

Maintainability addresses the needs of the maintainer, contributors, and (to some extent) the users.

Interface addresses the needs of

Portability addresses the needs of the users who should (rightfully) expect that the POSIX API layer behaves according to the specification.

Detailed RFC

Proposed change (Detailed)

Maintainability

The layering issues that exist between subsys/net and lib/posix can be described as a cyclic dependency. The issue can be resolved via mutual dependency.

I.e. Currently, we have:

┌───────────┐       ┌───────────┐
│           ├──────►│           │
│   POSIX   │       │   NET     │
│           │◄──────┤           │
└───────────┘       └───────────┘

But we need

          ┌───────────┐
          │           │
          │   ZVFS    │
          │           │
          └▲────────▲─┘
           │        │
           │        │
┌──────────┴┐       ├───────────┐
│           │       │           │
│   POSIX   ├───────►   NET     │
│           │       │           │
└───────────┘       └───────────┘

Diagrams made with https://asciiflow.com/

I think we may have settled on the name "ZVFS" in the above diagram.

Interface

[!TIP] Check the Current status of Suprofiling Option Group support from docs to verify that the items below are up to date

Dependencies

Concerns and Unresolved Questions

Alternatives

There was a previous massive PR that attempted to address a number of the goals at once (#43987), but it is impossible to review in the current state and needs to be broken down and simplified in separate individual PRs.

carlescufi commented 1 year ago

Architecture WG:

romkell commented 1 year ago
cfriedt commented 1 year ago
  • @cfriedt: when ever implementing dynamic features in the scheduler / core code (not yet defined safety scope) please consider safety where dynamic memory management and using heaps is problematic to get TÜV certification.

Absolutely

But I remember that we had that discussion earlier already and solved that by layering the dynamic part on top of the core code which can be passed statically OR dynamically allocated objects such as stack buffers, thread contexts etc. what so ever. I feel you are aware of that.

Yeah, so the suggested approach from an earlier PR was to keep thread stack allocation separate and selectable, so that will still be the case. However, the proposed kernel-level API will also allow

romkell commented 1 year ago

Yeah, so the suggested approach from an earlier PR was to keep thread stack allocation separate and selectable, so that will still be the case. However, the proposed kernel-level API will also allow

  • a static, pre-allocated pool of thread stacks to be used
  • dynamic, heap-allocated thread stacks to be used
  • both of the above (with a configurable preference)
  • and of course neither

Like it!

cfriedt commented 1 year ago

Would be kind of nice to be able to use socketpair() without networking. I wonder if something can be borrowed from eventfd.. 🤔

abrodkin commented 1 year ago

@cfriedt speaking of POSIX PSE profile test-suites (in the original issue description above), it might be a bit more correct to refer to "embedded versions" of those, such as:

abrodkin commented 1 year ago

@cfriedt you updated names. but references are still to non-PSE pages ;)

cfriedt commented 1 year ago

@cfriedt you updated names. but references are still to non-PSE pages ;)

Actually, all three are referenced from "Conformance Test Suites" already. I was just trying to provide a reference to the higher scope specifications as well.

In any case, the information is available.

cfriedt commented 5 months ago

TSC Attention Required

Please refer to

POSIX System Interfaces POSIX Application Environment Profiles

Why should the PRs below be considered for approval after Feature Freeze?

What are the PR's we are asking to merge?

Allow toolchains / platforms to override POSIX Option Groups via Kconfig (mainly for maintainers, libc compatibility)

Add stubs for the POSIX_SIGNALS Option Group (i.e. signalling processes. We do not yet have processes in Zephyr, so stubs for functions that are simply not supportable)

Implement POSIX_DEVICE_IO Option Group (depended on mmap, so a bit delayed. lays groundwork for zvfs, extremely useful in terms of reducing api-layer dependency cycles)

Implement POSIX_FD_MGMT Option Group (zvfs_poll(), zvfs_select(), extremely useful in terms of reducing api-layer dependency cycles)

What are we leaving behind?

System Inferfaces (Base Definitions)

For most practical purposes, the System interfaces are feature complete 🎉(with caveats).

PSE51 - Minimal Realtime System Profile

For most practical purposes, PSE51 is feature complete 🎉(with caveats).

PSE52 - Realtime Controller System Profile

PSE53 - Dedicated Realtime System Profile

Why did some features not make it in for Feature Freeze?

Post-freeze Bug-fixes and Stability-fixes (possible backports)

cfriedt commented 4 months ago

Updated comment to reflect that the the posix device_io, signals, and fd_mgmt features were reverted.

This puts LTSv3 in a kind of dilapidated state from the POSIX & libc perspective.