Define arch-specific set of memory attributes

[carlocaione]

For #61009 it was requested in https://github.com/zephyrproject-rtos/zephyr/pull/61009#pullrequestreview-1557351778 to have a comprehensive list of memory attribute that could make sense for the different architectures and define also a common set that is valid for all.

This issue will be used to gather this data from the various architecture maintainers (and other fellow devs).

So, dear all in the following list: could you list any memory-attribute that you think it could be useful for your architecture and specify if you can think of any generic attribute that could make sense besides the one already listed down here?

https://github.com/zephyrproject-rtos/zephyr/blob/5664796d1f5b525dc27bb6dfc862c35e7c5b7b13/include/zephyr/dt-bindings/memory-attr/memory-attr.h#L11-L21

• [ ] ARM (@microbuilder @SgrrZhf @ithinuel)

• [ ] ARC (@ruuddw @evgeniy-paltsev)

• [ ] ARM64 (@carlocaione @povergoing)

• [x] POSIX (@aescolar)

• [ ] NIOS-2 (@nashif)

• [x] RISC-V (@fkokosinski @kgugala @tgorochowik)

• [x] Xtensa (@dcpleung)

• [ ] X86 (@jhedberg)

• [ ] others (@hubertmis @galak @teburd)

[povergoing]

I think @SgrrZhf and @ithinuel can also be involved in terms of ARM

[povergoing]

@carlocaione I might miss something, but what if someone wants to define an arch-specific mem attr (e.g. inner/outer sharable attr for Arm64), we still have a chance to customize and define the arch-specific mem attr right?

[carlocaione]

@carlocaione I might miss something, but what if someone wants to define an arch-specific mem attr (e.g. inner/outer sharable attr for Arm64), we still have a chance to customize and define the arch-specific mem attr right?

Yes, basically part of the mask of zephyr,memory-attr is reserved for architectures (8-bits): https://github.com/zephyrproject-rtos/zephyr/blob/5664796d1f5b525dc27bb6dfc862c35e7c5b7b13/include/zephyr/dt-bindings/memory-attr/memory-attr.h#L23-L32

So every architecture must have its own /include/zephyr/dt-bindings/memory-attr/memory-attr-ARCH.h file with the set of defines it cares about. For example for ARM64:

#include <zephyr/dt-bindings/memory-attr/memory-attr.h>
#include <zephyr/dt-bindings/memory-attr/memory-attr-arm64.h>

   mem: memory@10000000 {
       compatible = "mmio-sram";
       reg = <0x10000000 0x1000>;
       zephyr,memory-attr = <( DT_MEM_CACHEABLE | DT_MEM_ARM64_INNER_SHAREABLE )>;
   };

[fkokosinski]

For RISC-V, we would additionally need to have attributes to decide the following:

• the memory region type: is it main, I/O or empty/illegal?
• the support atomic ops (AMO class): swap, logical or arithmetic?
• for I/O memory type - is it idempotent or not?

So the list would be something like as follows:

DT_MEM_RISCV_TYPE_MAIN
DT_MEM_RISCV_TYPE_IO
DT_MEM_RISCV_TYPE_EMPTY
DT_MEM_RISCV_AMO_SWAP
DT_MEM_RISCV_AMO_LOGICAL
DT_MEM_RISCV_AMO_ARITHMETIC
DT_MEM_RISCV_IO_IDEMPOTENT_READ
DT_MEM_RISCV_IO_IDEMPOTENT_WRITE

[dcpleung]

For Xtensa, there are instruction ROM/RAM, data ROM/RAM and XLMI memory. So we will need these to indicate where the memory regions reside:

DT_MEM_XTENSA_INSTR_ROM
DT_MEM_XTENSA_INSTR_RAM
DT_MEM_XTENSA_DATA_ROM
DT_MEM_XTENSA_DATA_RAM
DT_MEM_XTENSA_XLMI

Also will need DT_MEM_XTENSA_IDMA for each region to indicate whether the integrated DMA engine can do DMA transfer on them.

[teburd]

Generally some form of indicator of connectivity/placement would be useful I'd think.

TCM, SOC, BOARD kind of indicators of some sort.

E.g. intel adsp has a bit of each in a way. NXP's RT series absolutely has all 3 if external sdram/hyperram is used.

These different regions may have varying behavioral differences you want to check pointers for.

[carlocaione]

Also will need DT_MEM_XTENSA_IDMA for each region to indicate whether the integrated DMA engine can do DMA transfer on them.

Uhm, this could be very well a generic attribute TBH, like DT_MEM_DMA or something similar.

[carlocaione]

Generally some form of indicator of connectivity/placement would be useful I'd think.

TCM, SOC, BOARD kind of indicators of some sort.

E.g. intel adsp has a bit of each in a way. NXP's RT series absolutely has all 3 if external sdram/hyperram is used.

These different regions may have varying behavioral differences you want to check pointers for.

@teburd which architectures are we talking about here?

[teburd]

Generally some form of indicator of connectivity/placement would be useful I'd think.

TCM, SOC, BOARD kind of indicators of some sort.

E.g. intel adsp has a bit of each in a way. NXP's RT series absolutely has all 3 if external sdram/hyperram is used.

These different regions may have varying behavioral differences you want to check pointers for.

At least arm (nxp imxrt) and xtensa (intel adsp) that I know of that could use this sort of memory locality attribute

[teburd]

Generally some form of indicator of connectivity/placement would be useful I'd think. TCM, SOC, BOARD kind of indicators of some sort. E.g. intel adsp has a bit of each in a way. NXP's RT series absolutely has all 3 if external sdram/hyperram is used. These different regions may have varying behavioral differences you want to check pointers for.

@teburd which architectures are we talking about here?

intel adsp is xtensa lx6/lx7, nxp imxrt is arm cm7/cm33/xtensa variations

[evgeniy-paltsev]

About the generic attributes: first of all I'm wondering why do we have two options for some attributes (CACHEABLE vs NON_CACHEABLE; VOLATILE vs NON_VOLATILE) and only one option for other (DT_MEM_OOO in case of out of order)?

 #define DT_MEM_CACHEABLE       BIT(0) /* cacheable */ 
 #define DT_MEM_NON_CACHEABLE       BIT(1) /* non-cacheable */ 
 #define DT_MEM_VOLATILE        BIT(2) /* volatile */ 
 #define DT_MEM_NON_VOLATILE        BIT(2) /* von-volatile */ 
 #define DT_MEM_OOO         BIT(3) /* out-of-order */ 

[carlocaione]

About the generic attributes: first of all I'm wondering why do we have two options for some attributes (CACHEABLE vs NON_CACHEABLE; VOLATILE vs NON_VOLATILE) and only one option for other (DT_MEM_OOO in case of out of order)?

 #define DT_MEM_CACHEABLE     BIT(0) /* cacheable */ 
 #define DT_MEM_NON_CACHEABLE     BIT(1) /* non-cacheable */ 
 #define DT_MEM_VOLATILE      BIT(2) /* volatile */ 
 #define DT_MEM_NON_VOLATILE      BIT(2) /* von-volatile */ 
 #define DT_MEM_OOO           BIT(3) /* out-of-order */ 

Yes, that is something I'm going to fix in the next push. All the _NON_ variants are going to be removed.

[ruuddw]

For ARC there are various memories, e.g. closely coupled instruction and data (ICCM, DCCM) and XY memory for improved DSP performance. And more, like cluster shared memory for multicore clusters. To be honest, I'm not fully sure about these extra attributes, or rather about how they will be used. It looks like main purpose is to avoid programming MPU in C code board initialization, but move that to DTS. That may work for some specific and static cases, but not sure if it provides enough flexibility to avoid some dedicate logic in C code board initialization. If the main purpose is programming of MPU attributes, shouldn't the attributes simply be aligned with the MPU programming attributes (user/kernel/supervisor read/write/execute etc.)?

[aescolar]

@carlocaione In the POSIX arch we are not modelling any kind of MPU, and type of access limitations do not really apply. So this attributed would be ignored.

[carlocaione]

For ARC there are various memories, e.g. closely coupled instruction and data (ICCM, DCCM) and XY memory for improved DSP performance. And more, like cluster shared memory for multicore clusters. To be honest, I'm not fully sure about these extra attributes, or rather about how they will be used. It looks like main purpose is to avoid programming MPU in C code board initialization, but move that to DTS. That may work for some specific and static cases, but not sure if it provides enough flexibility to avoid some dedicate logic in C code board initialization. If the main purpose is programming of MPU attributes, shouldn't the attributes simply be aligned with the MPU programming attributes (user/kernel/supervisor read/write/execute etc.)?

@ruuddw No, this is not related to MPU. MPU is only one way to use this attribute but the memory attributes can be used as one pleases. Please, refer to the PR description in #61009. The zephyr,memory-attr property is a generic way to define the memory capabilities, and MPU is only one use-case for that but you can have many more.

[microbuilder]

@carlocaione I might miss something, but what if someone wants to define an arch-specific mem attr (e.g. inner/outer sharable attr for Arm64), we still have a chance to customize and define the arch-specific mem attr right?

Yes, basically part of the mask of zephyr,memory-attr is reserved for architectures (8-bits):

https://github.com/zephyrproject-rtos/zephyr/blob/5664796d1f5b525dc27bb6dfc862c35e7c5b7b13/include/zephyr/dt-bindings/memory-attr/memory-attr.h#L23-L32

So every architecture must have its own /include/zephyr/dt-bindings/memory-attr/memory-attr-ARCH.h file with the set of defines it cares about. For example for ARM64:

#include <zephyr/dt-bindings/memory-attr/memory-attr.h>
#include <zephyr/dt-bindings/memory-attr/memory-attr-arm64.h>

   mem: memory@10000000 {
       compatible = "mmio-sram";
       reg = <0x10000000 0x1000>;
       zephyr,memory-attr = <( DT_MEM_CACHEABLE | DT_MEM_ARM64_INNER_SHAREABLE )>;
   };

You have the same concept of shareable in ArmV{7|8}-M for Normal memory, which can be one of:

• Shareable: Shareable between multiple cores or bus masters
• Non-Shareable: The memory region is not shareable between multiple cores or bus masters.
• Outer Shareable: The memory region is shareable within both the inner and outer shareability domain.
• Inner Shareable: The memory region is only shareable within the inner shareability domain

If it exists in both Aarch64 and Aarch32, is it 'arch' specific at that point, or should it be represented generically? Any other arches have a comparable concept?

There is also a concept of Bufferable with Device memory (memory mapped peripherals, etc.) or Normal memory -- when set, write operation might not immediately affect the memory, allowing for more efficient burst writes if there is a wait state with the memory system.

[carlocaione]

You have the same concept of shareable in ArmV{7|8}-M for Normal memory, which can be one of:

At this point I really think we should limit the attributes to the ones we actually use. Do these have any real use-case in the Zephyr context?

There is also a concept of Bufferable with Device memory (memory mapped peripherals, etc.) or Normal memory -- when set, write operation might not immediately affect the memory, allowing for more efficient burst writes if there is a wait state with the memory system.

The zephyr,memory-attr property is supposed to be companion only to the mmio-sram compatible than is Normal memory. The Device (MMIO) memory is the one associated to devices and it is outside the scope of this PR. We are only dealing with Normal memory at this stage.