Documentation of TLS data structures

[PkmX]

We've been implementing TLS support in lld: https://reviews.llvm.org/D39324, but the documentation regarding the layout of TLS structures is missing.

If I'm understanding correctly from reading the various TLS headers in glibc: tp is set up to point to the end of the TCB and the beginning of the static TLS block; this means that it is neither variant I nor II as described in ELF Handling for Thread-Local Storage. The front of the TCB contains a pointer to the DTV, and each pointer in DTV points to 0x800 past the start of a TLS block to make full use of the range of load/store instructions.

Is it safe to assume that this will be the case across all platforms? These values are hardcoded in bfd as well. What is the motivation behind making tp to point to the end of TCB as opposed to following the two variants defined in the ELF TLS paper?

[aswaterman]

This is not a brand-new design; it is basically the same design as MIPS, PowerPC, Tilera, and Nios II.

The 0x800 offset is to maximize the displacement from the tp register, since the ISA can address within +0x7ff and -0x800 of a register.

[aswaterman]

We intend to keep this standard for all platforms.

[PkmX]

Thanks for the clarification, so I guess this should go into the ABI documentation as well?

[aswaterman]

Yeah, it should go into the psABI doc. If you have the time to write up a short description, please do; otherwise, let's keep this issue open until someone finds the time.

[MaskRay]

I noticed that TLS_TP_OFFSET (used by initial-exec and local-exec) is not defined as a non 0 in glibc while reading a musl riscv patch.

/* The thread pointer points to the first static TLS block.  */
#define TLS_TP_OFFSET       0

/* Dynamic thread vector pointers point 0x800 past the start of each
   TLS block.  */
#define TLS_DTV_OFFSET      0x800

On targets that define TLS_DTV_OFFSET (m68k, powerpc{32,64} and mips), TLS_TP_OFFSET is also defined.

/* The thread pointer points 0x7000 past the first static TLS block.  */
#define TLS_TP_OFFSET       0x7000

/* Dynamic thread vector pointers point 0x8000 past the start of each
   TLS block.  */
#define TLS_DTV_OFFSET      0x8000

Take powerpc as an example: a@tprel = st_value(a) - 0x7000 A load/write insn has an offset ranging from [-0x8000, 0x8000). This carefully picked TP offset allows it to load/write a TLS variable whose st_value ranges from [0,0x7000+0x8000).

Any reason riscv doesn't define TLS_TP_OFFSET?

[jim-wilson]

This support was written years ago at UC Berkeley. It may not be possible to determine why it was written this way.

The rv64 glibc ABI was frozen when we upstreamed it, so if this is an ABI change it is too late to change it. Technically the rv32 glibc ABI is not frozen yet, but having the rv32 ABI handle this differently from the rv64 ABI would be confusing and probably not worth the trouble.

It is odd that these numbers are 0 and 0x800. Maybe there was a mistake made somewhere along the way, or maybe there was a bug that they had to workaround and then forgot to change the numbers back. Probably no way to know for sure.

[richfelker]

The offset of 0 may be preferable for making optimal use of the compressed ISA. With the offset of 0x800 you pretty much always need the 32-bit instruction forms for real-world code. In any case, this is ABI and can't be changed, and the motivations for using nonzero offsets is dubious at best.

[jim-wilson]

The compressed ISA is probably the reason now that you point it out. That would explain the 0x800 offset. Agreed that we aren't changing the ABI.