dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.26k stars 4.73k forks source link

Unexpected padding with Auto layout structs #109585

Open cap5lut opened 4 hours ago

cap5lut commented 4 hours ago

Description

Looking at the value tuple type (bool, Nullable<int>) i expect the following layout:

1 byte for the bool + 3 padding bytes 1 byte for the bool of the nullable + 3 padding bytes 4 bytes for the int

which would sum up to 12 bytes, this however turns into a 16 bytes struct, because 4 bytes of padding are added at the end.

Reproduction Steps

(bool, Nullable<int>) x = (false, 5);
Inspect.Stack(x);

unsafe
{
    Console.WriteLine(sizeof((bool, Nullable<int>)));
}

I am using sharplab.io to showcase it: sharplab.io link

Expected behavior

The struct size should be 12 bytes, but has additionally some trailing 4 byte padding.

Actual behavior

The struct size is 16 bytes because of the trailing 4 byte padding.

cap5lut commented 3 hours ago

I tried to mimic the value tuples and how LayoutKind.Auto and LayoutKind.Sequential affect the size, using the following structs:

[System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Sequential)]
struct S1 {  bool Bool; Nullable<int> Num; }

[System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Auto)]
struct S2 { bool Bool; Nullable<int> Num; }

[System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Sequential)]
struct S3 { bool Bool; S34 Num; }

[System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Auto)]
struct S4 { bool Bool; S34 Num; }

struct S34 { bool Bool; int Num; 

the sequential ones are all 12 bytes, while the auto ones are 16 bytes.

just-ero commented 3 hours ago

For completion, I ran Console.WriteLine(sizeof((bool, int?))); on the following configurations with these results:

.NET 8.0 win-x86: 12
.NET 8.0 win-x64: 16
.NET 9.0 win-x86: 12
.NET 9.0 win-x64: 16

.NET 8.0 win-x86 NativeAOT: 12
.NET 9.0 win-x86 NativeAOT: 12
.NET 9.0 win-x64 NativeAOT: 16
tannergooding commented 3 hours ago

CC. @jkotas

We've fixed a few issues in the past, related to generics and nullable.

This one looks to be a general issue with AutoLayout and it rounding up to a multiple of 8. Even simply struct S { int a, b, c; } causes this to reproduce (or struct S { int a, b, c, d, e; } which makes it 24 instead of 20).

timcassell commented 3 hours ago

This looks completely expected to me. Types are padded up to the nearest word size. x86 is 32-bit platform, with 4-byte word size. 12 is a multiple of 4. x64 is a 64-bit platform with 8-byte word size. 12 is not a multiple of 8, so it rounds up to 16.

32-bit calculation: Nullable<int> = 4 (int) + 1 (bool) + 3 (padding) = 8 (bool, Nullable<int>) = 1 (bool) + 8 (Nullable<int>) + 3 (padding) = 12

64-bit calculation: Nullable<int> = 4 (int) + 1 (bool) + 3 (padding) = 8 (bool, Nullable<int>) = 1 (bool) + 8 (Nullable<int>) + 7 (padding) = 16

[Edit] I'm actually surprised that the sequential layouts are 12 bytes on 64-bit platform.

just-ero commented 3 hours ago

As I see it, Nullable<int> would still have an alignment of 4 while having a size of 8.
That would (should) result in 1 (bool) + 3 (padding) + 4 (int) + 1 (bool) + 3 (padding) = 12.

timcassell commented 3 hours ago

As I see it, Nullable<int> would still have an alignment of 4 while having a size of 8. That would (should) result in 1 (bool) + 3 (padding) + 4 (int) + 1 (bool) + 3 (padding) = 12.

That makes sense. Maybe the auto layout isn't taking into account the alignment of nested types.

tannergooding commented 2 hours ago

This looks completely expected to me. Types are padded up to the nearest word size. x86 is 32-bit platform, with 4-byte word size. 12 is a multiple of 4. x64 is a 64-bit platform with 8-byte word size. 12 is not a multiple of 8, so it rounds up to 16.

This isn't correct. Types are padded based on the largest natural alignment of all members.

The natural alignments are: Type Size Natural Alignment Additional Notes
bool 1 1
byte 1 1
char 2 2
double 8 8 Some 32-bit platforms use a natural alignment of 4
short 2 2
int 4 4
long 8 8 Some 32-bit platforms use a natural alignment of 4
nint 4 or 8 4 or 8 Same size as a pointer
sbyte 1 1
float 4 4
ushort 2 2
uint 4 4
ulong 8 8 Some 32-bit platforms use a natural alignment of 4
nuint 4 or 8 4 or 8 Same size as a pointer

The designated packing can be overridden to be smaller, but not larger, than the natural packing.

GC tracked types are pointer sized but cannot have the natural packing overridden.

Fields that are structs (not primitives) have their own natural alignment based on the members they contain. So struct S { bool b, int i; } has a natural packing of 4 and size of 8. This is impactful when it is a field of another struct, because it means that struct S2 { bool b; S s; } and struct S3 { bool b1; bool b2;, int i; } have different natural layouts and sizes. -- In particular S2 is going to be bool, padding x3, bool, int and thus have its own natural packing of 4 and size of 12. While S3 is also going to have a natural packing of 4 but be size of 8 because it is bool, bool, padding x2, int.

timcassell commented 2 hours ago

GC tracked types are pointer sized but cannot have the natural packing overridden.

Is this possibly related to https://stackoverflow.com/questions/67068942/c-sharp-why-do-class-fields-of-struct-types-take-up-more-space-than-the-size-of and https://stackoverflow.com/questions/24742325/why-does-struct-alignment-depend-on-whether-a-field-type-is-primitive-or-user-de?

Structs that contain reference types, or used as fields in reference types (classes), are padded more than expected. (I'm not sure if there is already a separate issue for those.)

tannergooding commented 2 hours ago

The bug you linked is potentially related to the overall issue here, but it wouldn't be related to the fact that you cannot override the natural packing/alignment of GC types.

Auto layout is meant to pick an "optimal" layout and while there are a few potential definitions for that, the default assumption is simply that it minimizes padding.

Typically to achieve this it means you sort fields based on natural packing. So you'd always have all fields that are 8-byte packed, then 4-byte packed, then 2-byte, then 1-byte.

You can optionally also do some sorting within those regions, such as grouping all GC fields together (typically first), or by also sorting by size within a group (so a struct that is 4-byte packed but has a size of 16 would come before a struct that is 4-byte packed and has a size of 8, which would come before just a regular int field), but this additional level shouldn't impact the computed size, just the order fields exist. -- This is called out because its typical in C or C++ when optimizing layouts to group same sized fields together, which can enable SIMD or other optimizations to kick in where relevant.


Notably the .NET auto layout algorithm has historically had some quirks and bugs; some of which we've fixed over time and others of which we've not. I'm not sure where this one in particular falls.