Proposal: Extend the tuple semantics

[ikskuh]

As discussed on the Discord yesterday:

The Problem

Right now, the definition of packed struct doesn't allow to include arrays. This is due to slicing an array should be a supported operation, but a bit-packed array will not have the properties of being sliceable, as it requires pointers that include bit offset properties.

Proposed solution

This proposal extends the semantics of packed struct as described in #10113 by adding a tiny feature to #4335:

• Allow tuples to be bit-packed (packed struct {u2,u2,u2,u2}) or extern(extern struct {u8, u16}).
• Allow bit-packed tuples in packed structs.

Evaluation

Right now, bit-packed tuples are a feature of stage1, whereas stage2 will produce a compile error:

test {
    // const T = packed struct { u2, u2, u2, u2 }
    const T = @Type(.{
        .Struct = .{
            .layout = .Packed,
            .fields = &.{
                .{ .name = "0", .field_type = u2, .default_value = null, .is_comptime = false, .alignment = 0 },
                .{ .name = "1", .field_type = u2, .default_value = null, .is_comptime = false, .alignment = 0 },
                .{ .name = "2", .field_type = u2, .default_value = null, .is_comptime = false, .alignment = 0 },
                .{ .name = "3", .field_type = u2, .default_value = null, .is_comptime = false, .alignment = 0 },
            },
            .decls = &.{},
            .is_tuple = true,
        },
    });

    @compileLog(T, @sizeOf(T), @alignOf(T), @bitSizeOf(T));
    @compileLog(@offsetOf(T, "0"), @bitOffsetOf(T, "0"));
    @compileLog(@offsetOf(T, "1"), @bitOffsetOf(T, "1"));
    @compileLog(@offsetOf(T, "2"), @bitOffsetOf(T, "2"));
    @compileLog(@offsetOf(T, "3"), @bitOffsetOf(T, "3"));

    const J = packed struct {
        a: u3,
        b: T,
        c: u5,
    };

    @compileLog(J, @sizeOf(J), @alignOf(J), @bitSizeOf(J));
    @compileLog(@offsetOf(J, "a"), @bitOffsetOf(J, "a"));
    @compileLog(@offsetOf(J, "b"), @bitOffsetOf(J, "b"));
    @compileLog(@offsetOf(J, "c"), @bitOffsetOf(J, "c"));
}

Compiling this code with stage1 yields the following:

| struct:4:22, 1, 1, 8
| 0, 0
| 0, 2
| 0, 4
| 0, 6
| J, 2, 1, 16
| 0, 0
| 0, 3
| 1, 11

This looks about right and follows the semantics of this propsal, but probably more by accident and less by semantic decisions.

But using stage2 will give us this error:

tup.zig:26:9: error: packed structs cannot contain fields of type 'tuple{u2, u2, u2, u2}'
        b: T,
        ^~~~
tup.zig:26:9: note: only packed structs layout are allowed in packed types

Compile Log Output:
@as(type, tuple{u2, u2, u2, u2}), @as(comptime_int, 4), @as(comptime_int, 1), @as(comptime_int, 8)
@as(comptime_int, 0), @as(comptime_int, 0)
@as(comptime_int, 1), @as(comptime_int, 8)
@as(comptime_int, 2), @as(comptime_int, 16)
@as(comptime_int, 3), @as(comptime_int, 24)

Which looks like the tuple type is properly recognized as a tuple, but the layout is completly ignored and will be set to .Auto.

Allowing a tuple to be put into a packed struct as long as the tuple would have a packed layout would make the language more generalized and allows for some nice uses.

Use Cases

As with other tuples, a packed tuple can be indexed with comptime known indices:

// using the same types as above:
test {
    var t: T = .{ 0, 1, 2, 3 };

    var j: J = .{
        .a = 3,
        .b = t,
        .c = 11,
    };

    j.b[1] = 1;
    j.b[3] = 2;

    std.debug.print("a={} b={{ {}, {}, {}, {} }}, c={}\n", .{
      j.a,
      j.b[0],
      j.b[1],
      j.b[2],
      j.b[3],
      j.c,
    });
}

This allows us to have quasi-arrays in a packed struct by grouping fields into indexed field structs instead of structs with named fields.

Other implications are that users of Zig can now create smaller tuples when it seems appropiate. One example here would be a packed struct { u31, bool } which can be passed around in a register instead of using either memory or two registers, saving us 32 bit of padding per element.

Pros

• Makes tuples and structs more alike, as a tuple struct is just a struct with a flag and decimal field names. It should be possible to allow packed or extern layout.
• Allows quasi-arrays in packed structs

Cons

I can't think of any negative downsides of this proposal, as it only slightly increases the syntactical surface, but at the same time removes forbidden properties, so actually less stuff to remember.

[sharanya-mitra]

nice proposal

[ominitay]

the layout is completly ignored and will be set to .Auto

If you look at the source code here: https://github.com/ziglang/zig/blob/4c750016eb9b1c0831cbb0398a4d6ee9dbdc932e/src/type.zig#L6273-L6282 The representation of the tuple type in AIR doesn't have very much to it. This definitely should be redone to be closer to how structs are represented. Related: #4335

I don't believe that this actually extends the semantics, as I think packed and extern tuples were already accepted as part of the proposal.

[InKryption]

Relevant inquiry: we can coerce anonymous tuples to arrays (so long as all elements can coerce to the array's element type), could we apply the same standard to tuples, and by extension, packed tuples? If yes, then this opens the door to doing runtime indexing on packed values, e.g.

const Tup4x2 = packed tuple{ u2, u2, u2, u2 };
var tup_4x2 = Tup4x2{ 1, 1, 1, 2 };
{
    var arr: [4]u2 = tup_4x2;
    for (arr) |*el| {
        el.* += 1;
        // more code that would get "copied" 4 times in an inline for loop.
        // ...
    }
    inline for (arr) |new, i| tup_4x2[i] = new;
}

Mostly thinking aloud, would like to know what the official stance on this is.

[ominitay]

I don't believe that this actually extends the semantics, as I think packed and extern tuples were already accepted as part of the proposal.

Should probably clarify that I think this should still stay open to ensure this is understood by whoever implements said proposal.

[ominitay]

@InKryption This wouldn't be runtime indexing on packed values, as it would require that the contents of the packed tuple are 'un-packed', so-to-speak. This would mean that Zig would have to copy of each element of the packed tuple individually into an array.

It would be nice to have some sort of proposal for runtime tuple indexing though.

[InKryption]

Yeah, I have no allusions as to the presence of copy/shift elisions in my example, just asking whether such a thing would be permitted, and offering a use case in favor. It would be of purely ergonomic benefit.

[praschke]

It would be nice to have some sort of proposal for runtime tuple indexing though.

is it even possible to runtime index tuples? the reason it's possible with arrays is the fixed element type and width, isn't it?

[ominitay]

is it even possible to runtime index tuples? the reason it's possible with arrays is the fixed element type and width, isn't it?

Where there is a single type, runtime indexing should be absolutely possible (with shifts and masks being calculated on-the-fly). With multiple types, there would need to be dynamic dispatch involved. Some crude version of this can already be achieved using comptime logic (inline switch cases come to mind), but it's not as good as having fully-supported runtime tuple indexing.

[Vexu]

Is there anything to this proposal that isn't covered by #4335?

[ominitay]

This looks like it's been fully-implemented by #13627, I think? So long as these can be used from packed structs, since I can't see any test case which covers that. Although I'd be very surprised if they didn't.

[mlugg]

packed and extern tuples have since been disallowed for language simplicity. This proposal was never explicitly rejected, so re-opening for now.

[ikskuh]

Here's a good case for packed tuples:

From LPC 1768 User Manual, PDF Page 110:

This register could be modeled as such:

const PINSEL4 = packed struct (u32) {
    // contains pins P2.0 … P2.13
    sel: packed struct(u28) { u2, u2, u2, u2, u2, u2, u2, u2, u2, u2, u2, u2, u2, u2 },
    _reserved: u4,
};

Another reason for this change is that we can emulate in userland easily, but it's clunky and has bad UX:

fn BitFieldMixin(comptime Bits: type) type {
    return struct {
        pub fn set(bits: *@This(), comptime index: comptime_int, value: u1) void {
            @field(bits, std.fmt.comptimePrint("b{}", .{index})) = value;
        }
        pub fn get(bits: Bits, comptime index: comptime_int) u1 {
            return @field(bits, std.fmt.comptimePrint("b{}", .{index}));
        }
    };
}

const Bits8 = packed struct {
    usingnamespace @BitFieldMixing(@This());
    b0: u1, b1: u1, b2: u1, b3: u1, b4: u1, b5: u1, b6: u1, b7: u1
};
const Bits9 = packed struct {
    usingnamespace @BitFieldMixing(@This());
    b0: u1, b1: u1, b2: u1, b3: u1, b4: u1, b5: u1, b6: u1, b7: u1, b8: u1
};

You can also do even more metaprogramming and just use struct field names @"0" like in tuples

[nektro]

packed tuples seem ripe for rejection under the same premises as arrays being blocked. the above example can be achieved with the following

const PINSEL4 = packed struct(u32) {
    p2_0: u2,
    p2_1: u2,
    p2_2: u2,
    p2_3: u2,
    p2_4: u2,
    p2_5: u2,
    p2_6: u2,
    p2_7: u2,
    p2_8: u2,
    p2_9: u2,
    p2_10: u2,
    p2_11: u2,
    p2_12: u2,
    p2_13: u2,
    p2_14: u2,
    p2_15: u2,
};

[mlugg]

packed tuples seem ripe for rejection under the same premises as arrays being blocked.

Please can you elaborate on this point? Arrays are not permitted in packed datastructures because they cannot meaningfully be sliced, nor can runtime-known element pointers be acquired, which are supposed to be supported operations. This style of argument is clearly not relevant to packed tuples, since they do not need to support these operations in the first place.

the above example can be achieved with the following

Yes, and all tuples can be replaced with structs with explicit named fields, but that generally, well, isn't very nice, hence this proposal.