Curiosity about complex structures

[ethindp]

So I was thinking of modifying a driver for NVMe hardware to use this crate (because I have to duplicate a lot of code in the driver as-is and that's just not practical, and I'm not sure how to do it in a generic). The layout of the structure has a couple things that I'm not sure how to repesent using this crate though:

1. It contains strings (in particular, the serial number, model number and firmware revision), all longer than 128 bits. What is the best way of representing this? Are arrays possible? (I'd think so but want to check.)
2. It contains bytes that are supposed to be read in big-endian. An example of this is the IEEE OUI identifier, which are bytes 75:73. Does this crate support mixed-endian structs?
3. It contains bytes with reserved bits. For example, byte 76, Controller Multi-Path I/O and Namespace Sharing Capabilities (CMIC), is 8 bits, but bits 07:04 are reserved, and only bits 03:00 are used. Similarly, bytes 95:92, Optional Asynchronous Events Supported (OAES), contains reserved bits all over the place, e.g.: bits 31:15, 10, and 07:00. How should I best represent these kinds of bytes?
4. In memory, the structure is packed, as though #[repr(C, packed)] was used. This is actually how I have to define them in my code at the moment because I can't read them in any other way unless I read them manually (oh god). Does the from_bytes()/to_bytes() structure count padding, and include that in consumption and generation of the byte stream, or does it only include the actual member fields?

For reference, the particular structures I'm talking about is over here, in figs. 249, 251, etc.: https://nvmexpress.org/wp-content/uploads/NVM-Express-1_4b-2020.09.21-Ratified.pdf

I would've written it here, but I didn't think that would be a good idea, seeing as its 4,096 bytes and I could just link to the information.

[Robbepop]

If the overall struct is 4096 bytes I think it makes sense to better split it up into different components. Not the entire thing must be wrapped in a #[modular_bitfield] proc. macro. I won't dig into the linked PDF and search for all the information required but generally you only need bitfields in cases where bits are stored in chunks that do not align with 8 bits. Storing an array probably won't need a modular bitfield - and if so I'd recommend you to rethink the data structure. It would only need a bitfield if the array is not aligned to 8 bits.

If you want you can paste the structure here with information as in:

Bits	Field	Type
0..5	some_flag	i8
5..6	is_alive	bool
etc.	etc.	etc.

And we sure will find some proper solution.

[ethindp]

I'm not sure why you can't look up the figures I indicated. The primary figure I'm speaking about is figure 251, in section 5.15.2.2. The one with reserved bits is also within that figure, bytes 95:92. That has the following layout:

Bits	Description	Type
31:15	Reserved	u16
14	Endurance Group Event Aggregate Log Page Change Notices event supported	bool
13	LBA Status Information Notices event supported	bool
12	Predictable Latency Event Aggregate Log Change Notices event supported	bool
11	Sending Asymmetric Namespace Access Change Notices supported	bool
10	Reserved	bool
9	Firmware activation notices event supported	bool
8	Namespace Attribute Notices event and Changed Namespace List log page supported	bool
7:0	Reserved	[bool; 8]

I could probably just make a recursive bitfield out of this, come to think of it.

[Robbepop]

Hi, sorry I completely forgot about this thread. From what it looks you can easily represent all the sub parts of the entire structure as you have just laid out 32 bits of one. The reserved bits in 7:0 should probably be represented as a single u8 instead of a [bool; 8] field. However, your suggestion to maybe add [bool; 8] is legit and might even be possible, however only with some conversion overhead I suppose since bool is not really a 1 bit width type.

So to answer your original question: Yes this crate should be sufficient for your needs from what it looks like on the surface. Of course trust yourself always the most. Use u8 or other small types such as BN instead of arrays for reserved holes.

[ethindp]

Thank you. Is the crate able to handle array types? If not that would be an appreciated addition. I'm thinking of using this on really large structs that have large blocks of reserved or undefined data, and right now I have to use crazy slice::align_to() magic to actually do it, as well as representing the struct using #[repr(C, packed)]. I'd love to add this myself but I honestly have no idea how; I've never worked with rust attribute macros before so I have no idea how they work.

On 2/28/21, Hero Bird notifications@github.com wrote:

Hi, sorry I completely forgot about this thread. From what it looks you can easily represent all the sub parts of the entire structure as you have just laid out 32 bits of one. The reserved bits in 7:0 should probably be represented as a single u8 instead of a [bool; 8] field. However, your suggestion to maybe add [bool; 8] is legit and might even be possible, however only with some conversion overhead I suppose since bool is not really a 1 bit width type.

So to answer your original question: Yes this crate should be sufficient for your needs from what it looks like on the surface. Of course trust yourself always the most. Use u8 or other small types instead of arrays for reserved holes.

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/Robbepop/modular-bitfield/issues/61#issuecomment-787532126

-- Signed, Ethin D. Probst

[Robbepop]

I think it is a better decision to use this crate's modular bitfields only for well defined subsets of your structs with giant reserved voids. So just have your entire struct and within the struct have reserved fields that simply use some arrays or whatever you used before and for the well defined subsets use this crate's modular bitfield structs as fields. This should work very efficiently as long as your reserved data is 8 bit aligned. (I hope it is.)

[ethindp]

I could do that, but how would I then load it into memory? I'm only given a stream of bytes, and there's a lot of bytes to load. I don't mind doing it manually, it'll just be a pain and I'd (preferably) like to make it as easy to read and as simple as possible. Like I don't want to be calling ::from_*_bytes() all over the place and that kind of thing.

On 2/28/21, Hero Bird notifications@github.com wrote:

I think it is a better decision to use this crate's modular bitfields only for well defined subsets of your structs with giant reserved voids. So just have your entire struct and within the struct have reserved fields that simply use some arrays or whatever you used before and for the well defined subsets use this crate's modular bitfield structs as fields.

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/Robbepop/modular-bitfield/issues/61#issuecomment-787546903

-- Signed, Ethin D. Probst