Conversion from pointer to integer

[danakj]

Right now you have to reinterpret_cast to a primitive value then construct a usize. Yuck.

Instead template <class T> usize::from(T*)? But usize has size equal to size_t. Should we now introduce uptr before things get out of hand with usize holding pointers?

I think it's clear from reading the Rusty stuff below there's regret they didn't do this sooner, they would use uptr as a name, and they would have it be the size of a pointer, not of ptraddr_t which is too theoretical to need to support in the stdlib.

TODO:

• [x] Update docs on isize/usize to say they are address-sized instead of pointer-sized. This is a no-brainer for now.
• [x] Add a uptr type. (No need for iptr.) https://github.com/chromium/subspace/pull/284
  • [x] Differentiate it from u* as laid out below.

Rust project thoughts

Here are Rusty thoughts on it: https://github.com/rust-lang/rust/issues/95228 Highlights:

• A secondary goal of this project is to try to disambiguate the many meanings of ptr as usize, in the hopes that it might make it plausible/tolerable to allow usize to be redefined to be an address-sized integer instead of a pointer-sized integer.
• This would... effectively redefine usize from intptr_t to size_t/ptrdiff_t/ptraddr_t (it would still generally conflate those concepts, absent a motivation to do otherwise).
• size_t != uintptr_t != ptraddr_t Zulip thread
  • On CHERI, these are both true: sizeof(void *) == 16 UINTPTR_MAX == UINT64_MAX

Rust stdlib docs on it: https://doc.rust-lang.org/std/ptr/#pointer-vs-addresses

Divergence from u* integer types.

MAX

Of interest here is that if uintptr_t holds 128 bits but UINTPTR_MAX reports (2^64-1), we will need to either make uptr::MAX not self-consistent (ouch) or be very clear in the docs that uptr::MAX is not the same as UINTPTR_MAX, or just not have uptr::MAX at all. If uptr::MAX is self-consistent and returns (2^128-1), any rewriting of UINTPTR_MAX into a Subspace constant should be rewritten to usize::MAX, which will convert up to the larger uptr::MAX if needed, unless the authors were sure they wanted to include the non-address part of the pointer in their max. Having to go through usize seems like the wrong path though, there should be a more clear well-lit Good Path.

Another possibility here would be to have two max values on uptr instead, like

• uptr::MAX_ADDR // 2^64-1 on CHERI
• uptr::MAX_BIT_PATTERN // 2^128-1 on CHERI This would break the use of uptr in ducktyped generic code that rely on T::MAX, though perhaps in a good way. If MAX == MAX_ADDR it would be incorrect to use in generic code that wants to do bit masking with it as it would miss half the bits. If MAX == MAX_BIT_PATTERN it would be incorrect to use in generic code that wants to write MAX into the type, as it would be writing non-zeros into the capabilities and then produce Bad Things in CHERI.

FWIW usize::MAX is not really a valid value for usize for the things it is intended for which is offsets into arrays, as the max size of any allocation/array is isize::MAX or a 31-bit number. But it can also just be used as "an integer" so MAX is not (2^31-1). That said uptr::MAX being uptr::MAX_BIT_PATTERN feels much worse as it will directly cause bad CHERI things to occur when converted back to a pointer, whereas a x_usize >= 2^31 can be caught by bounds checks (or compiler checks for Array sizes) if misused. So I don't see this as strong argument for having uptr::MAX == MAX_BIT_PATTERN.

Proposal

Let's start with MAX_ADDR and MAX_BIT_PATTERN and drop MAX for now (and also have MAX_ADDR_PRIMITIVE and MAX_BIT_PATTERN_PRIMITIVE).

Conversions

• [x] uptr should be implicitly constructible from any pointer, unlike other u*.
• [x] uptr should not be implicitly constructible from smaller integer types, unlike other u*.
• [x] uptr should not be explicitly convertible to primitive integer types except uintptr_t. Up for consideration if this should be through a method call to help avoid code compiling and relying on it converting to other types due to type aliases and then breaking under CHERI, but a method call would give an explicit name/intend to callers but would be less type-safe than an operator std::same_as<uintptr_t>().
• [x] To convert to a usize, it should provide .addr() (like Rust's proposal) which returns usize.
• [x] uptr can be constructible from a usize but explicitly through means that specify the upper bits.
  • [x] uptr.with_addr(usize) (like the the Rust proposal) can copy the high bits from *this to the new pointer. e.g. uptr(actual_pointer).with_addr(a_different_address).
  • [x] I considered a static uptr::from_ptr_and_addr(T*, usize) but it's basically a syntactic sugar for with_addr. It's not clear if it would be painful to construct a uptr and call .with_addr such that it's worth adding this shorter path.
• [x] uptr should not be From<u*> for explicit conversions, as it should instead be constructed through with_addr() through another pointer.
• [x] uptr should also not be Into<u*> (or IOW u* should not be From<uptr>). If size_of<uptr>() > size_of<usize>() then basically every conversion from uptr to usize will be out of bounds due to the capabilities in the high bits, and thus .addr() would need to be used to not panic.

[danakj]

Done by https://github.com/chromium/subspace/pull/284

[danakj]

RangeFrom needs to return a lower bound for its size_hint which is normally T::MAX. But if you constructed a RangeFrom over uptr then that doesn't work cuz there's no MAX.

MAX_ADDR would do the wrong thing, cuz that just excludes capabilities from its value, but pointer values may be larger with the capabilities present on CHERI.

MAX_BIT_PATTERN is the closest thing to correct here. If you iterate so far that you are incrementing past MAX_ADDR in the address part of the integer then you've done bad things to your pointer, but that's where the iteration would technically go.

Here MAX being 2^N-1 (not matching uintptr_t on cheri) would be the most correct thing to use, and it's annoying in this case that it does not exist. There's nothing... better.

Edit: Well it turns out uptr is weird in lots of ways so it doesn't have like try_from(unsigned) cuz it needs to be constucted with provenance, so this use case evaporated.

Maybe uptr shouldn't be included in Integer concept though..