[intro.object] Converting a pointer value that is returned from a malloc operation that creates an array object to the element type results in UB?

[xmh0511]

Full name of submitter (unless configured in github; will be published with the issue): Jim X

The original question is on SO

[intro.object] p11 says:

Further, after implicitly creating objects within a specified region of storage, some operations are described as producing a pointer to a suitable created object. These operations select one of the implicitly-created objects whose address is the address of the start of the region of storage, and produce a pointer value that points to that object, if that value would result in the program having defined behavior.

Consider this example:

struct T{
    int a;
};
int main(){
  auto ptr = (T*)malloc(sizeof(T)*10); // #1
  ptr++;  // #2 UB?
}

Since T is an implicit-lifetime class, the operation malloc can implicitly create an object of class type T. In this example, we intend to make ptr point to the initial element of the array object with 10 elements of type T, based on this assumption, ptr++ would have a valid behavior. This case assumes the first element has its address at the start of the region.

Since an array of any type is an implicit-lifetime type, malloc(sizeof(T)*10) can implicitly create an array object with 10 elements and start the lifetime of that array object. Since the initial element and the containing array are not pointer-interconvertible.

That means, ptr either point to the first element of that array or that complete array. However, if we consider the first element locates at the start of the region, how do we prove whether this assumption is true or not? After all, there is no rule to say that array and its first element must have the same address.

Furthermore, consider this example:

struct S{
   S(){}
   ~S(){}
};
int main(){
  auto ptr = (S*)malloc(sizeof(S)*10); // #1
  ptr++; // #2
}

Now, S is not an implicit-lifetime class, must #1 and #2 be replaces to

auto arrPtr = (S(*)[10])malloc(sizeof(S)*10);  // point to the array
auto ptr =*arrPtr; // use array-to-pointer conversion 

to make ptr well-defined to point to the first element?

[frederick-vs-ja]

After all, there is no rule to say that array and its first element must have the same address.

This is a known issue, although CWG issue is missing. See cplusplus/draft#3203 (and perhaps cplusplus/draft#4808).

Now, S is not an implicit-lifetime class, must #1 and #2 be replaces to

auto arrPtr = (S(*)[10])malloc(sizeof(S)*10);  // point to the array
auto ptr =*arrPtr; // use array-to-pointer conversion 

to make ptr well-defined to point to the first element?

std::start_lifetime_as_array provides a more flexible method... But I think it may be preferred to have a some well-defined approach without involving C++23 stuffs.

Approaches IMO:

1. Making the return value of operator new and malloc etc. also possibly point to the first element of a suitable created array object, assuming the issue on address sameness is resolved.
2. And/or, making the return value also possibly point to some other (non-living) complete objects or pointer-interconvertible subobjects of non-implicit-lifetime types, given size and alignment are suitable.

[xmh0511]

Making the return value of operator new and malloc etc. also possibly point to the first element of a suitable created array object, assuming the issue on address sameness is resolved.

If the first element is not of an implicit-lifetime type, it cannot because these operations select one of the implicitly-created objects.

making the return value also possibly point to some other (non-living) complete objects or pointer-interconvertible subobjects of non-implicit-lifetime types, given size and alignment are suitable.

This means, for the second example, we can only get the pointer value pointing to the array object and use array-to-pointer conversion, which is the only way to be right.

[jensmaurer]

If T is of implicit-lifetime type, so is "array of T".

What we need for the T example to be valid: Implicitly create an object of type "array of 10 T" in the storage and return a pointer to the first element.

It seems the only thing we need to say to make the quoted wording do the right thing is to say that a subobject (that is of implicit-lifetime type) of an implicitly created object is also an implicitly created object. Then, the quoted wording can return a pointer to the first element (instead of a pointer to the entire array).

[jensmaurer]

Unless we already have wording that says that a pointer to the array also points to the first element (but not vice versa).

[jensmaurer]

The S example can't be made to work without placement-new or std::start_lifetime_as.

[frederick-vs-ja]

The example in https://github.com/cplusplus/draft/issues/5782#issuecomment-1225697632 seems to be related.

[xmh0511]

If T is of implicit-lifetime type, so is "array of T".

What we need for the T example to be valid: Implicitly create an object of type "array of 10 T" in the storage and return a pointer to the first element.

It seems the only thing we need to say to make the quoted wording do the right thing is to say that a subobject (that is of implicit-lifetime type) of an implicitly created object is also an implicitly created object. Then, the quoted wording can return a pointer to the first element (instead of a pointer to the entire array).

However, we didn't say the array object and its first element have the same address, which means, if we consider the first element located at the start of the region, the array object may located at the outside of the region because no rule guarantee they are the same address.

[frederick-vs-ja]

the array object may located at the outside of the region

I don't think anything can permit an object to be located outside of an allocated storage. Is the currently wording happening not to forbid this?

[xmh0511]

the array object may located at the outside of the region

I don't think anything can permit an object to be located outside of an allocated storage. Is the currently wording happening not to forbid this?

Where does that wording in the current draft forbid the case?

[frederick-vs-ja]

the array object may located at the outside of the region

I don't think anything can permit an object to be located outside of an allocated storage. Is the currently wording happening not to forbid this?

Where does that wording in the current draft forbid the case?

[intro.object] p1 ~may be sufficient~:

An object occupies a region of storage in its period of construction ([class.cdtor]), throughout its lifetime, and in its period of destruction ([class.cdtor]).

It might be not. But... [intro.object] p10 says:

Some operations are described as implicitly creating objects within a specified region of storage.

which should cover the cases in this issue.

[xmh0511]

Some operations are described as implicitly creating objects within a specified region of storage.

It does not necessary to create complete objects, subobjects are also objects anyway. Moreover, it says

that operation implicitly creates and starts the lifetime of zero or more objects of implicit-lifetime types ([basic.types.general]) in its specified region of storage if doing so would result in the program having defined behavior.

No rule says the program will be undefined if the operation only creates the subobject in its region, it even didn't say the complete object of the subobject that occupies the region should also be within the region.

[jensmaurer]

@xmh0511 , as I said elsewhere, if it's sufficient for a subobject to be implicitly created in the region of storage, then there is no point in attempting to somehow create a (larger) complete object that starts before the storage. (You can just create the subobject as a complete object.)

I'd also like to point out that a subobject doesn't (conceptually) exist without its complete object, so the creation of a subobject implies that a complete object must have been created. I find the text "in its region of storage" sufficiently clear (together with "An object occupies a region of storage") to imply that a complete object can't be partially outside the given region of storage.

[dcl.array] p6 seems sufficiently clear ("consists of") that there is no padding in an array object outside of the elements, which implies that the address of the first element is the same as the address of the entire array.

[xmh0511]

there is no point in attempting to somehow create a (larger) complete object that starts before the storage.

I'd also like to point out that a subobject doesn't (conceptually) exist without its complete object, so the creation of a subobject implies that a complete object must have been created.

As I said above(or in another issue), the interpretation may feel absurd, however, we do lack the (sufficient)formal wording to make the interpretation irrefutable.

"An object occupies a region of storage"

It just means what it could literally mean. As per [basic.compound] p5, the storage occupied by a complete object is not necessarily reachable by its subobject, in other words, when we only have a pointer value points to a certain subobject, we may not reach the storage occupied its complete object, so, from the perspective of subobjects, it doesn't matter where their complete objects locates.

[jensmaurer]

Back to the original example.

"malloc" implicitly creates objects, in particular the "array of 10 T" and the T subobjects of that array. It then returns a pointer to the first element of the array (not to the entire array), which is one of the implicitly-created objects, which makes the rest of the program have defined behavior (in particular, pointer arithmetic is valid).

I'm not seeing a defect in the handling of your example.

[xmh0511]

I think it may exist two cases here:

1. implicitly creates the "array of 10 T" within the storage and hence its subobjects.
2. implicitly creates the "subobject" within its storage, the array object is not guaranteed since it may be outside of the storage.

For the second case, we may consider it in an extreme way, assuming the third element subobject locates at the start of the region created by malloc, which means the array object may or may not begin its lifetime since the malloc operation can only provide start the lifetime of these objects that are within its storage. No wording in the current standard forbid we think this way.

[jensmaurer]

No wording in the current standard forbid we think this way.

So, what's the actual problem with such thinking that would need fixing?

Obviously, the enclosing "array of 10 T" object hasn't started its lifetime in such a case, thus you can't do pointer arithmetic on the elements. If that's what you want, fine with me.

[xmh0511]

No wording in the current standard forbid we think this way.

So, what's the actual problem with such thinking that would need fixing?

Obviously, the enclosing "array of 10 T" object hasn't started its lifetime in such a case, thus you can't do pointer arithmetic on the elements. If that's what you want, fine with me.

So, are we allowed to do the pointer arithmetic to make the pointer points to the storage outside of the storage allocated by malloc since arithmetic is permitted when the array is non-living as discussed in https://github.com/cplusplus/CWG/issues/382#issuecomment-1657087613, consider the case in which the second element of the array locates at the start of the region of the storage?

[jensmaurer]

I continue to fail to see a problem, because my (alternative) interpretation is that an array of N-1 is implicitly created entirely within that storage, and pointer arithmetic is certainly valid there.

[xmh0511]

I continue to fail to see a problem, because my (alternative) interpretation is that an array of N-1 is implicitly created entirely within that storage, and pointer arithmetic is certainly valid there.

Whether an interpretation can be true is based on whether the program can result in well-defined behavior. In my interpretation, together with the clarification that pointer arithmetic can be done on a non-living array object, the program seems to be valid when P - 1 pointers to the outside of the storage allocated by malloc if P is the address of the start of the region allocated by malloc

T0	T1	T2	T3	...
P-1	P

As discussed above, no rule says that P-1 results in UB. In other words, your alternative interpretation is valid, and in your interpretation model P-1 would result in UB. However, my interpretation, as far as now, is also a valid interpretation, and P-1 is well-defined in my interpretation. This is similar to you can say there is an int object located at the start of the region and I say there is a double object located at the start of the region, such two interpretations are both valid as long as there exists no rule says the case will result in UB.

[frederick-vs-ja]

It may be better to say "a subobject is implicitly created only if its complete object is implicitly created". But I do think that this can be done without a CWG issue.

[languagelawyer]

It may be better to say "a subobject is implicitly created only if its complete object is implicitly created".

@frederick-vs-ja plz see e.g.

When the allocated object is not an array, the result of the new-expression is a pointer to the object created.

Do you see «the»? There is only one object «created» by a new-expression, even if it has subobjects. Subobjects are not considered «created» (except when https://timsong-cpp.github.io/cppwp/n4868/intro.object#2.sentence-4 applies).

If you think new-expression first creates the containing objects and then creates subobjects, or that it first creates subobjects and then glues them together into containing object, you have a very wrong mental model.

I think the same applies to other cases when objects are created, otherwise it would be a mess. Subobjects are not «created»!

[frederick-vs-ja]

Oh, subobjects are considered implicitly created in the comments in this example. Moreover, in [class.union.general] p6, some subobjects of the activated union member may also be considered created.

If we clarify that subobjects are not created, we'll need some different wording to specify implicitly starting the lifetime of a subobject.

The current wording seemingly implies that explicit operations only create the outermost object, while implicit creation may create the outermost object and its subobjects, which is inconsistent.