length_error exception not being thrown

[Quuxplusone]

Bugzilla Link	PR47140
Status	NEW
Importance	P normal
Reported by	M A (teammember0x01@gmail.com)
Reported on	2020-08-12 09:18:52 -0700
Last modified on	2020-08-13 11:41:44 -0700
Version	10.0
Hardware	Macintosh All
CC	blitzrakete@gmail.com, dgregor@apple.com, erik.pilkington@gmail.com, llvm-bugs@lists.llvm.org, mclow.lists@gmail.com, richard-llvm@metafoo.co.uk
Fixed by commit(s)
Attachments
Blocks
Blocked by
See also

This program should cause a length_error immediately but it doesn't. This
behavior is followed by GCC 10 and MetroWerks C++ compilers.

#include <iostream>
#include <string>

using namespace std;

int main(int argc, const char * argv[])
{
    string buffer("hi");
    try {
        std::string mystring(buffer.max_size()+1, 'X');
    }

    catch(length_error &l) {
        cout<<"Caught length_error exception: "<<l.what()<<endl;
    }

    catch(exception &e) {
        cout<<"Caught exception: "<<e.what()<<endl;
    }
    return 0;
}

What actually happens is the program crashes with EXC_BAD_ACCESS on Mac OS
10.8. Clang 5.1 and Clang 10.0.0.1 have this issue.

[Quuxplusone]

This works for me with Apple LLVM version 10.0.1 (clang-1001.0.46.4) on Mac OS 10.14, so perhaps it's already fixed? In any case, reclassifying as a libc++ bug.

[Quuxplusone]

After some help I think I know what is wrong. The std::string::max_size()
function returns the wrong value. It currently returns 18446744073709551599 or
2^64. That is the maximum amount of memory a 64-bit CPU can address. The actual
limit of my MacBook Pro is way less than that.

https://www.compuram.de/blog/en/how-much-ram-can-be-addressed-under-the-current-32-bit-and-64-bit-operating-systems/
According to this page the maximum value should be 96 GB of ram.

[Quuxplusone]

That is the maximum amount of memory a 64-bit CPU can address. The actual limit of my MacBook Pro is way less than that.

And that is the correct answer.

Why should your program be limited by the amount of memory that can be accessed by the machine it was built on?

[Quuxplusone]

It is true that a program should not be limited by the amount of memory the
machine that built the program had. The program could run on a machine with 100
TB of ram. No one is going to know how much memory a user's machine has without
checking first.

This is why I think these could be possible solutions:

1) Have max_size() return the amount of installed RAM on the users computer.
- Very simple to implement.
- The problem with this solution is it doesn't account for the fact that the
host computer could be using virtual memory.

2) Have max_size() return the virtual memory size.
- This value would be very specific to the machine it is one.
- It would probably represent the largest amount of memory that can be
allocated.
- Might not account memory usage of other processes.

3) Have max_size() return the value of some function that does know the maximum
amount of memory that is available to a process.
- I assume this function exists, just haven't located it yet.

4) Have max_size() figure out how much memory is available by calculating the
value.
- This could work by seeing how many times we can call malloc() before it fails.
- Not a pretty solution.

One thing I do know is not the solution is returning the maximum amount of
addressable memory for the current CPU's architecture. This value would simply
be wrong because of all the incorrect assumptions this value would make.

[Quuxplusone]

The actual requirement for max_size is not that helpful
(http://wg21.link/string.capacity):

     Returns: The largest possible number of char-like objects that can be stored in a basic_string.

but that's clearly not talking about the amount of available memory at any
given moment.

[Quuxplusone]

The actual correct answer (and what libc++ already does) is "ask the allocator".

Allocators have a member function 'max_size', which is the maximum number of bytes that the allocator can possibly allocate, or alternately, allocations above this size will always fail, no matter what.

The default behavior of that call is: numericlimits<size- type>::max() / sizeof (value_type)

for std::allocator, size_type is std::size_t (64 bits, probably), and value_type is char (sizeof(char) == 1), which gives you "2^^64 - 1", which is what you're seeing.

[Quuxplusone]

I think this discussion might not actually be relevant to the original reported
crash.

It shouldn't matter what max_size() returns, within reason, because the

  basic_string(size_type n, charT)

constructor starts by checking if n > max_size() and throwing if so (in the
corresponding __init function). So presumably the only issue is whether

  buffer.max_size() + 1

overflows. Which it doesn't, because basic_string::max_size() always returns a
value <= allocator::max_size() - 16.

So I think whatever problem was causing the reported crash is either already
fixed or is not a max_size() problem (perhaps something is going wrong as part
of throwing the exception?).

[Quuxplusone]

I tested this with a recently built clang and Apple clang version 11.0.3 (clang-1103.0.32.62) and in both cases got the following output:

Caught length_error exception: basic_string

[Quuxplusone]

Richard Smith thinks this is a libc++ issue. Is there a way to print the exact version of libc++ being used by clang? clang -v and clang --verbose do not display info about libc++.

[Quuxplusone]

(In reply to Marshall Clow (home) from comment #8)
> I tested this with a recently built clang and Apple clang version 11.0.3
> (clang-1103.0.32.62) and in both cases got the following output:
>
> Caught length_error exception: basic_string

Do you know what version of libc++ you are using?

[Quuxplusone]

Does throwing and catching exceptions work in general in your environment? It doesn't look like any relevant part of the libc++ code has changed in several years.

[Quuxplusone]

(In reply to M A from comment #10)
> (In reply to Marshall Clow (home) from comment #8)
> > I tested this with a recently built clang and Apple clang version 11.0.3
> > (clang-1103.0.32.62) and in both cases got the following output:
> >
> > Caught length_error exception: basic_string
>
> Do you know what version of libc++ you are using?

The one that Apple ships with Mac OS 10.15
_LIBCPP_VERSION is 9000

[Quuxplusone]

(In reply to Richard Smith from comment #11)
> Does throwing and catching exceptions work in general in your environment?
> It doesn't look like any relevant part of the libc++ code has changed in
> several years.

Yes exception do work on my computer. This is the test program I used try test
throwing exceptions:

#include <iostream>
#include <string>
#include <ios>

using namespace std;

int main(int argc, const char * argv[])
{
    try {
        //throw length_error("This is a test throw of length_error");
        throw exception();
    }

    catch(length_error &l) {
        cout<<"Caught length_error exception: "<<l.what()<<endl;
    }

    catch(exception &e) {
        cout<<"Caught exception: "<<e.what()<<endl;
    }
    return 0;
}

Both length_error and exception were caught.

[Quuxplusone]

(In reply to Marshall Clow (home) from comment #12)
> (In reply to M A from comment #10)
> > (In reply to Marshall Clow (home) from comment #8)
> > > I tested this with a recently built clang and Apple clang version 11.0.3
> > > (clang-1103.0.32.62) and in both cases got the following output:
> > >
> > > Caught length_error exception: basic_string
> >
> > Do you know what version of libc++ you are using?
>
> The one that Apple ships with Mac OS 10.15
> _LIBCPP_VERSION is 9000

I used this program to find the version:

#include <iostream>
#include <string>

using namespace std;

int main(int argc, const char * argv[])
{
    cout<<"Value = "<<_LIBCPP_VERSION<<endl;
    return 0;
}

The result was 10000 with clang 10.0.1.