Allocation size truncated to 32 bits in realloc then sign-extended back to 64 bits

[GoogleCodeExporter]

What steps will reproduce the problem?
1.  Allocate array on 64-bit platform
2.  Realloc array to size greater than 2GB

What is the expected output? What do you see instead?

Reallocation is expected to succeed, given sufficient memory on the system.  
Actual output depends on size.  Generally either ridiculously large "large 
alloc" warnings (due to sign extension, see below) or segmentation faults when 
accessing the realloc'd array due to the allocation being too small.

What version of the product are you using? On what operating system?

We are currently using 1.7 on Linux x86-64, but have confirmed the problem 
still exists in 1.9.1.

Please provide any additional information below.

The problem is in tcmalloc.cc in do_realloc_with_callback().  Specifically, 
lower_bound_to_grow and upper_bound_to_shrink are declared as const int, and 
thus are not big enough to handle large allocations on platforms (such as linux 
x86-64) where int is 32 bits while size_t is 64 bits.  Even worse, because int 
is signed while size_t is not, if one of these is negative and gets passed to 
another reallocation routine which expects a size_t, such as 
do_malloc_or_cpp_alloc(), the result is first sign-extended to a 64-bit signed 
integer, then cast to 64-bit unsigned.  This causes TCMALLOC to incorrectly 
trigger "large allocation" warnings.

The following patch seems to work for us.

1186,1187c1455,1456
<   const int lower_bound_to_grow = old_size + old_size / 4;
<   const int upper_bound_to_shrink = old_size / 2;

---
>   const size_t lower_bound_to_grow = old_size + old_size / 4ul;
>   const size_t upper_bound_to_shrink = old_size / 2ul;

Original issue reported on code.google.com by sdvor...@cray.com on 19 Sep 2012 at 3:16

[GoogleCodeExporter]

Original comment by chapp...@gmail.com on 3 Nov 2012 at 4:47

• Changed state: Accepted

[GoogleCodeExporter]

r187 | chappedm@gmail.com | 2012-12-22 14:02:52 -0500 (Sat, 22 Dec 2012) | 3 
lines

issue-467: Fixed issue with allocation size being narrowed to 32-bit

Original comment by chapp...@gmail.com on 22 Dec 2012 at 7:03

• Changed state: Fixed