paulfloyd / freebsd_valgrind

Git repo used to Upstream the FreeBSD Port of Valgrind
GNU General Public License v2.0
15 stars 4 forks source link

memcheck/tests/dw4 is failing [clang] #125

Closed paulfloyd closed 2 years ago

paulfloyd commented 4 years ago

I added an expected

paulf> diff dw4.stderr.exp dw4.stderr.exp-freebsd 
30,31c30,31
< Unaddressable byte(s) found during client check request
<    at 0x........: croak (dw4.c:25)
---
> Uninitialised byte(s) found during client check request
>    at 0x........: croak (dw4.c:32)

This is some peculiarity about accessing after brk. Seems to work on FreeBSD with clang.

In this test

  void *addr = sbrk(0);
  croak(addr); // in the first brk page, after brk_limit

I get Uninit until I ads 4096 to addr.

That means that either clang/FreeBSD is putting a page after brk, of the Valgrind code is out by 1 page.

If I try to write to addr I get a coredump.

So it looks like a Valgrind errror.

Run with -d -d and add a printf of addr to the testcase and compare the two brks.

The printf says

brk is 0x4020000

and aspacem

--22230:1: aspacem (0,4,7) /usr/home/paulf/scratch/valgrind/none/none-amd64-freebsd
--22230:1: aspacem (1,65,7) /usr/home/paulf/scratch/valgrind/memcheck/tests/dw4
--22230:1: aspacem (2,121,7) /libexec/ld-elf.so.1
--22230:1: aspacem (3,146,1) /tmp/vgdb-pipe-shared-mem-vgdb-22230-by-paulf-on-euler
--22230:1: aspacem (4,205,8) /usr/home/paulf/scratch/valgrind/coregrind/vgpreload_core-amd64-freebsd.so
--22230:1: aspacem (5,284,8) /lib/libc.so.7
--22230:1: aspacem (6,336,0) [free slot: size=48  next=0]
--22230:1: aspacem (7,388,1) /usr/home/paulf/scratch/valgrind/memcheck/tests/valgrind-dw4-test.22230
--22230:1: aspacem   0: RSVN 0000000000-00001fffff 2097152 ----- SmFixed
--22230:1: aspacem   1: file 0000200000-0000200fff    4096 r---- d=0x696e301b i=1461799 o=0       (1,65)
--22230:1: aspacem   2: file 0000201000-0000202fff    8192 r-xT- d=0x696e301b i=1461799 o=0       (1,65)
--22230:1: aspacem   3: file 0000203000-0000203fff    4096 r---- d=0x696e301b i=1461799 o=4096    (1,65)
--22230:1: aspacem   4: file 0000204000-0000204fff    4096 rw--- d=0x696e301b i=1461799 o=4096    (1,65)
--22230:1: aspacem   5: RSVN 0000205000-0003ffffff     61m ----- SmFixed
--22230:1: aspacem   6: file 0004000000-0004005fff   24576 r---- d=0x4190bc5c i=1081502 o=0       (2,121)
--22230:1: aspacem   7: file 0004006000-000401cfff   94208 r-xT- d=0x4190bc5c i=1081502 o=20480   (2,121)
--22230:1: aspacem   8: file 000401d000-000401dfff    4096 r---- d=0x4190bc5c i=1081502 o=110592  (2,121)
--22230:1: aspacem   9: file 000401e000-000401efff    4096 rw--- d=0x4190bc5c i=1081502 o=110592  (2,121)
--22230:1: aspacem  10: anon 000401f000-000401ffff    4096 rw---
--22230:1: aspacem  11: anon 0004020000-0004020fff    4096 rwx--
need to redo but I think SmLower RSVN was next

What are those two 4k pages at the end?

What does Linux say?

RHEL7.6, GCC 12ish

Standalone ./dw4 addr is 0x426000 Segmentation fault

Under Valgrind nullgrind

addr is 0x4245000

and

--28937:1: aspacem (0,4,9) /home/pafloyd/scratch/valgrind/none/none-amd64-linux                                           
--28937:1: aspacem (1,61,8) /home/pafloyd/scratch/valgrind/memcheck/tests/dw4                                             
--28937:1: aspacem (2,115,4) /usr/lib64/ld-2.17.so                                                                        
--28937:1: aspacem (3,141,1) /tmp/vgdb-pipe-shared-mem-vgdb-28937-by-pafloyd-on-ies-ams-dev-19                            
--28937:1: aspacem (4,211,8) /home/pafloyd/scratch/valgrind/coregrind/vgpreload_core-amd64-linux.so                       
--28937:1: aspacem (5,286,0) [free slot: size=17  next=0]                                                                 
--28937:1: aspacem (6,307,5) /usr/lib64/libc-2.17.so                                                                      
--28937:1: aspacem (7,335,1) /home/pafloyd/scratch/valgrind/memcheck/tests/valgrind-dw4-test.28937                        
--28937:1: aspacem   0: RSVN 0000000000-00003fffff 4194304 ----- SmFixed                                                  
--28937:1: aspacem   1: file 0000400000-0000400fff    4096 r---- d=0x028 i=2736641967 o=0       (1,61)                    
--28937:1: aspacem   2: file 0000401000-0000401fff    4096 r-xT- d=0x028 i=2736641967 o=4096    (1,61)                    
--28937:1: aspacem   3: file 0000402000-0000402fff    4096 r---- d=0x028 i=2736641967 o=8192    (1,61)                    
--28937:1: aspacem   4: file 0000403000-0000403fff    4096 r---- d=0x028 i=2736641967 o=8192    (1,61)                    
--28937:1: aspacem   5: file 0000404000-0000404fff    4096 rw--- d=0x028 i=2736641967 o=12288   (1,61)                    
--28937:1: aspacem   6: RSVN 0000405000-0003ffffff     59m ----- SmFixed                                                  
--28937:1: aspacem   7: file 0004000000-0004021fff  139264 r-xT- d=0x802 i=130254  o=0       (2,115)                      
--28937:1: aspacem   8: anon 0004022000-0004022fff    4096 rw---                                                          
--28937:1: aspacem   9: file 0004023000-0004023fff    4096 r---- d=0x028 i=2735370626 o=0       (4,211)                   
--28937:1: aspacem  10: file 0004024000-0004024fff    4096 r-xT- d=0x028 i=2735370626 o=4096    (4,211)                   
--28937:1: aspacem  11: file 0004025000-0004025fff    4096 r---- d=0x028 i=2735370626 o=8192    (4,211)                   
--28937:1: aspacem  12: file 0004026000-0004026fff    4096 r---- d=0x028 i=2735370626 o=8192    (4,211)                   
--28937:1: aspacem  13: file 0004027000-0004027fff    4096 rw--- d=0x028 i=2735370626 o=12288   (4,211)                   
--28937:1: aspacem  14: anon 0004028000-000402bfff   16384 rw---                                                          
--28937:1: aspacem  15: file 000402c000-000402cfff    4096 rw--- d=0x028 i=2736641968 o=0       (7,335)                   
--28937:1: aspacem  16: anon 000402d000-000402dfff    4096 rw---                                                          
--28937:1: aspacem  17:      000402e000-000405efff  200704                                                                
--28937:1: aspacem  18: anon 000405f000-0004061fff   12288 rw---
--28937:1: aspacem  19:      0004062000-0004220fff 1830912
--28937:1: aspacem  20: file 0004221000-0004221fff    4096 r---- d=0x802 i=130254  o=135168  (2,115)
--28937:1: aspacem  21: file 0004222000-0004222fff    4096 rw--- d=0x802 i=130254  o=139264  (2,115)
--28937:1: aspacem  22: anon 0004223000-0004223fff    4096 rw---
--28937:1: aspacem  23: anon 0004224000-0004244fff  135168 rwx--
--28937:1: aspacem  24: RSVN 0004245000-0004a23fff 8253440 ----- SmLower

Just a few comments. The Linux map has a fair number of differences. core.so is mapped lower than SmLower RSVN, ld.so is split.

Linus has this just before the free space

--28937:1: aspacem 23: anon 0004224000-0004244fff 135168 rwx--

That's 128k and brk is just after that.

So why on Linux is it correctly after the 128k anon rwx mapping but on FreeBSD it is wrong and not after the 4k anon rwx mapping? Lower case anon means client mapping.

There are (at least) 3 places that could be mapping this

VG_(am_notify_client_mmap)( Addr a, SizeT len, UInt prot, UInt flags,

SysRes VG_(am_mmap_anon_fixed_client) ( Addr start, SizeT length, UInt prot )

static SysRes am_mmap_anon_float_client ( SizeT length, Int prot, Bool isCH )

am_mmap_anon_fixedclient is called twice, SysRes VG(am_mmap_anon_floatclient) ( SizeT length, Int prot ) and SysRes VG(am_mmap_client_heap) ( SizeT length, Int prot )

can't be the second cos that would make the segment be marked with 'H' for heap can't be the first cos that is only called on macos

VG_(am_mmap_anon_fixed_client) is called twice from initimg-freebsd and once from elf.c

elf.c is from mapelf when memsz > filesz. This does involve brkaddr.

On Linux the elf call does get used, but it looks like it is just padding after the ld.so RW PT_LOAD.

before

--28568:0: aspacem   4: file 0000403000-0000404fff    8192 rw--- d=0x028 i=2735174488 o=8192    (1,61)
--28568:0: aspacem   5: RSVN 0000405000-0003ffffff     59m ----- SmFixed                              
--28568:0: aspacem   6: file 0004000000-0004021fff  139264 r-x-- d=0x802 i=130254  o=0       (2,115)  
--28568:0: aspacem   7:      0004022000-0004220fff 2093056                                            
--28568:0: aspacem   8: file 0004221000-0004222fff    8192 rw--- d=0x802 i=130254  o=135168  (2,115)  
--28568:0: aspacem   9:      0004223000-0057ffffff   1341m  

after

--28568:0: aspacem   4: file 0000403000-0000404fff    8192 rw--- d=0x028 i=2735174488 o=8192    (1,61)
--28568:0: aspacem   5: RSVN 0000405000-0003ffffff     59m ----- SmFixed
--28568:0: aspacem   6: file 0004000000-0004021fff  139264 r-x-- d=0x802 i=130254  o=0       (2,115)
--28568:0: aspacem   7:      0004022000-0004220fff 2093056
--28568:0: aspacem   8: file 0004221000-0004222fff    8192 rw--- d=0x802 i=130254  o=135168  (2,115)
--28568:0: aspacem   9: anon 0004223000-0004223fff    4096 rw---
--28568:0: aspacem  10:      0004224000-0057ffffff   1341m

final

--29705:1: aspacem  21: file 0004222000-0004222fff    4096 rw--- d=0x802 i=130254  o=139264  (2,115)
--29705:1: aspacem  22: anon 0004223000-0004223fff    4096 rw---
--29705:1: aspacem  23: anon 0004224000-0004244fff  135168 rwx--
--29705:1: aspacem  24: RSVN 0004245000-0004a23fff 8253440 ----- SmLower
--29705:1: aspacem  25: file 0004a24000-0004be5fff 1843200 r-xT- d=0x802 i=130261  o=0       (6,307)
--29705:1: aspacem  26: file 0004be6000-0004de5fff 2097152 ----- d=0x802 i=130261  o=1843200 (6,307)
--29705:1: aspacem  27: file 0004de6000-0004de9fff   16384 r---- d=0x802 i=130261  o=1843200 (6,307)
--29705:1: aspacem  28: file 0004dea000-0004debfff    8192 rw--- d=0x802 i=130261  o=1859584 (6,307)
--29705:1: aspacem  29: anon 0004dec000-0004df0fff   20480 rw---
--29705:1: aspacem  30:      0004df1000-0057ffffff   1330m

So that is the page before the mystery RWX page, but not the one that I want.

elf.c has some nice ifs that make this easy to trace.

@TODO check if this is changing brk correctly

split RW PT_LOAD does not affect this

init-img stack allocation nah

setup_client_dataseg

BINGO

On Linux before

-5235:0: aspacem   8: file 0004221000-0004222fff    8192 rw--- d=0x802 i=130254  o=135168  (2,115)  
--5235:0: aspacem   9: anon 0004223000-0004223fff    4096 rw---                                      
--5235:0: aspacem  10:      0004224000-0004224fff    4096                                            
--5235:0: aspacem  11: RSVN 0004225000-0004a23fff 8384512 ----- SmLower  

after

--5235:0: aspacem   8: file 0004221000-0004222fff    8192 rw--- d=0x802 i=130254  o=135168  (2,115)
--5235:0: aspacem   9: anon 0004223000-0004223fff    4096 rw---
--5235:0: aspacem  10: anon 0004224000-0004224fff    4096 rwx--
--5235:0: aspacem  11: RSVN 0004225000-0004a23fff 8384512 ----- SmLower

with

   VG_(am_show_nsegments)(0,"before mmap client heap");

   sres = VG_(am_mmap_anon_fixed_client)( 
             anon_start, 
             anon_size, 
             VKI_PROT_READ|VKI_PROT_WRITE|VKI_PROT_EXEC
          );

   VG_(am_show_nsegments)(0,"after mmap client heap");

DEBUG: setup_client_dataseg ok is 0 resvn start 0x406000 DEBUG: setup_client_dataseg ok is 1 new resvn start 0x4225000

that first attempt seems hopeless, it's where part of ld.so is already mapped

the second one looks OK, before the client heap gets grown

So where does the 4k RWX mapping get extended by 128k to 132k? Can't see any client mmaps doing it

Back on Freebsd, similar, first attempt at am_create_reservation fails so get advisory and try again.

That succeeds.

DEBUG: setup_client_dataseg ok is 0 resvn_start 0x206000 DEBUG: setup_client_dataseg ok is 1 try again resvn_start 0x4021000 anon_start 0x4020000

I think that this is wrong

         if (ok)
            VG_(brk_base) = VG_(brk_limit) = anon_start;

brk_limit should be anaon_start+anaon_size.

Ned to debug in do_brk. Modify debug flag, enable show segments.

Back to Linux

DEBUG: setup_client_dataseg ok is 0 resvn start 0x406000
DEBUG: setup_client_dataseg ok is 1 new resvn start 0x4225000 anon_start 0x4224000
...
SYSCALL[30705,1](12) sys_brk ( 0x0 )
do_brk: brk_base=0x4224000 brk_limit=0x4224000 newbrk=0x0
...
--30705:0: aspacem   8: file 0004221000-0004222fff    8192 rw--- d=0x802 i=130254  o=135168  (2,115)
--30705:0: aspacem   9: anon 0004223000-0004223fff    4096 rw---
--30705:0: aspacem  10: anon 0004224000-0004224fff    4096 rwx--
--32054:0: aspacem  11: RSVN 0004225000-0004a23fff 8384512 ----- SmLower
a load of syscalls looks like client running and loading libc
SYSCALL[32054,1](12) sys_brk ( 0x0 )
[as previously]
 --> [pre-success] Success(0x4224000) 
SYSCALL[32054,1](12) sys_brk ( 0x4245000 )
... extend ...
--32054:0: aspacem  17: file 0004221000-0004221fff    4096 r---- d=0x802 i=130254  o=135168  (2,115)
--32054:0: aspacem  18: file 0004222000-0004222fff    4096 rw--- d=0x802 i=130254  o=139264  (2,115)
--32054:0: aspacem  19: anon 0004223000-0004223fff    4096 rw---
--32054:0: aspacem  20: anon 0004224000-0004244fff  135168 rwx--

Calls to sbrk From _dl_sysdep_start with a size of 0 From sysmalloc From _int_malloc From malloc From main

Hmm. OK. And what about with memcheck? What does FreeBSD malloc do? @TODO gdb FreeBSD b sbrk

I guess this is a difference between Linux malloc and FreeBSD malloc. To me it looks like the 32bye malloc on Linux is using sbrk for 132k which then gets used for small allocations (?)

memcheck on Linux, obviously now there is memcheck.so

DEBUG: setup_client_dataseg ok is 0 resvn start 0x406000 DEBUG: setup_client_dataseg ok is 1 new resvn start 0x4225000 anon_start 0x4224000

SYSCALL8864,1 sys_brk ( 0x0 ) SYSCALL8864,1 sys_brk ( 0x0 ) SYSCALL8864,1 sys_brk ( 0x0 ) SYSCALL8864,1 sys_brk ( 0x4225000 ) SYSCALL8864,1 sys_brk ( 0x0 ) SYSCALL8864,1 sys_brk ( 0x4224400 )

paulfloyd commented 2 years ago

And on FreeBSD 14.0 more diff

--- dw4.stderr.exp-freebsd      2022-04-03 13:49:47.961688000 +0200
+++ dw4.stderr.out      2022-05-31 22:51:08.784810000 +0200
@@ -25,12 +25,12 @@
 Uninitialised byte(s) found during client check request
    at 0x........: croak (dw4.c:32)
    by 0x........: main (dw4.c:82)
- Address 0x........ is in a rw- mapped file valgrind-dw4-test.PID segment
+ Address 0x........ is in a rw- mapped file [unknown] segment

This should be coming from the filter

$SED "s/file .*valgrind-dw4-test.[1-9][0-9]*/file valgrind-dw4-test.PID/"

so for some reason the file is not found :-(((

paulfloyd commented 2 years ago

Got it.

At the end of initimg-linux.c there is this

/ Tell the tool about the client data segment and then kill it which will make it inaccessible/unaddressable. / const NSegment seg = VG_(am_findnsegment)(VG(brk_base)); vg_assert(seg); vg_assert(seg->kind == SkAnonC); VG_TRACK(new_membrk, VG(brkbase), seg->end + 1 - VG(brk_base), 1/tid*/); VG_TRACK(die_membrk, VG(brkbase), seg->end + 1 - VG(brk_base));

which is missing from FreeBSD.

It is also missing from Solaris and Darwin. Hmm. I wonder if they pass this test?

paulfloyd commented 2 years ago

commit 7b0cd3ac2884d2557a476e5010d44e60ff531846 (HEAD -> master, origin/master, origin/HEAD) Author: Paul Floyd pjfloyd@wanadoo.fr Date: Thu Jun 2 20:31:12 2022 +0200

Add code that makes 1st page after brk inaccessible.

Also remove FreeBSD expected.

Note: this code is also mssing from Solaris and macOS initimg's.