search

LouisBrunner / valgrind-macos

A valgrind mirror with latest macOS support
GNU General Public License v2.0
1.14k stars 58 forks source link

Errors on every program run through Valgrind on Monterey 12.7.2 #104

Open pjaglom opened 10 months ago

pjaglom commented 10 months ago

Context

I am running valgrind on a 2016 Intel-based MacBook Pro running macOS 12.7.2. For context this is my first foray into C, gdb, and valgrind so my experience level is low. I recently installed valgrind-macos to be able to run it on my own machine instead of VMs (thank you for this excellent work - it was such a relief not to have to deal with the latency!). Valgrind seemed to install without error, and does identify issues with the code I've run through it, but it also identifies errors and leaks that don't seem to be associated with my programs (since they valgrind cleanly on linux). They seem to be related to dylib or dyld. I read through #11, #19, and #74 which all suggested this kind of issue should have been suppressed/corrected so I wonder if I'm missing a dependency or should install a different branch of valgrind-macos. (I followed the instructions in the README.)

I do note that 103 other errors are suppressed, but I'm wondering if there's a way to get the last few suppressed as well, or if I should simply ignore any errors related to dylib or dyld.

What went wrong?

Here is an example of the kinds of errors I see:

❯ valgrind ./sortLines input.txt                                                                                                                    ─╯
==78927== Memcheck, a memory error detector
==78927== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==78927== Using Valgrind-3.22.0.GIT-lbmacos and LibVEX; rerun with -h for copyright info
==78927== Command: ./sortLines input.txt
==78927==
==78927== Invalid read of size 32
==78927==    at 0x7FF81870BA21: ???
==78927==    by 0x7FF8185C83FD: __sfvwrite (in /usr/lib/system/libsystem_c.dylib)
==78927==    by 0x7FF8185C690D: __vfprintf (in /usr/lib/system/libsystem_c.dylib)
==78927==    by 0x7FF8185D6C45: vfprintf_l (in /usr/lib/system/libsystem_c.dylib)
==78927==    by 0x7FF8185F6058: printf (in /usr/lib/system/libsystem_c.dylib)
==78927==    by 0x100003E2B: main (sortLines.c:72)
==78927==  Address 0x10020bd60 is 0 bytes after a block of size 32 alloc'd
==78927==    at 0x1001F2941: realloc (in /usr/local/Cellar/valgrind/HEAD-d14be1d/libexec/valgrind/vgpreload_memcheck-amd64-darwin.so)
==78927==    by 0x7FF8185F54DB: sappend (in /usr/lib/system/libsystem_c.dylib)
==78927==    by 0x7FF8185F53D5: getdelim (in /usr/lib/system/libsystem_c.dylib)
==78927==    by 0x100003C0B: readInput (sortLines.c:24)
==78927==    by 0x100003DE5: main (sortLines.c:67)
==78927==
This is a sample test case
go!
==78927== Invalid read of size 32
==78927==    at 0x7FF81870BA1D: ???
==78927==    by 0x7FF8185C83FD: __sfvwrite (in /usr/lib/system/libsystem_c.dylib)
==78927==    by 0x7FF8185C690D: __vfprintf (in /usr/lib/system/libsystem_c.dylib)
==78927==    by 0x7FF8185D6C45: vfprintf_l (in /usr/lib/system/libsystem_c.dylib)
==78927==    by 0x7FF8185F6058: printf (in /usr/lib/system/libsystem_c.dylib)
==78927==    by 0x100003E2B: main (sortLines.c:72)
==78927==  Address 0x10020bdc0 is 16 bytes after a block of size 16 in arena "client"
==78927==
it checks your program's one argument behavior
==78927==
==78927== HEAP SUMMARY:
==78927==     in use at exit: 12,475 bytes in 170 blocks
==78927==   total heap usage: 182 allocs, 12 frees, 16,804 bytes allocated
==78927==
==78927== LEAK SUMMARY:
==78927==    definitely lost: 4,160 bytes in 130 blocks
==78927==    indirectly lost: 0 bytes in 0 blocks
==78927==      possibly lost: 600 bytes in 3 blocks
==78927==    still reachable: 7,715 bytes in 37 blocks
==78927==         suppressed: 0 bytes in 0 blocks
==78927== Rerun with --leak-check=full to see details of leaked memory
==78927==
==78927== For lists of detected and suppressed errors, rerun with: -s
==78927== ERROR SUMMARY: 3 errors from 2 contexts (suppressed: 103 from 19)

If I run with --leak-check=full I get additional errors/leaks, but didn't want to spam the issue with a wall of text. If helpful, I can add.

I modified the following program, which was from an assignment in an online course. (Please forgive the messy code...still learning.)

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

//This function is used to figure out the ordering
//of the strings in qsort.  You do not need
//to modify it.
int stringOrder(const void * vp1, const void * vp2) {
  const char * const * p1 = vp1;
  const char * const * p2 = vp2;
  return strcmp(*p1, *p2);
}
//This function will sort and print data (whose length is count).
void sortData(char ** data, size_t count) {
  qsort(data, count, sizeof(char *), stringOrder);
}

char ** readInput(FILE * f, size_t * count) {
  char ** lines = NULL;
  char * curr = NULL;
  size_t sz;
  *count = 0;
  while (getline(&curr, &sz, f) >= 0) {
    lines = realloc(lines, (*count+1) * sizeof(*lines));
    lines[*count] = curr;
    curr = NULL;
    (*count)++;
  }
  free(curr);
  return lines;
}

void closeOnError(char * errMsg, int i, size_t * count) {
  fprintf(stderr, errMsg, i);
  free(count);
  exit(EXIT_FAILURE);
}

int main(int argc, char ** argv) {
  FILE * f = NULL;
  char ** strings = NULL; // array to hold lines to sort
  size_t * count = malloc(sizeof(*count));

  if (argc == 1) {
    // read from stdin
    f = stdin;
    // obtain list of strings and write into an array
    strings = readInput(f, count);
    // sort the lines of input
    sortData(strings, *count);
    // print the sorted strings and free memory for each string
    for (int i = 0; i<*count; i++) {
      printf("%s", strings[i]);
      free(strings[i]);
    }
    // free memory for array of strings
    free(strings);
  } else {
    // read from one or more input files
    for (int i = 1; i < argc; i++) {
      f = fopen(argv[i], "r");
      if (f == NULL) {
        closeOnError("Could not open input file %d!", i, count);
      }
      // obtain list of strings and write into an array
      strings = readInput(f, count);
      // sort the lines of input
      sortData(strings, *count);
      // print the sorted lines
      for (int i = 0; i<*count; i++) {
        printf("%s", (strings)[i]);
        free(strings[i]);
      }
      // free memory
      free(strings);
      strings = NULL;
      if (fclose(f) != 0) {
        closeOnError("Could not close input file %d!", i, count);
      }
    }
  }
  free(count);
  return EXIT_SUCCESS;
}

I compiled with gcc -ggdb3 -Wall -Werror -std=gnu99 -pedantic -o sortLines sortLines.c.

I ran valgrind ./sortLines input.txt where input.txt is:

This is a sample test case
it checks your program's one argument behavior
go!

What did you expect to happen?

I expected the following output, which is what I get when running through valgrind on a linux VM.

==270== Memcheck, a memory error detector
==270== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==270== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==270== Command: ./sortLines input.txt
==270== 
This is a sample test case
go!
it checks your program's one argument behavior
==270== 
==270== HEAP SUMMARY:
==270==     in use at exit: 0 bytes in 0 blocks
==270==   total heap usage: 11 allocs, 11 frees, 10,224 bytes allocated
==270== 
==270== All heap blocks were freed -- no leaks are possible
==270== 
==270== For lists of detected and suppressed errors, rerun with: -s
==270== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Information

LouisBrunner commented 10 months ago

Hi @pjaglom,

Thanks for all the context! You are right that sometimes we need to add suppressions to prevent false positives to show in Valgrind.

Unfortunately I don't have access to a macOS 12 x86 computer, so I did my testing on my macOS 14 x86 one. I get a slightly different output

==7807== Memcheck, a memory error detector
==7807== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==7807== Using Valgrind-3.22.0.GIT-lbmacos and LibVEX; rerun with -h for copyright info
==7807== Command: ./sortLines input.txt
==7807==
...
snip, unrelated error linked to macOS 13 or 14 support
...
==7807== Invalid read of size 32
==7807==    at 0x7FF80C2A521D: ???
==7807==    by 0x7FF80C147B7F: getdelim (in /usr/lib/system/libsystem_c.dylib)
==7807==    by 0x100003C7B: readInput (sortLines.c:24)
==7807==    by 0x100003E55: main (sortLines.c:67)
==7807==  Address 0x100746280 is 32 bytes before a block of size 4,096 in arena "client"
==7807==
==7807== Invalid read of size 32
==7807==    at 0x7FF80C2A521D: ???
==7807==    by 0x7FF80C11FA35: __sfvwrite (in /usr/lib/system/libsystem_c.dylib)
==7807==    by 0x7FF80C11DE9D: __vfprintf (in /usr/lib/system/libsystem_c.dylib)
==7807==    by 0x7FF80C12CACD: vfprintf_l (in /usr/lib/system/libsystem_c.dylib)
==7807==    by 0x7FF80C1488C0: printf (in /usr/lib/system/libsystem_c.dylib)
==7807==    by 0x100003E9B: main (sortLines.c:72)
==7807==  Address 0x1007472c0 is 32 bytes before a block of size 32 in arena "client"
==7807==
This is a sample test case
go!
it checks your program's one argument behavior
==7807==
==7807== HEAP SUMMARY:
==7807==     in use at exit: 12,569 bytes in 174 blocks
==7807==   total heap usage: 193 allocs, 19 frees, 17,530 bytes allocated
==7807==
==7807== LEAK SUMMARY:
==7807==    definitely lost: 4,352 bytes in 136 blocks
==7807==    indirectly lost: 48 bytes in 1 blocks
==7807==      possibly lost: 576 bytes in 2 blocks
==7807==    still reachable: 7,593 bytes in 35 blocks
==7807==         suppressed: 0 bytes in 0 blocks

My errors seem to indicate that some macOS function (at 0x7FF80C2A521D) is doing some invalid read and being called by both getline and printf (could be some new flavor of stack checking, I don't know). Your error is a bit different, as it only happens on printf and thus could be genuine. However, I couldn't find any obvious error after reviewing the code, so it might be some kind of false positive.

Unfortunately, because this issue happens in the printf (when the data is being read), we cannot add a suppression in Valgrind itself as it would hide any issue happening in printf, some of which would be genuine. But you can hide it on your side by creating a suppression file (example: https://github.com/LouisBrunner/valgrind-macos/blob/main/darwin23.supp, you can then use it with --suppressions=yourfile, you can also get Valgrind to help with creating this tricky format using --gen-suppressions=all).

pjaglom commented 10 months ago

I could be wrong, but I don't think my error is genuine, because the clean output I wrote in the "what did I expect to happen" section is what I get when I compile and run the exact same code in a linux environment.

I'll take a look at creating my own suppression file - thanks!

The specifics of what is really happening in memory during these errors is a little over my head, but I'm just curious if this is something that could theoretically be corrected (not just suppressed), so valgrind runs as cleanly in macOS as it does in linux? Not that I'm asking you to do that - I'm about to start an Intro to OS course, and after a few more level-ups, I'd like to be able to start contributing to projects like yours.

LouisBrunner commented 9 months ago

The specifics of what is really happening in memory during these errors is a little over my head, but I'm just curious if this is something that could theoretically be corrected (not just suppressed), so valgrind runs as cleanly in macOS as it does in linux?

It's entirely possible and it's what I am (slowly) trying to do. In general I would want macOS to run without any suppressions (which is what happens for most systems, macOS has the most suppressions by far IIRC). Some recent work as pushed us in this direction but we aren't there yet.

Not that I'm asking you to do that - I'm about to start an Intro to OS course, and after a few more level-ups, I'd like to be able to start contributing to projects like yours.

That's great to hear! 😄 You would be more than welcome here or on upstream valgrind (where most of the work happens, all we do here is macOS support)

Starcraft426 commented 5 months ago

I reproduced the same bug with a much more simple code:

#include <stdio.h>
#include <stdlib.h>

int main(void){
    char* string = malloc(11*sizeof(char));
    for (int i = 0; i<10; i++){
        string[i] = 48+i; // put numbers from 0 to 9
    }
    string[10] = '\0';
    printf("%s\n", string);
    free(string);
}

with this one I get the following errors:

==30882== Memcheck, a memory error detector
==30882== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==30882== Using Valgrind-3.23.0.GIT-lbmacos and LibVEX; rerun with -h for copyright info
==30882== Command: ./a.out
==30882== 
==30882== Invalid read of size 32
==30882==    at 0x7FF81119BE41: ???
==30882==    by 0x7FF81101B889: __sfvwrite (in /usr/lib/system/libsystem_c.dylib)
==30882==    by 0x7FF811019CF1: __vfprintf (in /usr/lib/system/libsystem_c.dylib)
==30882==    by 0x7FF81102890D: vfprintf_l (in /usr/lib/system/libsystem_c.dylib)
==30882==    by 0x7FF811044840: printf (in /usr/lib/system/libsystem_c.dylib)
==30882==    by 0x100000F78: main (test.c:10)
==30882==  Address 0x1006463a0 is 5 bytes after a block of size 11 alloc'd
==30882==    at 0x100501FE0: malloc (in /usr/local/Cellar/valgrind/HEAD-93191c1/libexec/valgrind/vgpreload_memcheck-amd64-darwin.so)
==30882==    by 0x100000F28: main (test.c:5)
==30882== 
==30882== Conditional jump or move depends on uninitialised value(s)
==30882==    at 0x7FF81119BE5F: ???
==30882==    by 0x7FF81101B889: __sfvwrite (in /usr/lib/system/libsystem_c.dylib)
==30882==    by 0x7FF811019CF1: __vfprintf (in /usr/lib/system/libsystem_c.dylib)
==30882==    by 0x7FF81102890D: vfprintf_l (in /usr/lib/system/libsystem_c.dylib)
==30882==    by 0x7FF811044840: printf (in /usr/lib/system/libsystem_c.dylib)
==30882==    by 0x100000F78: main (test.c:10)
==30882== 
0123456789
==30882== 
==30882== HEAP SUMMARY:
==30882==     in use at exit: 12,914 bytes in 172 blocks
==30882==   total heap usage: 183 allocs, 11 frees, 13,738 bytes allocated
==30882== 
==30882== LEAK SUMMARY:
==30882==    definitely lost: 0 bytes in 0 blocks
==30882==    indirectly lost: 0 bytes in 0 blocks
==30882==      possibly lost: 0 bytes in 0 blocks
==30882==    still reachable: 4,096 bytes in 1 blocks
==30882==         suppressed: 8,818 bytes in 171 blocks
==30882== Rerun with --leak-check=full to see details of leaked memory
==30882== 
==30882== Use --track-origins=yes to see where uninitialised values come from
==30882== For lists of detected and suppressed errors, rerun with: -s
==30882== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 126 from 40)

which is pretty similar. I think what is going on is if the size of the malloc'd bytes is less than 32, it is rounded to this number. Going over 32 gets rid of the error. As for the second error, I don't know what is considered wrong because as my string is closed by a '\0' it shouldn't be happening. The weirdest is that this error is only on macos (same code has been compiled with gcc on a linux machine and put through valgrind without any errors), which could suggest that the error comes from either clang or the macos version of valgrind. Hence I don't think supressions will help at all, because the error will fire for all objects with a size less than 32, in all c programs.