Closed GoogleCodeExporter closed 9 years ago
Do you have a code snippet that reproduces the problem? Also, be more
specific: what version of os x you're using, what compiler you're using, and so
forth. And show the *exact* compile and link commands you used to build the
problem binary.
There could be several things wrong. One is that the strdup is being called by
a global constructor before libtcmalloc is loaded. Another is that the code is
compiled with a linker option that affects how binding is done. There may be
others as well.
Original comment by csilv...@gmail.com
on 5 Nov 2010 at 8:45
See also issue 108. Are you using dlopen?
Original comment by csilv...@gmail.com
on 5 Nov 2010 at 8:46
The test case is very plain, no dlopen and no special link options.
code tst.C:
#include <string.h>
#include <stdlib.h>
main() {
char *d = strdup("abc");
free(d);
}
compile command:
g++ -o tst tst.C -L/opt/local/lib -ltcmalloc_minimal
output:
src/tcmalloc.cc:372] Attempt to free invalid pointer: 0x100190
Abort trap
linked libs:
MacBook:app yurivict$ otool -L tst
tst:
/opt/local/lib/libtcmalloc_minimal.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.0)
OS version:
Darwin MacBook.local 10.4.0 Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53
PDT 2010; root:xnu-1504.7.4~1/RELEASE_I386 i386
Machine:
MacBook Pro
Original comment by yuriv...@yahoo.com
on 5 Nov 2010 at 9:41
perftools is installed from macports:
sudo port install google-perftools
Original comment by yuriv...@yahoo.com
on 5 Nov 2010 at 9:42
I see that libtool adds in -Wl,-bind_at_load to the link flags that it uses
when running the perftools unittest:
g++ -o tst tst.C -L/opt/local/lib -ltcmalloc_minimal -Wl,-bind_at_load
Does this fix the problem for you?
This isn't anything that I added, so it must be a known Best Practice (at least
in some circumstances) for building on OS X.
Original comment by csilv...@gmail.com
on 9 Nov 2010 at 4:15
Any more word on this?
Original comment by csilv...@gmail.com
on 26 Nov 2010 at 12:00
No, for some reason -Wl,-bind_at_load makes no difference in this testcase.
Original comment by y...@tsoft.com
on 25 Dec 2010 at 9:30
Looks like dynamic libraries are bound "hard" to whatever they are linked to at
compile time and its impossible to change this on Apple.
Unless some workaround is found, this limits applicability of perftools to some
toy cases where no cross-library allocation takes place, and only to particular
apps/libraries that are explicitly linked with tcmalloc. This example
(strdup/free) just illustrates the general problem.
Original comment by y...@tsoft.com
on 26 Dec 2010 at 6:02
Apparently google-perftools approach isn't considered to be natural on Mac OS.
Here is the relevant information how such things are done in recommended by
Apple way:
http://developer.apple.com/library/mac/#documentation/Performance/Conceptual/Man
agingMemory/Articles/MallocDebug.html
http://guiheneuf.org/mach%20inject%20for%20intel.html
But, with the limitations of no cross-allocation (alloc in one module and free
in another not linked with tcmalloc) and explicit linking with libtcmalloc for
all modules in question, perftools still should be useful on Apple.
Original comment by y...@tsoft.com
on 26 Dec 2010 at 9:48
Ok, that's for that report.
I'm still hoping there's *some* sort of workaround, however. It would be
terrible not to be able to use strdup() with tcmalloc, or other libc routines
that allocate memory that the user is responsible for freeing.
Do you have a reference that describes how dylib's are bound "hard" to whatever
they are linked to at compile time, and that it's impossible to change this?
At the least, I can provide a link to that in the README.
Original comment by csilv...@gmail.com
on 5 Jan 2011 at 8:30
Actually the right thing to do on Apple to avoid the problem is to do the same
that Apple's Guard Malloc is doing. Their guard Malloc allows some degree of
memory debugging and its user interface is described here:
http://developer.apple.com/library/mac/#documentation/Performance/Conceptual/Man
agingMemory/Articles/MallocDebug.html
But it looks like Guard Malloc is closed source and I wasn't able to find the
documentation on its system side interface. But its definitely different and
more complex than that of google malloc due to this "hard" binding. I guess
Apple left some back door just for that: memory debugging.
Another supposed workaround is to use the environment variable
DYLD_FORCE_FLAT_NAMESPACE=1. It was able to fix the simple testcase with
strdup/free. But it failed in the situation when libtcmalloc is dynamically
loaded. Maybe DYLD_FORCE_FLAT_NAMESPACE is an acceptable workaround in most
situations.
I will post more links when I find them.
Original comment by y...@tsoft.com
on 6 Jan 2011 at 9:54
If I understand you right, the main problem you're seeing is tcmalloc trying to
free memory allocated with the system alloc? I think this is another vote in
favor of implementing 'try system free if tcmalloc free' fails, in tcmalloc.cc.
This is possible: we can detect if we allocated the memory being passed to
free, and if not, we can just pass it to the system alloc. Do you happen to
know if the system malloc is available via an alias (similar to __libc_malloc
for glibc malloc?)
Original comment by csilv...@gmail.com
on 7 Jan 2011 at 11:20
For right now, I'll just update the documentation. I see that libgmalloc used
to require DYLD_FORCE_FLAT_NAMESPACE as well:
http://lists.apple.com/archives/xcode-users/2004/Dec/msg00022.html
so this seems like it's not a surprising requirement. I also see that
libgmalloc doesn't require that envvar any more, so perhaps the problem has
been fixed in more recent versions of the OS. Unfortunately, I don't know much
about OS X so can't really cover this issue very well.
Original comment by csilv...@gmail.com
on 10 Jan 2011 at 2:11
No, its not that the problem was fixed in OS.
Guard Malloc used to have the same interface as perftools has (overloading of
malloc interface) and otherwise would have been breaking on strdup just like we
see it with perftools here.
Later they switched the the new interface: DYLD interposing and therefore they
don't require DYLD_FORCE_FLAT_NAMESPACE any more.
Original comment by y...@tsoft.com
on 15 Jan 2011 at 1:09
I updated the README for perftools 1.7, just released. I will try to fix this
'for reals' for the next release, possibly using malloc_default_zone.
Original comment by csilv...@gmail.com
on 5 Feb 2011 at 12:22
I will try to add to perftools the same interface that Guard Malloc uses so
that it can be just loaded into process like in linux without causing such
issues like this strdup one.
Original comment by y...@tsoft.com
on 20 Feb 2011 at 12:19
Thanks, we've been looking into the guard-malloc technique, but it requires
using DYLD_INTERPOSE_LIBRARY which isn't as easy to use as we'd like. We think
we can get a good solution using malloc_default_zone, though I haven't had time
to play around with it yet. If that doesn't work, something using
INTERPOSE_LIBRARY is probably our backup plan.
Original comment by csilv...@gmail.com
on 21 Feb 2011 at 12:22
Their usage model is this:
set DYLD_INSERT_LIBRARIES to /usr/lib/libgmalloc.dylib
What bothers me is this: what if I also need to dynamically load malloc library
in addition to this. Which one should I load, original one and it will be
magically interposed or the new one?
Original comment by y...@tsoft.com
on 21 Feb 2011 at 4:05
I don't know the answer, but I'm hoping it will be a moot point if we can use
malloc_default_zone instead of relying on DYLD_INSERT_LIBRARIES.
Original comment by csilv...@gmail.com
on 23 Feb 2011 at 2:24
Original comment by csilv...@gmail.com
on 8 Jul 2011 at 12:22
OS X support is entirely revamped in perftools 1.8, just released. They use
malloc-zones, and should totally avoid this problem. *knock on wood*
Original comment by csilv...@gmail.com
on 16 Jul 2011 at 1:19
Original issue reported on code.google.com by
yuriv...@yahoo.com
on 5 Nov 2010 at 6:14