setting malloc/free/realloc

[GoogleCodeExporter]

Hi,

Is it currently possible to specify memory allocation functions through code 
for pagespeed automatic? It would be very nice if I can let the traffic server 
port use the allocation routines that trafficserver provides. 

ps. we are planning to open source the port very soon, we will donate the code 
to apache. So you can expect it to turn up soon around here:  
https://github.com/apache/trafficserver/tree/master/plugins/experimental
:) 

Cheers,

Otto van der Schaaf

Original issue reported on code.google.com by osch...@gmail.com on 30 Jul 2012 at 9:05

[GoogleCodeExporter]

For the most part, we use operator new/delete/[]new/[]delete to allocate 
memory.  Overrides for those operators can be linked in as desired, though be 
aware that src/net/instaweb/util/mem_debug.cc already does that in debug-mode 
only.

There may be a few library routines that we call that use malloc/free/delete 
directly.  Is it essential that 100% of the allocations go through the 
trafficserver allocator?   Or would it still be beneficial if only 99% went 
through the standard x-ray system, with the rest leaking into malloc/free?

Original comment by jmara...@google.com on 7 Aug 2012 at 5:05

• Changed state: RequestClarification

[GoogleCodeExporter]

Hi.  I just checked 
https://github.com/apache/trafficserver/tree/master/plugins/experimental and 
still did not see your ATS plugin.  How is that going?

Do you have any clarification for my questions about your memory allocator 
requirements?

Original comment by jmara...@google.com on 5 Sep 2012 at 2:36

[GoogleCodeExporter]

- I just replied to your email regarding the plugin.  

I couldn't get your suggestion to work at the time, and haven't looked into it 
since; i have been distracted.
I guess 99% would be okay, but to i'd have to inquire some people better suited 
then me to judge which allocations would be most important to catch here. the 
nice thing about it, is that in turn traffic server allows choosing an 
allocator at configure time. For example jemalloc 
http://www.facebook.com/notes/facebook-engineering/scalable-memory-allocation-us
ing-jemalloc/480222803919

Original comment by osch...@gmail.com on 5 Sep 2012 at 9:52

[GoogleCodeExporter]

I think we should consider:

a) having a reasonable way for PSOL API users to set the allocator
b) overriding the allocator in the mod_pagespeed binary
c) overriding the allocator in the ngx_pagespeed build instructions

jemalloc is one possibility
tcmalloc is another.  I think we need to benchmark.

This may be associated with an apparent leak as discussed here: 
https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/mod-pag
espeed-discuss/NjGRuTRGPyo/TvcudyMdcjUJ
It's possible that this leak is due to fragmentation, and that switching to a 
better memory allocator will solve it.

It's also possible that we could use pooling more/better in the parser and that 
would also solve it.  Conversely we may be *causing* fragmentation issues with 
the pooling that we *are* doing.

It also may be associated with load-test performance on a pure HTML load as 
discussed here:
https://groups.google.com/forum/#!topic/mod-pagespeed-discuss/HyIBCaBC_Es

Original comment by jmara...@google.com on 23 Oct 2014 at 2:55

• Changed state: New

[GoogleCodeExporter]

jemalloc discussion on chromium: 
https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/EUqoIz2iFU4

Original comment by jmara...@google.com on 10 Dec 2014 at 3:53