search

eminence / udt-rs

UDT bindings for Rust
BSD 2-Clause "Simplified" License
32 stars 12 forks source link

memory consumption #5

Closed mcginty closed 8 years ago

mcginty commented 8 years ago

I noticed that shoop was using a lot of CPU even in relatively low-bandwidth situations, so I ran it through valgrind and Instruments. I definitely do some excess allocations in shoop, but it looks like UDT has a lot of allocation and memory thrash. My first guess is just that recvmsg allocates a new Vec for every call instead of accepting a &mut [u8] argument.

Vangrind heap summary for a shoop transfer of a 5.7MB file.

==51911== HEAP SUMMARY:
==51911==     in use at exit: 34,307 bytes in 248 blocks
==51911==   total heap usage: 17,127 allocs, 16,879 frees, 5,647,010,936 bytes allocated

Instruments profiling info:

screen shot 2016-08-08 at 3 43 59 pm

Again, the valgrind info especially is probably not that useful because I think I'm doing a lot of excess allocation in shoop too, but hopefully the profiler info is useful.

mcginty commented 8 years ago

Sorry, that profile screenie was with the system allocator. I did it again with rust's jemalloc, it seems to lead to a more clear profile in this case at least.

screen shot 2016-08-08 at 3 51 49 pm
eminence commented 8 years ago

Have you updated to udt-rs 0.2.0 yet? I'm curious if things are any different now that we're using the new recv API you suggested

mcginty commented 8 years ago

Yeah I have - this bug report was pre-0.2. I haven't run much extensive memory analysis on shoop after 0.2 but I know it's much lower. I think the biggest problem is now sodiumoxide doing the same thing, where it's allocating new Vecs with every open/seal operation. I'm going to close this.