High memory usage for Rust bindings compared with goczmq

[ericho]

I started to use these bindings and I noticed that the memory consumption of Rust examples (as well as my code) are higher than the goczmq examples. For example, the helloworld_client example, in my system, consumes around 11MB of RES, while the same example of goczmq takes around 5MB.

Also, in my program, that just sends data with the PULL socket, the memory increased from 5MB (before sending any data to the network) to 19MB, just by adding the rust-bindings. I compile the examples in release mode with the same results.

I'm not sure if I'm doing something wrong or if this is an issue in Rust linking with other libraries. Anyway I hope you can give me some advice on how to solve this or how to provide more useful information :smile:

[rotty]

Could you provide a minimal example? As the zmq crate tries to be mostly a "zero cost abstraction" over the ZMQ C API, we should not see large overhead compared to the equivalent C code. There will be some inefficiency due to making the binding safe, but it should be negligible.

[ericho]

Sure @rotty

Using this python hello world server:

import time                                                                                                                                                                  
import zmq                                                                                                                                                                   

context = zmq.Context()                                                                                                                                                      
socket = context.socket(zmq.REP)                                                                                                                                             
socket.bind("tcp://*:5555")                                                                                                                                                  

while True:                                                                                                                                                                  
    #  Wait for next request from client                                                                                                                                     
    message = socket.recv()                                                                                                                                                  
    print("Received request: %s" % message)                                                                                                                                  

    #  Do some 'work'                                                                                                                                                        
    time.sleep(0.1)                                                                                                                                                          

    #  Send reply back to client                                                                                                                                             
    socket.send(b"World")    

I run two equivalente clients, in Rust and Go. This is the rust client.

fn main() {                                                                                                                                                                  
    println!("Connecting...");                                                                                                                                               

    let context = zmq::Context::new();                                                                                                                                       
    let requester = context.socket(zmq::REQ).unwrap();                                                                                                                       

    assert!(requester.connect("tcp://localhost:5555").is_ok());                                                                                                              

    let mut msg = zmq::Message::new().unwrap();                                                                                                                              

    for request_nbr in 0..1024 {                                                                                                                                             
        println!("Sending hello {}...", request_nbr);                                                                                                                        
        requester.send("Hello".as_bytes(), 0).unwrap();                                                                                                                      

        requester.recv(&mut msg, 0).unwrap();                                                                                                                                
        println!("Received world {}: {}", msg.as_str().unwrap(), request_nbr);                                                                                               
    }                                                                                                                                                                        
}                

And the Go client:

package main                                                                                                                                                                 

import (                                                                                                                                                                     
        "fmt"                                                                                                                                                                
        "gopkg.in/zeromq/goczmq.v4"                                                                                                                                          
)                                                                                                                                                                            

func main() {                                                                                                                                                                
        context, err := goczmq.NewReq("tcp://localhost:5555")                                                                                                                
        if err != nil {                                                                                                                                                      
                fmt.Println(err)                                                                                                                                             
        }                                                                                                                                                                    

        defer context.Destroy()                                                                                                                                              

        fmt.Printf("Connecting to hello world server…")                                                                                                                      

        for i := 0; i < 1024; i++ {                                                                                                                                           

                msg := fmt.Sprintf("Hello %d", i)                                                                                                                            

                payload := [][]byte{[]byte(msg)}                                                                                                                             
                context.SendMessage(payload)                                                                                                                                 

                println("Sending ", msg)                                                                                                                                     

                reply, _ := context.RecvMessage()                                                                                                                            
                println("Received ", string(reply[0]))                                                                                                                       
        }                                                                                                                                                                    
}         

So, 1024 messages are sent, the number of messages is just to have time to retrieve the memory usage. I get the Rss value using this command:

echo 0 $(awk '/Rss/ {print "+", $2}' /proc/`pidof zeromq`/smaps) | bc

The results of the execution are:

Language	Rss
Rust	12700
Go	8044

Not sure if it's useful but I attached the maps and smaps output for the two processes go_maps.txt go_smaps.txt rust_maps.txt rust_smaps.txt

Also, I'm not sure why yet, but I'm seeing today different numbers as the one I reported initially, however, there's still an interesting difference between Go and Rust.

[birkenfeld]

Tested here locally. Go 9072k, Rust 13296k, Rust with lto=true 9488k.

So it seems to get the lowest possible RSS, you should build with LTO. In any case, I don't think this is a rust-zmq issue.

[ericho]

I agree :smile: Thanks for the help.

[birkenfeld]

I also remembered that it could be due to the default allocator in Rust being jemalloc, which is quite liberal with the amount of memory it reserves. Switching to the sytem allocator (which is not stable yet at the moment) gives me only around 5048k with Rust (LTO or not).

[rotty]

I think it is clear by now that this is not an issue with rust-zmq, but instead it's just Rust using a bit more RSS than Go in the default configuration, caused by jemalloc and code size (as enabling LTO cuts down on the RSS number).