iggy-rs / iggy

Iggy is the persistent message streaming platform written in Rust, supporting QUIC, TCP and HTTP transport protocols, capable of processing millions of messages per second.
https://iggy.rs
MIT License
1.84k stars 85 forks source link

Dump `iggy-bench` results to csv #229

Open hubcio opened 11 months ago

hubcio commented 11 months ago

Append them to given csv, and if possible generate excel file with relevant charts.

Molter23 commented 11 months ago

I would like to work on it.

BartoszCiesla commented 11 months ago

Go ahead @Molter23 , it's assigned to you now

seroze commented 5 months ago

@hubcio / @BartoszCiesla

  1. I think we can go with --bechmark_csv_file as argument name for csv file
  2. I assume only one line(info in the below snippet) will be written to this file depending on if it's send_benchmark/poll_benchmark/..etc. We can have something that reads this csv files and generates charts on top of this.
    fn display_settings(&self) {
        let total_messages = self.total_messages();
        let total_size_bytes = total_messages * self.args().message_size() as u64;
        info!(
                "\x1B[32mBenchmark: {}, total messages: {}, total size: {} bytes, {} streams, {} messages per batch, {} batches, {} bytes per message, {} producers\x1B[0m",
                self.kind(),
                total_messages,
                total_size_bytes,
                self.args().number_of_streams(),
                self.args().messages_per_batch(),
                self.args().message_batches(),
                self.args().message_size(),
                self.args().producers(),
            );
    }
BartoszCiesla commented 5 months ago

Yes, that's good idea. Basic use case would be to call bench multiple times so file would have to be updated with each run in order to have data from all the tests. Then charts could be created by importing to spreadsheet or using tools like GNUplot or other tools.