search

the-benchmarker / web-frameworks

Which is the fastest web framework?
MIT License
6.98k stars 665 forks source link

Add resources consumption #31

Open schovi opened 7 years ago

schovi commented 7 years ago

One library can have an awesome performance for the cost of ten times bigger CPU consumption than other which has like 2 times worse performance. I think it is important metrics.

Metrics can be normalized to something like "10 requests per 1%CPU or 1MB Ram" (just idea :)

tbrand commented 7 years ago

Ok, let's think about this. Is there any elegant way to measure them?

schovi commented 7 years ago

@tbrand It is few days I see someone use Docker to measure the performance of something. Maybe that could be the way.

BTW: Using Docker could make this easier to run and integrate for more people. Have you ever consider to use Docker there?

tbrand commented 7 years ago

That's good approach:+1: If you have a specific reference, please let me know.

I'm thinking using docker, give me a time to do that.:smile:

schovi commented 7 years ago

@tbrand I tried to do something fast there is running Docker for crystal and rails. https://github.com/tbrand/which_is_the_fastest/pull/33

brightball commented 7 years ago

From a consistency standpoint, you could grab a snapshot of system load and ram usage at the 3/4 mark of the test. That would give you a fairly consistent "under stress" measurement that could be shown as a percentage of the whole.

OvermindDL1 commented 7 years ago

Or snapshots at certain intervals as well.

tbrand commented 7 years ago

Sounds pretty good:+1:

tbrand commented 7 years ago

This might be good? https://docs.docker.com/engine/reference/commandline/stats/#examples

OvermindDL1 commented 7 years ago

This might be good? https://docs.docker.com/engine/reference/commandline/stats/#examples

Ooo, wonder when that was added, looks like it may work well!

kalicki commented 6 years ago

Close?!

waghanza commented 6 years ago

Using Docker could have several advantages, and one of them could be getting consumption, #123

kokizzu commented 4 years ago

I use standard time to measure most things:

alias time='/usr/bin/time -f "\nCPU: %U s\tReal: %e s\tRAM: %M KB"'
waghanza commented 4 years ago

in what is that helpful ?

kokizzu commented 4 years ago

in measuring the max RAM usage '__')

waghanza commented 4 years ago

ah, I think this measure will come from the cloud provider we use ... but thanks for the tip :heart:

Tienisto commented 10 months ago

This would be great if we also have RAM metrics like avg, min, max.

You could take a snapshot of docker stats every sec of the docker container I guess. I don't know if there is a simpler solution for this.