search

snapframework / snap-website

Snapframework.com source code
http://snapframework.com/
BSD 3-Clause "New" or "Revised" License
77 stars 26 forks source link

Benchmarks need to be updated #43

Open ocharles opened 9 years ago

ocharles commented 9 years ago

https://twitter.com/ndm_haskell/status/665969896027758596:

"Visited http://snapframework.com/benchmarks , "The recent release of GHC 7.0.1..." - while checking out the alternatives before recommending #yesod."

I am tired of this passive aggression - but we should update our benchmarks.

gregorycollins commented 9 years ago

I'd appreciate it if someone else could get this done for Snap 1.0 -- I have a lot of other tasks to do here. I know imalsogreg from IRC was also working on this recently.

imalsogreg commented 9 years ago

I'm theoretically working on this - but at an extremely slow pace - on this branch of the snap-benchmarks repo: https://github.com/snapframework/snap-benchmarks/tree/runtests

This probably is over-engineered but I started trying to automate the test running and parameter-picking since I got strange numbers out and didn't have a sense of how to design experiments properly, so I was just haphazardly choosing values (heap size, number of connection to accept at once, etc). I hope that with a little test-running script I can take the subjectivity and tedium out of the process, and we can easily do the whole procedure on a proper test machine.

Maybe it would be a good idea to change course and simply rerun the exact same commands used here: https://github.com/snapframework/snap-benchmarks/blob/51b1c5b24c09ced61e937d7d5b1d88f8e513663b/results.txt

gregorycollins commented 9 years ago

Httperf sucks and there are better alternatives now, from what I understand -- don't ask me what they are though!

RTS tuning parameters for the running Haskell binaries are what are most important -- we are especially sensitive to the values of -A and -N.

imalsogreg commented 9 years ago

@hvr mentioned his project https://github.com/hvr/uhttpc a while back. Shall we try that?