TechEmpower Benchmarks

[synapsechris]

https://www.techempower.com/benchmarks/

[circlespainter]

Thanks for pointing this out!

[circlespainter]

Ongoing in https://github.com/circlespainter/FrameworkBenchmarks (comsat-webactors and comsat-servlet at present).

[circlespainter]

I'd say it's ready for PRs but I'd like to understand better a few more things about the Framework Benchmarks; no feedback so far via https://groups.google.com/forum/#!topic/framework-benchmarks/Gujdipuua7Q though.

@synapsechris maybe you can help answering those questions?

[circlespainter]

• https://github.com/TechEmpower/FrameworkBenchmarks/pull/1719
• https://github.com/TechEmpower/FrameworkBenchmarks/pull/1720

[mrserverless]

@circlespainter nice work on the framework and it's great to see you've added tests cases into TechEmpower.

I've been trying to do some testing locally and noticed that the comsat version needs to be upgraded to 0.5.0 to avoid compilation issue with 0.5.0-SNAPSHOT which doesn't exist anymore. I'll try to submit a PR, although the tests are in a branch which might be a bit complicated.

[circlespainter]

@yunspace Thanks, I was planning to update the PRs to 0.5.0 stable once released so I'll do that shortly.

[mrserverless]

hi @circlespainter I tried to run the tests again using your latest commits:

toolset/run-tests.py --install server --mode verify --test comsat-servlet
toolset/run-tests.py --install server --mode verify --test comsat-webactors

I think for comsat-webactors, the setup-undertow.sh should be:

fw_depends java8 maven

Otherwise there'll be compilation errors. Afterward fixing that I got the following error:

Setup comsat-servlet: WARNING: Error injecting: capsule.CapsuleBuildMojo
Setup comsat-servlet: com.google.inject.ProvisionException: Guice provision errors:
Setup comsat-servlet: 
Setup comsat-servlet: 1) No implementation for org.eclipse.aether.RepositorySystem was bound.
Setup comsat-servlet:   while locating capsule.CapsuleBuildMojo
Setup comsat-servlet: 

Which seem to be caused by the default mvn being 3.0.5 and not the 3.1.0 required by capsule-maven-plugin: https://github.com/chrischristo/capsule-maven-plugin/issues/4

Were you able run these tests locally in deployment/vagrant-develop? The PR actually passed Travis CI which leads me to suspect the CI server setup is different to what in vagrant-develop. :(

[mrserverless]

btw if you haven't tried running the benchmark locally, the steps I've been following steps are:

cd FrameworkBenchmarks/deployment/vagrant-development
vagrant up
vagrant ssh

toolset/run-tests.py --install server --mode verify --test comsat-servlet
toolset/run-tests.py --install server --mode verify --test comsat-webactor

rm -rf results && toolset/run-tests.py --test comsat-servlet
rm -rf results && toolset/run-tests.py --test comsat-webactors

as per documentation here http://frameworkbenchmarks.readthedocs.org/en/latest/Development/Installation-Guide/#vagrant-development-environment

[circlespainter]

Thanks for looking into it; yes bringing it up-to-date took more than I expected and indeed I'll need to re-run a test in the Vagrant environment as well. I'll get this sorted out by the end of the week.

[circlespainter]

@yunspace Ok, now both run fine in my Vagrant env but they fail on CI; it looks like the java8 dependency can't be setup for some reason:

'ERROR: /toolset/setup/linux/languages/java8.sh: Command '\''sudo update-alternatives --set java $J7_HOME/jre/bin/java'\'' exited with status 2 (dependency=java8) (cwd=$FWROOT/installs)'
Setup comsat-webactors-servlet-tomcat: ERROR: /toolset/setup/linux/languages/java8.sh: Command 'sudo update-alternatives --set java $J7_HOME/jre/bin/java' exited with status 2 (dependency=java8) (cwd=$FWROOT/installs)

Is there anything in the framework's build source that could cause this (apart from the dependency to java8 - which is there only because it's more performant, although the framework support java7 too)?

[circlespainter]

@yunspace Just tried with the most recent ubuntu/trusty64 box in the Vagrant development env and everything seems to run fine, so not really sure where that CI issue comes from.

[mrserverless]

I had some issues updating to java8 in the dropwizard benchmark also previously where the dependency just wasn't getting installed. I will double check how I worked around it and get back to you.

On Mon, Sep 21, 2015 at 1:20 AM, Fabio Tudone notifications@github.com wrote:

@yunspace https://github.com/yunspace Just tried with the most recent ubuntu/trusty64 box in the Vagrant development env and everything seems to run fine, so not really sure where that CI issue comes from.

— Reply to this email directly or view it on GitHub https://github.com/puniverse/comsat/issues/23#issuecomment-141796415.

[circlespainter]

The servlet framework has been merged, still having a strange failure on CI on the webactors one (on a fresh vagrant-development still works), https://github.com/TechEmpower/FrameworkBenchmarks/pull/1719.

[circlespainter]

Update: https://github.com/TechEmpower/FrameworkBenchmarks/pull/1719#issuecomment-159963564

[circlespainter]

Update: https://github.com/TechEmpower/FrameworkBenchmarks/pull/1719#issuecomment-164388683

[circlespainter]

The web-only tests of comsat-servlet and comsat-webactors (both with configurations for all servers/backends) have been merged by TechEmpower.