search

puniverse / comsat

Fibers and actors for web development
docs.paralleluniverse.co/comsat
Other
598 stars 103 forks source link

TechEmpower Benchmarks #23

Open synapsechris opened 9 years ago

synapsechris commented 9 years ago

https://www.techempower.com/benchmarks/

circlespainter commented 9 years ago

Thanks for pointing this out!

circlespainter commented 9 years ago

Ongoing in https://github.com/circlespainter/FrameworkBenchmarks (comsat-webactors and comsat-servlet at present).

circlespainter commented 9 years ago

I'd say it's ready for PRs but I'd like to understand better a few more things about the Framework Benchmarks; no feedback so far via https://groups.google.com/forum/#!topic/framework-benchmarks/Gujdipuua7Q though.

@synapsechris maybe you can help answering those questions?

circlespainter commented 9 years ago
mrserverless commented 9 years ago

@circlespainter nice work on the framework and it's great to see you've added tests cases into TechEmpower.

I've been trying to do some testing locally and noticed that the comsat version needs to be upgraded to 0.5.0 to avoid compilation issue with 0.5.0-SNAPSHOT which doesn't exist anymore. I'll try to submit a PR, although the tests are in a branch which might be a bit complicated.

circlespainter commented 9 years ago

@yunspace Thanks, I was planning to update the PRs to 0.5.0 stable once released so I'll do that shortly.

mrserverless commented 9 years ago

hi @circlespainter I tried to run the tests again using your latest commits:

toolset/run-tests.py --install server --mode verify --test comsat-servlet
toolset/run-tests.py --install server --mode verify --test comsat-webactors

I think for comsat-webactors, the setup-undertow.sh should be:

fw_depends java8 maven

Otherwise there'll be compilation errors. Afterward fixing that I got the following error:

Setup comsat-servlet: WARNING: Error injecting: capsule.CapsuleBuildMojo
Setup comsat-servlet: com.google.inject.ProvisionException: Guice provision errors:
Setup comsat-servlet: 
Setup comsat-servlet: 1) No implementation for org.eclipse.aether.RepositorySystem was bound.
Setup comsat-servlet:   while locating capsule.CapsuleBuildMojo
Setup comsat-servlet:

Which seem to be caused by the default mvn being 3.0.5 and not the 3.1.0 required by capsule-maven-plugin: https://github.com/chrischristo/capsule-maven-plugin/issues/4

Were you able run these tests locally in deployment/vagrant-develop? The PR actually passed Travis CI which leads me to suspect the CI server setup is different to what in vagrant-develop. :(

mrserverless commented 9 years ago

btw if you haven't tried running the benchmark locally, the steps I've been following steps are:

cd FrameworkBenchmarks/deployment/vagrant-development
vagrant up
vagrant ssh

toolset/run-tests.py --install server --mode verify --test comsat-servlet
toolset/run-tests.py --install server --mode verify --test comsat-webactor

rm -rf results && toolset/run-tests.py --test comsat-servlet
rm -rf results && toolset/run-tests.py --test comsat-webactors

as per documentation here http://frameworkbenchmarks.readthedocs.org/en/latest/Development/Installation-Guide/#vagrant-development-environment

circlespainter commented 9 years ago

Thanks for looking into it; yes bringing it up-to-date took more than I expected and indeed I'll need to re-run a test in the Vagrant environment as well. I'll get this sorted out by the end of the week.

circlespainter commented 9 years ago

@yunspace Ok, now both run fine in my Vagrant env but they fail on CI; it looks like the java8 dependency can't be setup for some reason:

'ERROR: /toolset/setup/linux/languages/java8.sh: Command '\''sudo update-alternatives --set java $J7_HOME/jre/bin/java'\'' exited with status 2 (dependency=java8) (cwd=$FWROOT/installs)'
Setup comsat-webactors-servlet-tomcat: ERROR: /toolset/setup/linux/languages/java8.sh: Command 'sudo update-alternatives --set java $J7_HOME/jre/bin/java' exited with status 2 (dependency=java8) (cwd=$FWROOT/installs)

Is there anything in the framework's build source that could cause this (apart from the dependency to java8 - which is there only because it's more performant, although the framework support java7 too)?

circlespainter commented 8 years ago

@yunspace Just tried with the most recent ubuntu/trusty64 box in the Vagrant development env and everything seems to run fine, so not really sure where that CI issue comes from.

mrserverless commented 8 years ago

I had some issues updating to java8 in the dropwizard benchmark also previously where the dependency just wasn't getting installed. I will double check how I worked around it and get back to you.

On Mon, Sep 21, 2015 at 1:20 AM, Fabio Tudone notifications@github.com wrote:

@yunspace https://github.com/yunspace Just tried with the most recent ubuntu/trusty64 box in the Vagrant development env and everything seems to run fine, so not really sure where that CI issue comes from.

— Reply to this email directly or view it on GitHub https://github.com/puniverse/comsat/issues/23#issuecomment-141796415.

circlespainter commented 8 years ago

The servlet framework has been merged, still having a strange failure on CI on the webactors one (on a fresh vagrant-development still works), https://github.com/TechEmpower/FrameworkBenchmarks/pull/1719.

circlespainter commented 8 years ago

Update: https://github.com/TechEmpower/FrameworkBenchmarks/pull/1719#issuecomment-159963564

circlespainter commented 8 years ago

Update: https://github.com/TechEmpower/FrameworkBenchmarks/pull/1719#issuecomment-164388683

circlespainter commented 8 years ago

The web-only tests of comsat-servlet and comsat-webactors (both with configurations for all servers/backends) have been merged by TechEmpower.