search

twitter / summingbird

Streaming MapReduce with Scalding and Storm
https://twitter.com/summingbird
Apache License 2.0
2.14k stars 267 forks source link

Fix tests #594

Closed MansurAshraf closed 9 years ago

MansurAshraf commented 9 years ago

Change travis config to make tests run in parallel. Increase heap to 2 gb and permgem to 512mb. Goog news is that storm-tests are finally passing. Bad news is that there is a legit test failure in summingbird-scalding-test and summingbird-batch build just hangs (even locally)

screen shot 2015-02-14 at 10 23 34 am

MansurAshraf commented 9 years ago

I have commented Two tests to get a green build for now. One of them is failing and other just hangs. Both of them are in batch/scalding layer

johnynek commented 9 years ago

If you're going to take this approach can you make a pull request based on this branch that re enables both? We need to get the both green.

Also, I wonder when they started failing? Are they broken locally with 0.6.0 which we just released?

On Saturday, February 14, 2015, Mansur Ashraf notifications@github.com wrote:

I have commented Two tests to get a green build for now. One of them is failing and other just hangs. Both of them are in batch/scalding layer

— Reply to this email directly or view it on GitHub https://github.com/twitter/summingbird/pull/594#issuecomment-74389375.

Oscar Boykin :: @posco :: http://twitter.com/posco

MansurAshraf commented 9 years ago

they are failing locally as well. Not sure when it started happening but I will enable them in this branch and then we can take a look at them

johnynek commented 9 years ago

It would be good not to move things into test that we don't need to because the test coverage tools can't handle testing moduleA in moduleB, so it lowers our reported test coverage. Here the ExpressionDag stuff and the Dependants tests don't need to be moved as far as I can tell. They are not abstract test harnesses, but instead concrete tests for code in core. Let's leave that (and anything similar) in core/src/test.

MansurAshraf commented 9 years ago

There was too much going on in this PR so I have split it into 3 different PRs

1) core-test separation PR https://github.com/twitter/summingbird/pull/593 2) parallel build with commented failing tests https://github.com/twitter/summingbird/pull/599 3) parallel build with failing tests https://github.com/twitter/summingbird/pull/598