search

sbt / sbt-remote-control

Create and manage sbt process using unicorns and forks
Other
74 stars 14 forks source link

Various improvements to the JSON code and test reporting bits #187

Closed dpratt closed 10 years ago

havocp commented 10 years ago

Thanks! Maybe useful to you,

Some thoughts on test reporting here: https://groups.google.com/d/msg/sbt-dev/vPhEH_RyZYY/_AQw2FKgvZoJ

Also we have a sample project here which uses 4 of the 5 test frameworks (not specs 1 at the moment I think): https://github.com/typesafehub/activator-multi-test-framework we were using this to see how they each report things.

We could really use some help on how to overhaul the test stuff, we've done some initial investigation but I think won't get to really fixing it for a while.

havocp commented 10 years ago

Good improvements I think, nice json cleanups in particular.

jsuereth commented 10 years ago

I think we merged some things which conflict with this. LIke the cleanups, would love a rebase so we can try to get this in before too much divergence happens again :)

dpratt commented 10 years ago

I've rebased and also added some bits that start to help with the test logging stuff. With respect to that, I don't know if there's much sbt-remote-control can do. By and large, TestEvents are up to the individual test frameworks to generate, and it looks like Specs2 doesn't really return much info outside of the fact that a test failed - there's not even really a way to associate a failure event with a semantic name. It looks like they rely mainly on the console output to communicate feedback to the user.

dpratt commented 10 years ago

Do you guys have a list for sbtrc? I've built an internal (for now) plugin for TeamCity that runs SBT jobs and also collects build metadata to be used for triggered execution of other builds with SNAPSHOT dependencies. I've also done a bit of (internal for now) work on the SBT support in IDEA, but I've sort of hit a wall - the TeamCity thing hooks into SBT in the same way that sbtrc does, and it works really well, but it relies on adding a plugin to the build and also some launcher magic. I'd really prefer to have our CI server use a running SBT instance as close to stock as possible. I've preliminarily got most of it up and working with sbtrc, but the genesis of this pull request was that I was running into unexpected runtime exceptions due to the use of JsValue.as instead of proper play-json combinators. After taking a look at stuff, I figured anything worth doing is worth doing right.

I'd love to help out more where I can - I've spent the past few months transitioning my company away from maven using a custom packaging scheme that I whipped up a year or so ago to SBT for most of our projects. So far, I'm loving it.

I'm likely going to take one of the recent releases of sbtrc and see what I can do with both TeamCity and IDEA - do you guys have plans to do an 'official' 0.1.0 release (or similar) any time soon? If not, I can just use the most recent SHA versioned one from the typesafe repo, and keep upgrading as they come out.

havocp commented 10 years ago

we are using the sbt-dev list since all of sbt-rc will probably roll into sbt proper once we are feeling the ABI is ready to lock down. The stuff you're working on sounds great and we love PRs like this one!

I can make "official" releases anytime if you need one. For our own use I guess we've just been using the releases with git hashes. I can post those whenever even and then we don't have to figure out what version we are on :-) once things are stable we'll just roll into the main sbt release anyhow.

as you saw in that email I linked to I think we have to extend the test frameworks to give us more information. For now we just do what we can I guess.

dpratt commented 10 years ago

So removing a comment caused a build failure. I'm stumped.

havocp commented 10 years ago

sometimes Travis just runs out of memory or something. I restarted the build and maybe we'll have better luck.

havocp commented 10 years ago

I think there's a race in one of the tests:

[info] Looks like client has closed!
[info] Not executing runnable because we only run one thing: Runnable(onError(reconnecting=false, Connection closed))
[info] 0: Client has not closed yet, so sleeping until death (1 seconds).

I'll just merge this and then we can debug that separately. We know Travis passed before you deleted the two comments ;-)