Unit test failures in com.github.jmchilton.blend4j.galaxy.IntegrationTest, LibrariesTest

[heuermh]

Hello,

I am seeing unit test failures related to 501 and 403 HTTP errors with TestGalaxyInstance (https://main.g2.bx.psu.edu/)

$ mvn install ... Running workflow in history 1db3a2621ee83293 Workflow Output ID bbd44e69cb8906b5b1b3268bfc864c57 Tests run: 14, Failures: 7, Errors: 0, Skipped: 0, Time elapsed: 13.684 sec <<< FAILURE!

Results :

Failed tests: testCreatePrivateDataLibrary(com.github.jmchilton.blend4j.galaxy.IntegrationTest): Expected 200 status code, got 501. testDirectoryLibraryUpload(com.github.jmchilton.blend4j.galaxy.IntegrationTest): A message body reader for Java class com.github.jmchilton.blend4j.galaxy.beans.Library, and Java type class com.github.jmchilton.blend4j.galaxy.beans.Library, and MIME media type text/plain; charset=utf8 was not found testFilesystemPathsLibraryUpload(com.github.jmchilton.blend4j.galaxy.IntegrationTest): A message body reader for Java class com.github.jmchilton.blend4j.galaxy.beans.Library, and Java type class com.github.jmchilton.blend4j.galaxy.beans.Library, and MIME media type text/plain; charset=utf8 was not found testLibraries(com.github.jmchilton.blend4j.galaxy.IntegrationTest): Expected 200 status code, got 403. 403 Forbidden testUsers(com.github.jmchilton.blend4j.galaxy.IntegrationTest): Expected 200 status code, got 501. testFileUpload(com.github.jmchilton.blend4j.galaxy.LibrariesTest): A message body reader for Java class com.github.jmchilton.blend4j.galaxy.beans.Library, and Java type class com.github.jmchilton.blend4j.galaxy.beans.Library, and MIME media type text/plain; charset=utf8 was not found testPathPaste(com.github.jmchilton.blend4j.galaxy.LibrariesTest): A message body reader for Java class com.github.jmchilton.blend4j.galaxy.beans.Library, and Java type class com.github.jmchilton.blend4j.galaxy.beans.Library, and MIME media type text/plain; charset=utf8 was not found

Tests run: 14, Failures: 7, Errors: 0, Skipped: 0

[heuermh]

Still having trouble with this, even after the recent changes.

Why is a unit test trying to run mercurial? Unit tests should test java code in isolation, without calling to external dependencies or resources.

Functional and/or integration tests may have external dependencies, but then those usually don't fail the build if they fail.

$ mvn install
...
Running TestSuite
Configuring TestNG with: org.apache.maven.surefire.testng.conf.TestNG652Configurator@4f016505
Tests run: 35, Failures: 1, Errors: 0, Skipped: 34, Time elapsed: 0.986 sec <<< FAILURE!
bootStrapGalaxy(com.github.jmchilton.blend4j.galaxy.TestGalaxyInstance)  Time elapsed: 0.545 sec  <<< FAILURE!
java.lang.RuntimeException: java.io.IOException: Cannot run program "hg": error=2, No such file or directory
    at com.github.jmchilton.galaxybootstrap.IoUtils.executeAndWait(IoUtils.java:79)
    at com.github.jmchilton.galaxybootstrap.IoUtils.executeAndWait(IoUtils.java:86)
    at com.github.jmchilton.galaxybootstrap.BootStrapper.setupGalaxy(BootStrapper.java:112)
    at com.github.jmchilton.blend4j.galaxy.TestGalaxyInstance.bootStrapGalaxy(TestGalaxyInstance.java:23)
Caused by: java.io.IOException: Cannot run program "hg": error=2, No such file or directory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1042)
    at com.github.jmchilton.galaxybootstrap.IoUtils.executeAndWait(IoUtils.java:72)
    ... 31 more
Caused by: java.io.IOException: error=2, No such file or directory
    at java.lang.UNIXProcess.forkAndExec(Native Method)
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:135)
    at java.lang.ProcessImpl.start(ProcessImpl.java:130)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1023)
    ... 32 more

[jmchilton]

Okay it looks like you don't have mercurial installed. It should be able to fallback to direct downloads, I'll try to find time to add this.

[heuermh]

Thanks, John.

My point is more that $ mvn install shouldn't be calling out to the network, say if I'm working on an airplane or at a client site that doesn't allow hg traffic through the firewall.

I need to spend more time with this codebase before I can make helpful suggestions, but perhaps BootStrapper or GalaxyDaemon in galaxy-bootstrap could be made an interface which could then be mocked out in unit tests in blend4j. I use Mockito and PowerMock for this

http://code.google.com/p/mockito/ https://code.google.com/p/powermock/

Having a full integration test suite that actually bootstraps a Galaxy instance is very useful, I just don't think it should be part of the normal build.

[jmchilton]

Opps sorry, I hadn't seen your full comment on my phone interface. I understand what you are saying about this not being good behavior for the normal build process. Is this http://maven.apache.org/surefire/maven-surefire-plugin/examples/skipping-test.html the right way to disable it?

Stepping back a little, I think blend4j is providing a small interface to an external service (Galaxy), so I think integration tests are much more valuable than unit tests. The things most likely to go wrong are behavior changes to Galaxy over time or incompatibilities between what blend4j is asking for and what Galaxy does - mocks are just not going to catch this sort of thing.