cli-2.3.3 Throws Exceptions on test case assert resulting in no .rcs file

[GoogleCodeExporter]

What steps will reproduce the problem?

1. Create command with a test case that has an assert that will trigger (a test 
case that fails).
2. Run the test with the cli-2.3.3 jar file.
3. Exception is thrown, no .rcs file is produced.

What is the expected output? What do you see instead?

- If all asserts don't trigger you normally get a file matching the name of 
your .rcq file but with .rcs file extension. No matter what assert is thrown, 
it throws an Exception (see attached) and no .rcs file is produced. Running the 
same test in the ui-2.3.3 GUI produces the additional information you'd expect 
to find in the missing .rcs file, including the line number and error message 
of any asserts triggered.

What version of the product are you using? On what operating system?

cli-2.3.3 on Linux (Fedora) using java 1.6U14 also Windows XP Pro SP2 using 
Java 1.6U30. It looks like RESTClient cli-2.3.3 was built with 1.6U20.

Please provide any additional information below.

I've attached 2 files, one a simple rcq file with a simp,le test that actually 
passes, and the other is the captured result from stdout/sdterr which has the 
exception stacktrace. Note that in this example rnning it with java 1.6U30 on 
the PC, no matter what is in the test I get the same result (see result.txt 
attached). In the Linux environment, and in the UI in all environments, this 
test file works, even using the CLI because the assert does not trigger. But if 
you change 200 in the test to 404, it triggers the assert, which works fine in 
the UI but the CLI throws a different exception, see result2.txt.

Below is the command I used on the PC using Java 1.6U30; -o test tells it to 
put the .rcs files ina  local subdirectory named test, but there are no files 
in there after this runs):

java -jar restclient-cli-2.3.3-jar-with-dependencies.jar -o test 
test-get-google.rcq >result.txt 2>&1

Original issue reported on code.google.com by sydd.so...@gmail.com on 7 Feb 2012 at 10:47

Attachments:

• test-get-google.rcq
• result.txt
• result2.txt

[GoogleCodeExporter]

I have since developed another test case that works fine in the UI but always 
throws the exception indicated in attachment marked result2.txt using the CLI. 
I'm attaching the test which validates the XML returned in the body of the 
response against a specified XML Schema. This is a very useful test I'd like to 
use in the automated tests I run using the CLI but that all result in the 
exception being thrown so I can only run them by hand using the UI jar file. It 
should be noted that all my testing is on a network not connected to the 
internet so if there are typos in the attachments it's because i had to type 
them in from printouts. But you should get the idea. - Sydd

P.S. I have lots of examples of .rcq files with test scripts I could contribute 
as the examples so far have been quite sparse.

Original comment by sydd.so...@gmail.com on 16 Mar 2012 at 5:53

Attachments:

• testValidateBodyAgainstXSD.java

[GoogleCodeExporter]

To clarify, unlike the other tests where the NullPointerException in 
result2.txt is only thrown when an assert is triggered (i.e. the test fails), 
the above testValidateBodyAgainstXSD.java test causes the NullPointerException 
regardless of whether the test results in the assert or fail in it is called or 
not. I.e. it ALWAYS throws NullPointerExecption using the CLI jar file but 
works just fine using the UI jar file. 

Original comment by sydd.so...@gmail.com on 16 Mar 2012 at 5:59

[GoogleCodeExporter]

Checking...

Original comment by subwiz on 26 Mar 2012 at 4:29

• Changed state: Accepted

[GoogleCodeExporter]

Thanks for reporting the bug in such detail :-) This is an already identified 
and fixed bug. Its been fixed, but not part of any release. Please get this 
pre-release Jar to test it:

http://dl.dropbox.com/u/2505020/restclient-cli-2.4-SNAPSHOT-jar-with-dependencie
s.jar

Original comment by subwiz on 26 Mar 2012 at 5:52

• Changed state: Fixed

[GoogleCodeExporter]

Thanks that seems to have fixed it. One more question:

In the testValidateBodyAgainstXSD.java attachment I posted above, there’s a 
line that looks like this:

<code>java.io.File schemaLocation = new Java.io.File("MyXMLSchema.xsd"); // on 
linux this ends up being in ~/Documents</code>

When I run it with the 2.3.3 UI or CLI jar file, because I did not provide a 
path for the file, it looks for it in ~/Documents on a Linux platform.
When I run the same test using the CLI 2.4-SNAPSHOT jar file you sent me, it 
now expects to find the file in the local directory rather than ~/Documents. 

Was this changed in response to another bug?

And when you release both as 2.4, will the UI still default to ~/Documents? If 
not, where?

Original comment by sydd.so...@gmail.com on 29 Mar 2012 at 6:50

[GoogleCodeExporter]

Note that cli and ui makes use of the same code: so the behavior is going to be 
the same in both. My recommendation is to use full path instead of relative 
path. When you give relative filename, the file is created in the folder 
represented by user.dir system property [Usage: 
System.getProperty("user.dir");]. So, if you run the UI app in some other 
directory, and cli app in some other directory, the results are bound to 
differ. As said earlier, my strong suggestion would be to use full path.

Original comment by subwiz on 30 Mar 2012 at 4:20

[GoogleCodeExporter]

Thanks, and sorry to have bothered you on this. We purposefully don't use full 
paths because multiple people may run the tests, or even eventually our 
automated continuous integration tool, after having checked things out from 
Subversion. So we can't guarantee the location to which the developer checks 
things out nor that they are even running it on the same type of platform (i.e. 
Linux, PC, etc.). Understanding that it's simply using the user.dir property is 
all I needed to know and I should have known that: it's not your code doing it, 
it's the java.io.File class. At some point we'll probably serve the XSD file 
from a web site since the SchemaFactory expects a URL anyway, which makes this 
all moot.

Thanks to this fix I now have a few hundred .rcs scripts that test a REST 
interface including running validation against the XMLSchema. Right now for 
Linux we also have a "runRESTtests" shell script that automates the test runs, 
and can even do limited test scenarios by running a sequence of .rcs files in a 
row (achieved by simple file naming convention: 
test-Scenario001-test01-YadaYada.rcs, test-Scenario001-test02-YadaYoda.rcs, 
etc. and just running them in alphabetical order).

Original comment by sydd.so...@gmail.com on 30 Mar 2012 at 8:11

[GoogleCodeExporter]

> Thanks to this fix I now have a few hundred .rcs scripts that test a REST 
interface including running validation against the XMLSchema.

Really impressive :-)

Original comment by subwiz on 31 Mar 2012 at 5:49