rosinstall tests are fragile to ordering changes

[tfoote]

rosinstall/test/local/test_diff_functions_multi.py is failing 12 tests at the moment which mostly appear to be ordering issues.

For example:

======================================================================                                                                      
FAIL: Test status output for svn when run outside workspace                                                                                 
----------------------------------------------------------------------                                                                      Traceback (most recent call last):                                                                                                            File "/tmp/rosinstall/test/local/test_diff_functions_svn.py", line 203, in test_Rosinstall_status_svn_untracked                               self.assertEqual('?       clone/added-fs.txt\nA       clone/added.txt\nD       clone/deleted.txt\n!       clone/deleted-fs.txt\nM       clone/modified.txt\n', output)                                                                                                              AssertionError: u'?       clone/added-fs.txt\nA       clone/added.txt\nD       clone/deleted.txt [truncated]... != u'?       clone/added-fs.txt\nD       clone/deleted.txt\n!       clone/deleted-f [truncated]...                                                                        ?       clone/added-fs.txt                                                                                                                
- A       clone/added.txt                                                                                                                   
  D       clone/deleted.txt                                                                                                                 
  !       clone/deleted-fs.txt                                                                                                              
  M       clone/modified.txt                                                                                                                
+ A       clone/added.txt  

[tkruse]

will look into this error, tests passed fine for me when running all, will run individually.

On Mon, Oct 22, 2012 at 5:06 PM, Tully Foote notifications@github.comwrote:

rosinstall/test/local/test_diff_functions_multi.py is failing 12 tests at the moment which mostly appear to be ordering issues.

For example:

FAIL: Test status output for svn when run outside workspace ---------------------------------------------------------------------- Traceback (most recent call last): File "/tmp/rosinstall/test/local/test_diff_functions_svn.py", line 203, in test_Rosinstall_status_svn_untracked self.assertEqual('? clone/added-fs.txt\nA clone/added.txt\nD clone/deleted.txt\n! clone/deleted-fs.txt\nM clone/modified.txt\n', output) AssertionError: u'? clone/added-fs.txt\nA clone/added.txt\nD clone/deleted.txt [truncated]... != u'? clone/added-fs.txt\nD clone/deleted.txt\n! clone/deleted-f [truncated]... ? clone/added-fs.txt

• A clone/added.txt D clone/deleted.txt ! clone/deleted-fs.txt M clone/modified.txt

• A clone/added.txt

  — Reply to this email directly or view it on GitHubhttps://github.com/vcstools/rosinstall/issues/14.

[tfoote]

So I get this error on a precise machine. While I don't on my lucid machine. However there are some errors due to using test skips: skipIf, skipUnless, and assertIsNotNone don't exist on lucid. Ticketed separately in #15

[tfoote]

Everything is passing on lucid now. This got 8 of the 14 errors on precise.

The failure summary is below.

======================================================================
FAIL: Test rosinstall diff output from inside workspace.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/rosinstall/test/local/test_diff_functions_multi.py", line 146, in test_multi_diff_rosinstall_inside
    self.check_diff_output(output)
  File "/tmp/rosinstall/test/local/test_diff_functions_multi.py", line 108, in check_diff_output
    self.assertTrue("@@\n+foo\ndiff --git clone_hg/added.txt" in output, 'svn diff misses newline')
AssertionError: svn diff misses newline

======================================================================
FAIL: Test rosinstall diff output from outside workspace.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/rosinstall/test/local/test_diff_functions_multi.py", line 121, in test_multi_diff_rosinstall_outside
    self.check_diff_output(output)
  File "/tmp/rosinstall/test/local/test_diff_functions_multi.py", line 108, in check_diff_output
    self.assertTrue("@@\n+foo\ndiff --git clone_hg/added.txt" in output, 'svn diff misses newline')
AssertionError: svn diff misses newline

======================================================================
FAIL: Test rosws diff output from inside workspace.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/rosinstall/test/local/test_diff_functions_multi.py", line 158, in test_multi_diff_rosws_inside
    self.check_diff_output(output)
  File "/tmp/rosinstall/test/local/test_diff_functions_multi.py", line 108, in check_diff_output
    self.assertTrue("@@\n+foo\ndiff --git clone_hg/added.txt" in output, 'svn diff misses newline')
AssertionError: svn diff misses newline

======================================================================
FAIL: Test rosws diff output from outside workspace.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/rosinstall/test/local/test_diff_functions_multi.py", line 132, in test_multi_diff_rosws_outside
    self.check_diff_output(output)
  File "/tmp/rosinstall/test/local/test_diff_functions_multi.py", line 108, in check_diff_output
    self.assertTrue("@@\n+foo\ndiff --git clone_hg/added.txt" in output, 'svn diff misses newline')
AssertionError: svn diff misses newline

======================================================================
FAIL: Test diff output for svn when run inside workspace
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/rosinstall/test/local/test_diff_functions_svn.py", line 136, in test_Rosinstall_diff_svn_inside
    self.check_diff_output(output)
  File "/tmp/rosinstall/test/local/test_diff_functions_svn.py", line 104, in check_diff_output
    self.assertEqual('Index: clone/added.txt\n===================================================================\n--- clone/added.txt\t(revision 0)\n+++ clone/added.txt\t(revision 0)\n@@ -0,0 +1 @@\n+flam\nIndex: clone/modified.txt\n===================================================================\n--- clone/modified.txt\t(revision 1)\n+++ clone/modified.txt\t(working copy)\n@@ -0,0 +1 @@\n+foo\n', output)
AssertionError: u'Index: clone/added.txt\n====================================================== [truncated]... != u'Index: clone/modified.txt\n=================================================== [truncated]...
+ Index: clone/modified.txt
+ ===================================================================
+ --- clone/modified.txt    (revision 1)
+ +++ clone/modified.txt    (working copy)
+ @@ -0,0 +1 @@
+ +foo
  Index: clone/added.txt
  ===================================================================
  --- clone/added.txt   (revision 0)
  +++ clone/added.txt   (revision 0)
  @@ -0,0 +1 @@
  +flam
- Index: clone/modified.txt
- ===================================================================
- --- clone/modified.txt    (revision 1)
- +++ clone/modified.txt    (working copy)
- @@ -0,0 +1 @@
- +foo

======================================================================
FAIL: Test diff output for svn when run outside workspace
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/rosinstall/test/local/test_diff_functions_svn.py", line 115, in test_Rosinstall_diff_svn_outside
    self.check_diff_output(output)
  File "/tmp/rosinstall/test/local/test_diff_functions_svn.py", line 104, in check_diff_output
    self.assertEqual('Index: clone/added.txt\n===================================================================\n--- clone/added.txt\t(revision 0)\n+++ clone/added.txt\t(revision 0)\n@@ -0,0 +1 @@\n+flam\nIndex: clone/modified.txt\n===================================================================\n--- clone/modified.txt\t(revision 1)\n+++ clone/modified.txt\t(working copy)\n@@ -0,0 +1 @@\n+foo\n', output)
AssertionError: u'Index: clone/added.txt\n====================================================== [truncated]... != u'Index: clone/modified.txt\n=================================================== [truncated]...
+ Index: clone/modified.txt
+ ===================================================================
+ --- clone/modified.txt    (revision 1)
+ +++ clone/modified.txt    (working copy)
+ @@ -0,0 +1 @@
+ +foo
  Index: clone/added.txt
  ===================================================================
  --- clone/added.txt   (revision 0)
  +++ clone/added.txt   (revision 0)
  @@ -0,0 +1 @@
  +flam
- Index: clone/modified.txt
- ===================================================================
- --- clone/modified.txt    (revision 1)
- +++ clone/modified.txt    (working copy)
- @@ -0,0 +1 @@
- +foo

[tfoote]

Down to 3 failures. I upped the diff limit to get the full errors:

======================================================================
FAIL: Test rosinstall diff output from outside workspace.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/rosinstall/test/local/test_diff_functions_multi.py", line 121, in test_multi_diff_rosinstall_outside
    self.check_diff_output(output)
  File "/tmp/rosinstall/test/local/test_diff_functions_multi.py", line 108, in check_diff_output
    self.assertTrue("@@\n+foo\ndiff --git clone_hg/added.txt" in output, output)
AssertionError: diff --git clone_git/added.txt clone_git/added.txt
new file mode 100644
index 0000000..8d63207
--- /dev/null
+++ clone_git/added.txt
@@ -0,0 +1 @@
+flam
diff --git clone_git/deleted-fs.txt clone_git/deleted-fs.txt
deleted file mode 100644
index e69de29..0000000
diff --git clone_git/deleted.txt clone_git/deleted.txt
deleted file mode 100644
index e69de29..0000000
diff --git clone_git/modified-fs.txt clone_git/modified-fs.txt
index e69de29..257cc56 100644
--- clone_git/modified-fs.txt
+++ clone_git/modified-fs.txt
@@ -0,0 +1 @@
+foo
diff --git clone_git/modified.txt clone_git/modified.txt
index e69de29..257cc56 100644
--- clone_git/modified.txt
+++ clone_git/modified.txt
@@ -0,0 +1 @@
+foo
Index: clone_svn/modified.txt
===================================================================
--- clone_svn/modified.txt      (revision 1)
+++ clone_svn/modified.txt      (working copy)
@@ -0,0 +1 @@
+foo
Index: clone_svn/added.txt
===================================================================
--- clone_svn/added.txt (revision 0)
+++ clone_svn/added.txt (revision 0)
@@ -0,0 +1 @@
+flam
diff --git clone_hg/added.txt clone_hg/added.txt
new file mode 100644
--- /dev/null
+++ clone_hg/added.txt
@@ -0,0 +1,1 @@
+flam
diff --git clone_hg/deleted.txt clone_hg/deleted.txt
deleted file mode 100644
diff --git clone_hg/modified-fs.txt clone_hg/modified-fs.txt
--- clone_hg/modified-fs.txt
+++ clone_hg/modified-fs.txt
@@ -0,0 +1,1 @@
+foo
diff --git clone_hg/modified.txt clone_hg/modified.txt
--- clone_hg/modified.txt
+++ clone_hg/modified.txt
@@ -0,0 +1,1 @@
+foo
=== added file 'added.txt'
--- clone_bzr/added.txt 1970-01-01 00:00:00 +0000
+++ clone_bzr/added.txt 2012-10-25 09:03:13 +0000
@@ -0,0 +1,1 @@
+flam

=== removed file 'deleted-fs.txt'
=== removed file 'deleted.txt'
=== modified file 'modified-fs.txt'
--- clone_bzr/modified-fs.txt   2012-10-25 09:03:11 +0000
+++ clone_bzr/modified-fs.txt   2012-10-25 09:03:13 +0000
@@ -0,0 +1,1 @@
+foo

=== modified file 'modified.txt'
--- clone_bzr/modified.txt      2012-10-25 09:03:11 +0000
+++ clone_bzr/modified.txt      2012-10-25 09:03:13 +0000
@@ -0,0 +1,1 @@
+foo
diff --git clone_git2/added.txt clone_git2/added.txt
new file mode 100644
index 0000000..8d63207
--- /dev/null
+++ clone_git2/added.txt
@@ -0,0 +1 @@
+flam
diff --git clone_git2/deleted-fs.txt clone_git2/deleted-fs.txt
deleted file mode 100644
index e69de29..0000000
diff --git clone_git2/deleted.txt clone_git2/deleted.txt
deleted file mode 100644
index e69de29..0000000
diff --git clone_git2/modified-fs.txt clone_git2/modified-fs.txt
index e69de29..257cc56 100644
--- clone_git2/modified-fs.txt
+++ clone_git2/modified-fs.txt
@@ -0,0 +1 @@
+foo
diff --git clone_git2/modified.txt clone_git2/modified.txt
index e69de29..257cc56 100644
--- clone_git2/modified.txt
+++ clone_git2/modified.txt
@@ -0,0 +1 @@
+foo

======================================================================
FAIL: Test rosws diff output from inside workspace.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/rosinstall/test/local/test_diff_functions_multi.py", line 158, in test_multi_diff_rosws_inside
    self.check_diff_output(output)
  File "/tmp/rosinstall/test/local/test_diff_functions_multi.py", line 108, in check_diff_output
    self.assertTrue("@@\n+foo\ndiff --git clone_hg/added.txt" in output, output)
AssertionError: diff --git clone_git/added.txt clone_git/added.txt
new file mode 100644
index 0000000..8d63207
--- /dev/null
+++ clone_git/added.txt
@@ -0,0 +1 @@
+flam
diff --git clone_git/deleted-fs.txt clone_git/deleted-fs.txt
deleted file mode 100644
index e69de29..0000000
diff --git clone_git/deleted.txt clone_git/deleted.txt
deleted file mode 100644
index e69de29..0000000
diff --git clone_git/modified-fs.txt clone_git/modified-fs.txt
index e69de29..257cc56 100644
--- clone_git/modified-fs.txt
+++ clone_git/modified-fs.txt
@@ -0,0 +1 @@
+foo
diff --git clone_git/modified.txt clone_git/modified.txt
index e69de29..257cc56 100644
--- clone_git/modified.txt
+++ clone_git/modified.txt
@@ -0,0 +1 @@
+foo
Index: clone_svn/modified.txt
===================================================================
--- clone_svn/modified.txt      (revision 1)
+++ clone_svn/modified.txt      (working copy)
@@ -0,0 +1 @@
+foo
Index: clone_svn/added.txt
===================================================================
--- clone_svn/added.txt (revision 0)
+++ clone_svn/added.txt (revision 0)
@@ -0,0 +1 @@
+flam
diff --git clone_hg/added.txt clone_hg/added.txt
new file mode 100644
--- /dev/null
+++ clone_hg/added.txt
@@ -0,0 +1,1 @@
+flam
diff --git clone_hg/deleted.txt clone_hg/deleted.txt
deleted file mode 100644
diff --git clone_hg/modified-fs.txt clone_hg/modified-fs.txt
--- clone_hg/modified-fs.txt
+++ clone_hg/modified-fs.txt
@@ -0,0 +1,1 @@
+foo
diff --git clone_hg/modified.txt clone_hg/modified.txt
--- clone_hg/modified.txt
+++ clone_hg/modified.txt
@@ -0,0 +1,1 @@
+foo
=== added file 'added.txt'
--- clone_bzr/added.txt 1970-01-01 00:00:00 +0000
+++ clone_bzr/added.txt 2012-10-25 09:03:13 +0000
@@ -0,0 +1,1 @@
+flam

=== removed file 'deleted-fs.txt'
=== removed file 'deleted.txt'
=== modified file 'modified-fs.txt'
--- clone_bzr/modified-fs.txt   2012-10-25 09:03:11 +0000
+++ clone_bzr/modified-fs.txt   2012-10-25 09:03:13 +0000
@@ -0,0 +1,1 @@
+foo

=== modified file 'modified.txt'
--- clone_bzr/modified.txt      2012-10-25 09:03:11 +0000
+++ clone_bzr/modified.txt      2012-10-25 09:03:13 +0000
@@ -0,0 +1,1 @@
+foo
diff --git clone_git2/added.txt clone_git2/added.txt
new file mode 100644
index 0000000..8d63207
--- /dev/null
+++ clone_git2/added.txt
@@ -0,0 +1 @@
+flam
diff --git clone_git2/deleted-fs.txt clone_git2/deleted-fs.txt
deleted file mode 100644
index e69de29..0000000
diff --git clone_git2/deleted.txt clone_git2/deleted.txt
deleted file mode 100644
index e69de29..0000000
diff --git clone_git2/modified-fs.txt clone_git2/modified-fs.txt
index e69de29..257cc56 100644
--- clone_git2/modified-fs.txt
+++ clone_git2/modified-fs.txt
@@ -0,0 +1 @@
+foo
diff --git clone_git2/modified.txt clone_git2/modified.txt
index e69de29..257cc56 100644
--- clone_git2/modified.txt
+++ clone_git2/modified.txt
@@ -0,0 +1 @@
+foo

Name                          Stmts   Miss  Cover   Missing
-----------------------------------------------------------
rosinstall                        1      0   100%   
rosinstall.__version__            1      0   100%   
rosinstall.cli_common           178      9    95%   66, 103, 151-152, 181, 276-277, 325, 329
rosinstall.common               189     23    88%   41-42, 62-71, 265-273, 292, 339-344, 353
rosinstall.config               111      7    94%   85, 104-105, 144, 167, 184, 196
rosinstall.config_elements      200     26    87%   114, 118, 126, 133, 136, 140, 170, 173, 188, 191, 201, 204, 213, 216, 240, 267, 288, 292-294, 301-303, 310-311, 348, 423-424
rosinstall.config_yaml          176      8    95%   72, 81, 108, 189, 195, 198, 297, 343
rosinstall.helpers               58      3    95%   64, 80, 115
rosinstall.multiproject_cli     320     90    72%   62-74, 129-130, 136, 168-169, 174, 239-241, 248-249, 256-257, 259-260, 265-266, 270, 273, 325-327, 349-350, 410-411, 430, 442-444, 451-452, 459-461, 560-562, 598-603, 616-619, 625, 631, 635-638, 647-698
rosinstall.multiproject_cmd     194     62    68%   81, 117, 190-208, 236, 271, 290, 295-296, 332-337, 353-391
rosinstall.rosinstall_cli        93     16    83%   61-62, 67, 141-142, 145, 152, 156, 158, 163, 175-180
rosinstall.rosinstall_cmd        35      9    74%   92-100
rosinstall.rosws_cli            271     90    67%   86-87, 118, 126, 129-130, 163, 215-217, 219-221, 227-237, 242-243, 245-246, 251, 254, 280-320, 384-385, 395, 400, 414, 422, 424, 430-432, 437-440, 484, 486, 488-489, 492-501, 504-505, 517-523, 543-552, 564-571
rosinstall.setupfiles            48      2    96%   247, 302
rosinstall.ui                    36     23    36%   48, 59-61, 64-67, 73-97
-----------------------------------------------------------
TOTAL                          1911    368    81%   
----------------------------------------------------------------------
Ran 163 tests in 70.169s

FAILED (failures=3)

[tfoote]

Ran 163 tests in 71.822s

OK