Capitains / HookTest

Testing script for Hook
Mozilla Public License 2.0
3 stars 3 forks source link

Remaining print in console ? #137

Closed PonteIneptique closed 6 years ago

PonteIneptique commented 6 years ago

If you look at the following repository https://travis-ci.org/PonteIneptique/digiliblt and you do hooktest . -w 7 you get

['', '', '', '', '', '', '', '', '', '', '', '', '']
['u.c. 248 b.', 'u.c. 252 b.', 'u.c. 256 b.', 'u.c. 260 b.', 'u.c. 264 b.', 'u.c. 268 b.', 'u.c. 272 b.', 'u.c. 276 b.', 'u.c. 280 b.', 'u.c. 284 b.', 'u.c. 288 b.', 'u.c. 292 b.', 'u.c. 296 b.', 'u.c. 300 b.', 'u.c. 304 b.', 'u.c. 308 b.', 'u.c. 312 b.', 'u.c. 316 b.', 'u.c. 320 b.', 'u.c. 324 b.', 'u.c. 328 b.', 'u.c. 332 b.', 'u.c. 336 b.', 'u.c. 340 b.', 'u.c. 344 b.', 'u.c. 348 b.', 'u.c. 352 b.', 'u.c. 356 b.', 'u.c. 360 b.', 'u.c. 364 b.', 'u.c. 368 b.', 'u.c. 372 b.', 'u.c. 376 b.', 'u.c. 380 b.', 'u.c. 384 b.', 'u.c. 388 b.', 'u.c. 392 b.', 'u.c. 396 b.', 'u.c. 400 b.', 'u.c. 404 b.', 'u.c. 408 b.', 'u.c. 412 b.', 'u.c. 416 b.', 'u.c. 420 b.', 'u.c. 424 b.', 'u.c. 428 b.', 'u.c. 432 b.', 'u.c. 436 b.', 'u.c. 440 b.', 'u.c. 444 b.', 'u.c. 448 b.', 'u.c. 452 b.', 'u.c. 456 b.', 'u.c. 460 b.', 'u.c. 465 b.', 'u.c. 469 b.', 'u.c. 473 b.', 'u.c. 477 b.', 'u.c. 481 b.', 'u.c. 485 b.', 'u.c. 489 b.', 'u.c. 493 b.', 'u.c. 497 b.', 'u.c. 501 b.', 'u.c. 505 b.', 'u.c. 509 b.', 'u.c. 513 b.', 'u.c. 517 b.', 'u.c. 521 b.', 'u.c. 525 b.', 'u.c. 529 b.', 'u.c. 533 b.', 'u.c. 537 b.', 'u.c. 541 b.', 'u.c. 545 b.', 'u.c. 549 b.', 'u.c. 553 b.', 'u.c. 557 b.', 'u.c. 561 b.', 'u.c. 565 b.', 'u.c. 569 b.', 'u.c. 573 b.', 'u.c. 577 b.', 'u.c. 581 b.', 'u.c. 585 b.', 'u.c. 589 b.', 'u.c. 593 b.', 'u.c. 597 b.', 'u.c. 601 b.', 'u.c. 605 b.', 'u.c. 609 b.', 'u.c. 613 b.', 'u.c. 617 b.', 'u.c. 621 b.', 'u.c. 625 b.', 'u.c. 629 b.', 'u.c. 633 b.', 'u.c. 637 b.', 'u.c. 641 b.', 'u.c. 645 b.', 'u.c. 649 b.', 'u.c. 653 b.', 'u.c. 657 b.', 'u.c. 661 b.', 'u.c. 665 b.', 'u.c. 669 b.', 'u.c. 673 b.', 'u.c. 677 b.', 'u.c. 681 b.', 'u.c. 685 b.', 'u.c. 689 b.', 'u.c. 693 b.', 'u.c. 697 b.', 'u.c. 701 b.', 'u.c. 705 b.', 'u.c. 709 b.', 'u.c. 713 b.', 'u.c. 717 b.', 'u.c. 721 b.', 'u.c. 725 b.', 'u.c. 729 b.', 'u.c. 733 b.', 'u.c. 737 b.', 'u.c. 741 b.', 'u.c. 745 b.', 'u.c. 749 b.', 'u.c. 753 b.', 'p. Chr. 4 b.', 'p. Chr. 8 b.', 'p. Chr. 16 b.', 'p. Chr. 20 b.', 'p. Chr. 24 b.', 'p. Chr. 28 b.', 'p. Chr. 32 b.', 'p. Chr. 36 b.', 'p. Chr. 40 b.', 'p. Chr. 44 b.', 'p. Chr. 48 b.', 'p. Chr. 52 b.', 'p. Chr. 56 b.', 'p. Chr. 60 b.', 'p. Chr. 64 b.', 'p. Chr. 68 b.', 'p. Chr. 72 b.', 'p. Chr. 76 b.', 'p. Chr. 80 b.', 'p. Chr. 84 b.', 'p. Chr. 88 b.', 'p. Chr. 92 b.', 'p. Chr. 96 b.', 'p. Chr. 100 b.', 'p. Chr. 104 b.', 'p. Chr. 108 b.', 'p. Chr. 112 b.', 'p. Chr. 116 b.', 'p. Chr. 120 b.', 'p. Chr. 124 b.', 'p. Chr. 128 b.', 'p. Chr. 132 b.', 'p. Chr. 136 b.', 'p. Chr. 140 b.', 'p. Chr. 144 b.', 'p. Chr. 148 b.', 'p. Chr. 152 b.', 'p. Chr. 156 b.', 'p. Chr. 160 b.', 'p. Chr. 164 b.', 'p. Chr. 168 b.', 'p. Chr. 172 b.', 'p. Chr. 176 b.', 'p. Chr. 180 b.', 'p. Chr. 184 b.', 'p. Chr. 188 b.', 'p. Chr. 192 b.', 'p. Chr. 196 b.', 'p. Chr. 200 b.', 'p. Chr. 204 b.', 'p. Chr. 208 b.', 'p. Chr. 212 b.', 'p. Chr. 216 b.', 'p. Chr. 220 b.', 'p. Chr. 224 b.', 'p. Chr. 228 b.', 'p. Chr. 232 b.', 'p. Chr. 236 b.', 'p. Chr. 240 b.', 'p. Chr. 244 b.', 'p. Chr. 248 b.', 'p. Chr. 252 b.', 'p. Chr. 256 b.', 'p. Chr. 260 b.', 'p. Chr. 264 b.', 'p. Chr. 268 b.', 'p. Chr. 272 b.', 'p. Chr. 276 b.', 'p. Chr. 280 b.', 'p. Chr. 284 b.', 'p. Chr. 288 b.', 'p. Chr. 292 b.', 'p. Chr. 296 b.', 'p. Chr. 300 b.', 'p. Chr. 304 b.', 'p. Chr. 312 b.', 'p. Chr. 316 b.', 'p. Chr. 320 b.', 'p. Chr. 324 b.', 'p. Chr. 328 b.', 'p. Chr. 332 b.', 'p. Chr. 336 b.', 'p. Chr. 340 b.', 'p. Chr. 344 b.', 'p. Chr. 348 b.', 'p. Chr. 352 b.', 'p. Chr. 312 b.', 'p. Chr. 316 b.', 'p. Chr. 320 b.', 'p. Chr. 324 b.', 'p. Chr. 328 b.', 'p. Chr. 332 b.', 'p. Chr. 336 b.', 'p. Chr. 340 b.', 'p. Chr. 344 b.', 'p. Chr. 348 b.', 'p. Chr. 352 b.', 'p. Chr. 356 b.', 'p. Chr. 369 b.', 'p. Chr. 373 b.', 'p. Chr. 377 b.', 'p. Chr. 380 b.', 'p. Chr. 384 b.', 'p. Chr. 388 b.', 'p. Chr. 392 b.', 'p. Chr. 396 b.', 'p. Chr. 400 b.', 'p. Chr. 408 b.']
['215 s.', '217s.', '357 s.', '363 s.', '410 s.', '425 s.', '478 s.', '2 s.', '5 s.', '301 s.', '481 s.']
sonofmun commented 6 years ago

This is the full printout? E.g., there is no introductory message that tests are being started or a final report about the texts? Because this looks a lot like reports about empty or illegal references that comes after the textual table but before the final report. Are there no labels for those lines, e.g., "Empty references found:"?

PonteIneptique commented 6 years ago

It is normal that nothing is shown because I did not put verbose or console. But even in other mode, you'd have those remaining print out of context...

sonofmun commented 6 years ago

Gotcha. I saw -w 7 and read -v 7. I will take a look and see what is happening.

sonofmun commented 6 years ago

This seems to be something that was introduced with the empty references test. I will continue to check.

sonofmun commented 6 years ago

The problem looks to have been with MyCapytain. When I was working on the EmptyReference exception there, I used a print statement for debugging that I accidentally left in there. I will do a PR there to fix it. But I will add a test for it in HookTest as well.

sonofmun commented 6 years ago

Fixed in https://github.com/Capitains/MyCapytain/pull/176

sonofmun commented 6 years ago

I can't seem to find where this is printing during the tests. When I check stderr and stdout, they are both empty. E.g., the following code (which uses https://github.com/Capitains/HookTest/blob/master/tests/test_run.py#L74-L95):

    def test_run_local_no_option_empty_refs(self):
           """ Test a run on the local tests passages
            This test is specifically to check whether empty references are printed despite --console being False
        """
        status, logs = self.hooktest(["./tests/failing_empties", "-w", "3"])
        self.assertEqual(len(logs), 0, "There should be no logs")
        self.assertEqual(status, "failed", "Test should fail")

We want to get where this test does pass as it is. But right now, it shouldn't because it should produce the artifacts described above. (This test does produce and print these artifacts: https://github.com/Capitains/HookTest/blob/master/tests/test_units.py#L579-L597)

PonteIneptique commented 6 years ago

Hey, This being a non obstructing bug, I do not think it's a good idea to test it. It was a bug in another library. Do not worry about it. Sorry for the late weight in, I had a lot on my plate today :)

PonteIneptique commented 6 years ago

Thanks for the investigation. Please close if you are okay with my answer :)

sonofmun commented 6 years ago

OK. Closing. And NP with the late response. I also had other things to do today.