search

ohsewon / test

0 stars 0 forks source link

[ISSUE][CLOSED] [Test Framework] Incorrect result if .log is null or cmp differs #275

Closed ohsewon closed 6 years ago

ohsewon commented 6 years ago

Issue by myungjoo-ham Thursday Jul 12, 2018 at 10:45 GMT Originally opened as https://github.sec.samsung.net/STAR/nnstreamer/issues/275

  1. If resulting .log is null, do_test returns OK (0).
  2. Failing gstTest in do_test is ignored.
  3. CMP fail is not counted: 
    
Testing: nnstreamer_converter
=================================================
Test Case Generation Started
testcase02_BGRx_642x480.golden testcase02_BGRx_642x480.log differ: byte 1, line 1
testcase02_BGRx_642x480.golden testcase02_BGRx_642x480.nonip.log differ: byte 1, line 1
[PASSED] gst-launch with case 1
[PASSED] golden test comparison case 1-1
[PASSED] golden test comparison case 1-2
[PASSED] gst-launch with case 2-1
[PASSED] golden test comparison case 2-1
[PASSED] gst-launch with case 2-2
[PASSED] golden test comparison case 2-2
[PASSED] gst-launch with case 2-3
[PASSED] golden test comparison case 2-3
[PASSED] gst-launch with case 2-4
[PASSED] golden test comparison case 2-4
[PASSED] gst-launch with case 3-1
[PASSED] golden test comparison case 3-1
[PASSED] gst-launch with case 3-2
[PASSED] golden test comparison case 3-2
[PASSED] gst-launch with case 3-3
[PASSED] golden test comparison case 3-3
[PASSED] gst-launch with case 3-4
[PASSED] golden test comparison case 3-4
[PASSED] (Supposed to be failed and actually failed) gst-launch with case 5-F
[PASSED] (Supposed to be failed and actually failed) gst-launch with case 6-F
[PASSED] (Supposed to be failed and actually failed) gst-launch with case 7-F
[PASSED] gst-launch with case 8
[PASSED] golden test comparison case 8
=================================================