PoV should pass or fail?

[pfsun]

Hi All, When we test PoV for each CGC binary, should it pass or fail? Based on your result, PoV will pass for each CGC binary. What is the PoV testing? Thanks.

[pfsun]

For testing CROMU_00011, I think you call it hawaii_sets, right? If I use your script to test POV, the result will be ''' ... negotiating negotiation type: 1 type 1 masks: 00000000 00000000 type 1 pov: 00000000 00000000 0 done waiting END REPLAY POV type 1 negotated masks: 00000000 00000000 0 ok - TYPE 1 POV POV type 1 negotated masks: 00000000 00000000 0 ok - TYPE 1 POV TOTAL TESTS: 2 TOTAL PASSED: 2

=> Passed 2/2

'''

If I don't use your script, and directly use the command and cqe pov xml format. ''' CROMU_00011 - pov/POV_00000.xml connected to ('127.84.221.67', 2346) ok 1 - match: string ok 2 - write: sent 201 bytes ok 3 - match: string ok 4 - write: sent 201 bytes ok 5 - match: string ok 6 - write: sent 201 bytes ok 7 - match: string ok 8 - write: sent 201 bytes ok 9 - match: string ok 10 - write: sent 201 bytes ok 11 - match: string ok 12 - write: sent 201 bytes ok 13 - match: string ok 14 - write: sent 201 bytes ok 15 - match: string ok 16 - write: sent 201 bytes ok 17 - match: string ok 18 - write: sent 201 bytes ok 19 - match: string ok 20 - write: sent 201 bytes ok 21 - match: string ok 22 - write: sent 201 bytes ok 23 - match: string ok 24 - write: sent 201 bytes ok 25 - match: string ok 26 - write: sent 201 bytes ok 27 - match: string ok 28 - write: sent 201 bytes ok 29 - match: string ok 30 - write: sent 201 bytes ok 31 - match: string ok 32 - write: sent 36 bytes ok 33 - match: string ok 34 - write: sent 36 bytes ok 35 - match: string ok 36 - write: sent 36 bytes ok 37 - match: string ok 38 - write: sent 36 bytes ok 39 - match: string ok 40 - write: sent 36 bytes ok 41 - match: string ok 42 - write: sent 36 bytes ok 43 - match: string ok 44 - write: sent 36 bytes ok 45 - match: string ok 46 - write: sent 36 bytes ok 47 - match: string ok 48 - write: sent 36 bytes ok 49 - match: string ok 50 - write: sent 36 bytes ok 51 - match: string ok 52 - write: sent 36 bytes ok 53 - match: string ok 54 - write: sent 37 bytes ok 55 - match: string ok 56 - write: sent 37 bytes ok 57 - match: string ok 58 - write: sent 37 bytes ok 59 - match: string ok 60 - write: sent 37 bytes ok 61 - match: string ok 62 - write: sent 41 bytes ok 63 - match: string ok 64 - write: sent 7 bytes tests passed: 64 tests failed: 0 service - pov/POV_00001.xml connected to ('127.84.221.67', 2346) ok 1 - read length ok 2 - write: sent 20 bytes ok 3 - read length ok 4 - write: sent 22 bytes ok 5 - read length ok 6 - write: sent 8 bytes not ok 7 - recv failed. ([''] so far) stopping due to failure tests passed: 6 tests failed: 1 total tests passed: 70 total tests failed: 1 polls passed: 1 polls failed: 1 not ok - process cored: (signal 11: SIGSEGV) '''

[krx]

When testing a vulnerable challenge with cb-test, the --should_core flag needs to be used. This will make it confirm that the process crashed + has correct register states (Type 1), or if bytes were successfully leaked from the flag page (Type 2). The test is considered passed if either of these verify correctly. Without --should_core it assumes the challenge is patched, and the test passes on no errors/leaks.

tester.py already handles passing that flag for you, so that's why you're seeing a difference in the results there.

[pfsun]

Thanks @krx. If there is fail when passing --should_core flag for vulnerable challenge, which means there is problem when compiling vulnerable challenge, right?

[krx]

It could be a problem with either the challenge or the POV. Some of the exploits make assumptions about the stack layout, where things are in memory, etc. that may not carry over from OS to OS. It's a bit hard to tell exactly what the issue is without looking into each challenge that doesn't pass all of its POVs.

[pfsun]

Thanks @krx