Security-Industry-Association / libosdp-conformance

Apache License 2.0
27 stars 14 forks source link

CP Conformance tool lists tests as passed without a connection to the CP #18

Closed mpserafin closed 3 years ago

mpserafin commented 7 years ago

I failed to enable the USB port in my virtual machine and tried the CP Conformance test and noticed that some of the tests were passing which confused me at first and then I realized I didn’t even have a connection to my controller from my laptop. Below is the report that indicates that 28 tests had passed. Not sure how this can be if I never connected my controller.


Conformance Report: Testing PD with Peripheral (testing PD) Profile

Version: 0.1 Build 2

2-1-1 Physical Interface Exercised 2-2-1 Signalling (9600) Exercised 2-2-2 Signalling (19200) Failed 2-2-3 Signalling (38400) Failed 2-2-4 Signalling (115200) Failed 2-3-1 Character Encoding Skipped 2-4-1 Channel Access Skipped 2-4-2 Timeout resend Skipped 2-4-3 Busy resend Skipped 2-4-4 New on busy Skipped 2-5-1 Multi-byte Data Encoding Exercised 2-6-1 Packet Size Limits Exercised 2-6-2 Packet size from PD Skipped 2-6-3 Packet size stress CP Skipped 2-7-1 Timing Skipped 2-7-2 Max delay Skipped 2-7-3 Offline test Skipped 2-8-1 Message Synchronization Skipped 2-9-1 Packet Formats Exercised 2-10-1 SOM Start of Message Exercised 2-10-2 SOM sent Exercised 2-11-1 ADDR Skipped 2-11-2 No data on 7F Skipped 2-11-3 Config (0x7F) Address Exercised 2-12-1 LEN Exercised 2-13-1 CTRL Exercised 2-13-2 Secure Control Block [5] Skipped 2-13-3 SCB absent Exercised 2-13-4 Sequence numbers Skipped 2-14-1 Security Block (hdr process only) Skipped 2-15-1 Incoming C/R valid Exercised 2-15-2 No invalid C/R received Skipped 2-16-1 CHKSUM/CRC16 Exercised 2-16-2 Checksum Exercised 2-17-1 Large Data Messages Skipped 3-1-1 Poll Exercised 3-1-2 Poll raw Untested 3-1-3 Poll response 3 Skipped 3-1-4 Poll response 4 Skipped 3-2-1 ID Report Request Exercised 3-3-1 Peripheral Device Capabilities Req Exercised 3-4-1 Diagnostic Function Request Skipped 3-5-1 Local Status Report Request Exercised 3-6-1 Input Status Report Request Exercised 3-7-1 Output Status Report Request Untested 3-7-2 Ostat ack Skipped 3-8-1 Reader Status Report Request Untested 3-9-1 Output Control Command Untested 3-10-1 LED Test (Red) Exercised 3-10-2 LED Test (Green) Exercised 3-11-1 Buzzer Control Untested 3-12-1 Text output Exercised 3-14-1 COMSET Exercised 3-16-1 Reader Prompt Skipped 3-17-1 Scan and send bio template Skipped 3-18-1 Scan and match bio template Skipped 3-19-1 Multipart message continuation Skipped 3-20-1 Manufacturer specific command Skipped 3-21-1 Stop multipart message Skipped 3-22-1 Maximum acceptable reply size Skipped 4-1-1 General Ack Nothing to Report Exercised 4-2-1 Negative Ack Error Response Exercised 4-3-1 Device Identification Report Exercised 4-3-2 Ident report consistent Failed 4-4-1 Device Capabilities Report Exercised 4-4-2 Capabilities report consistent Skipped 4-5-1 osdp_LSTATR Local Status Report Untested 4-5-2 osdp_LSTATR Tamper Untested 4-5-3 osdp_LSTATR Power Untested 4-6-1 Input Status Report Untested 4-6-2 Input report consistent Skipped 4-7-1 osdp_OSTATR Untested 4-7-2 osdp_OSTATR for POLL Untested 4-7-3 osdp_OSTATR for POLL Skipped 4-8-1 osdp_RSTATR Untested 4-9-1 RAW Read Untested 4-10-1 Formatted Read Skipped 4-11-1 Keypad input Exercised 4-12-1 COM Report Untested 4-13-1 Biometrics Read Skipped 4-14-1 Biometrics Match Skipped 4-15-1 Mfg Response Skipped 4-16-1 Busy Untested === Passed: 28 Failed: 4 Untested: 15 Skipped: 36 Total: 83 Testing PD with Peripheral (testing PD) Profile

Version: 0.1 Build 2

---end of report---

rsgmodelworks commented 7 years ago

It is supposed to be the case that tests are not declared exercised until we get a response or sync up. Should be a glitch in that logic that caused this effect.

rsgmodelworks commented 4 years ago

revisiting this artifact now with OSDP Verified test jig. Calibration process checks for this so should be able to address these.

rsgmodelworks commented 3 years ago

as far as I know this is all resolved now. The current "calibration" process checks to confirm this.