The test is designed to expect the ssh command to fail because login permission is denied, but if the ssh command fails for other reasons the test case still passes, even though it has not fulfilled the test case goal of checking the ciphers.
In my working environment (see below), the test case fails because the ssh command line doesn't contain '-o' in front of each of the second and third ssh option arguments, as demonstrated below.
failing code:
I observed this failure on Ubuntu 18.04 with OpenSSH 7.6p1 / OpenSSL 1.0.2 and Python 3.6.9.
I propose improving the test case by first verifying that the command returns the kind of information desired.
In my environment the remediated code returns the error 'Permission denied' since the ssh attempt requests no password prompt ('-o NumberOfPasswordPrompts=0). My refactored code expects that string to be returned, and returns SKIP if that criterion is not satisfied.
Here is the git diff showing how I have refactored the code:
diff --git a/security/test_bmc_ssh_security.robot b/security/test_bmc_ssh_security.robot
index fae06281..c70a3972 100644
--- a/security/test_bmc_ssh_security.robot
+++ b/security/test_bmc_ssh_security.robot
@@ -41,6 +41,8 @@ Verify BMC SSH Weak Cipher And Algorithm
# debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,
# hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,
# umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
+ # ...
+ # ... Permission denied (publickey,password).
# Example of weak algorithms to check:
# - encryption: triple-DES ("DES-CBC3").
@@ -49,7 +51,16 @@ Verify BMC SSH Weak Cipher And Algorithm
# - KEX: diffie-hellman-group1(any) , (any) SHA1
Printn
- ${ssh_cmd_buf}= Catenate ssh -o NumberOfPasswordPrompts=0 UserKnownHostsFile=/dev/null
- ... StrictHostKeyChecking=no -vv ${OPENBMC_HOST} 2>&1
+ ${ssh_cmd_buf}= Catenate ssh -o NumberOfPasswordPrompts=0 -o UserKnownHostsFile=/dev/null
+ ... -o StrictHostKeyChecking=no -vv ${OPENBMC_HOST} 2>&1
+
+ ${rc} ${std_err}= Shell Cmd ! ${ssh_cmd_buf}
+ Log std_err=${std_err} console=yes
+ Log rc=${rc} console=yes
+
+ ${has_it}= Run Keyword And Return Status Should Contain ${std_err} Permission denied
+ Skip If not ${has_it}
+ ... Skipping test case since response is not as expected
+
Shell Cmd ! ${ssh_cmd_buf} | egrep -- "${weak_key_regex}"
Shell Cmd ! ${ssh_cmd_buf} | egrep -- "${mac_key_regex}"
The test is designed to expect the ssh command to fail because login permission is denied, but if the ssh command fails for other reasons the test case still passes, even though it has not fulfilled the test case goal of checking the ciphers.
In my working environment (see below), the test case fails because the ssh command line doesn't contain '-o' in front of each of the second and third ssh option arguments, as demonstrated below. failing code:
remediated code:
I observed this failure on Ubuntu 18.04 with OpenSSH 7.6p1 / OpenSSL 1.0.2 and Python 3.6.9.
I propose improving the test case by first verifying that the command returns the kind of information desired. In my environment the remediated code returns the error 'Permission denied' since the ssh attempt requests no password prompt ('-o NumberOfPasswordPrompts=0). My refactored code expects that string to be returned, and returns SKIP if that criterion is not satisfied.
Here is the git diff showing how I have refactored the code: