search

jvoisin / snuffleupagus

Security module for php7 and php8 - Killing bugclasses and virtual-patching the rest!
https://snuffleupagus.readthedocs.io
GNU Lesser General Public License v3.0
783 stars 92 forks source link

test suite on FreeBSD #108

Closed fichtner closed 6 years ago

fichtner commented 6 years ago

As discussed with @jvoisin

The apparent issue is that hash.so is not advertised in "tmp-php.ini", after that the test suite seems to run, see below.

Not sure how to proceed and wether the FreeBSD port needs more adjustments (e.g. for currently skipped tests).

===>  Testing for snuffleupagus-0.1.0

Build complete.
Don't forget to run 'make test'.

if test ! -z "/usr/local/bin/php" && test -x "/usr/local/bin/php"; then  INI_FILE=`/usr/local/bin/php -d 'display_errors=stderr' -r 'echo php_ini_loaded_file();' 2> /dev/null`;  if test "$INI_FILE"; then  /usr/bin/egrep -h -v '^(magic_quotes_(gpc|runtime|sybase)?|(zend_)?extension(_debug)?(_ts)?)[\t\ ]*=' "$INI_FILE" > /usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/tmp-php.ini || :;  else  echo "extension=/usr/local/lib/php/20160303/hash.so" > /usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/tmp-php.ini;  fi;  INI_SCANNED_PATH=`/usr/local/bin/php -d 'display_errors=stderr' -r '$a = explode(",\n", trim(php_ini_scanned_files())); echo $a[0];' 2> /dev/null`;  if test "$INI_SCANNED_PATH"; then  INI_SCANNED_PATH=`/usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/build/shtool path -d $INI_SCANNED_PATH`;  /usr/bin/egrep -h -v '^(magic_quotes_(gpc|runtime|sybase)?|(zend_)?extension(_debug)?(_ts)?)[\t\ ]*=' "$INI_SCANNED_PATH"/*.ini >> /usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/tmp-php.ini || :;  fi;  TEST_PHP_EXECUTABLE=/usr/local/bin/php  TEST_PHP_SRCDIR=/usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src  CC="cc"  /usr/local/bin/php -n -c /usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/tmp-php.ini -d 'open_basedir=' -d 'output_buffering=0' -d 'memory_limit=-1' /usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/run-tests.php -n -c /usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/tmp-php.ini -d extension_dir=/usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/modules/ `  if test "x/usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/modules/snuffleupagus.la" != "x"; then  for i in /usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/modules/snuffleupagus.la""; do  . $i; /usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/build/shtool echo -n -- " -d extension=$dlname";  done;  fi;  if test "x" != "x"; then  for i in ""; do  . $i; /usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/build/shtool echo -n -- " -d zend_extension=/usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/modules/$dlname";  done;  fi` ;  TEST_RESULT_EXIT_CODE=$?;  rm /usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/tmp-php.ini;  exit $TEST_RESULT_EXIT_CODE;  else  echo "ERROR: Cannot run tests without CLI sapi.";  fi

=====================================================================
PHP         : /usr/local/bin/php 
PHP_SAPI    : cli
PHP_VERSION : 7.1.12
ZEND_VERSION: 3.1.0
PHP_OS      : FreeBSD - FreeBSD sensey64 11.0-RELEASE-p15 FreeBSD 11.0-RELEASE-p15 #0: Tue Nov 14 08:04:40 UTC 2017     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
INI actual  : /usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src/tmp-php.ini
More .INIs  :   
CWD         : /usr/ports/security/snuffleupagus/work/snuffleupagus-0.1.0/src
Extra dirs  : 
VALGRIND    : Not used
=====================================================================
TIME START 2017-12-28 21:36:15
=====================================================================
TEST 1/190 [tests/broken_conf.phpt]
PASS Broken configuration [tests/broken_conf.phpt] 
TEST 2/190 [tests/broken_conf2.phpt]
PASS Broken configuration [tests/broken_conf2.phpt] 
TEST 3/190 [tests/broken_conf_config_regexp.phpt]
PASS Broken configuration [tests/broken_conf_config_regexp.phpt] 
TEST 4/190 [tests/broken_conf_config_regexp_no_closing_paren.phpt]
PASS Broken configuration - regexp without a closing parenthesis [tests/broken_conf_config_regexp_no_closing_paren.phpt] 
TEST 5/190 [tests/broken_conf_enable_disable.phpt]
PASS Global strict mode [tests/broken_conf_enable_disable.phpt] 
TEST 6/190 [tests/broken_conf_expecting_bool.phpt]
PASS Bad boolean value in configuration [tests/broken_conf_expecting_bool.phpt] 
TEST 7/190 [tests/broken_conf_invalid_cidr.phpt]
PASS Broken configuration [tests/broken_conf_invalid_cidr.phpt] 
TEST 8/190 [tests/broken_conf_invalid_cidr6.phpt]
PASS Broken configuration [tests/broken_conf_invalid_cidr6.phpt] 
TEST 9/190 [tests/broken_conf_invalid_cidr6_no_slash.phpt]
PASS Broken configuration, invalid cidr for ipv6 because there is no `/` in it [tests/broken_conf_invalid_cidr6_no_slash.phpt] 
TEST 10/190 [tests/broken_conf_invalid_cidr6_too_big.phpt]
PASS Broken configuration, cidr for ipv6 is too big, that will `mod` to 25.
(13337%128 = 25) [tests/broken_conf_invalid_cidr6_too_big.phpt] 
TEST 11/190 [tests/broken_conf_invalid_cidr_value.phpt]
PASS Broken configuration, invalid cidr value
(13337%128 = 25) [tests/broken_conf_invalid_cidr_value.phpt] 
TEST 12/190 [tests/broken_conf_invalid_filename.phpt]
PASS Broken configuration filename without absolute path [tests/broken_conf_invalid_filename.phpt] 
TEST 13/190 [tests/broken_conf_invalid_type.phpt]
PASS Broken conf with wrong type [tests/broken_conf_invalid_type.phpt] 
TEST 14/190 [tests/broken_conf_key_value.phpt]
PASS Broken configuration [tests/broken_conf_key_value.phpt] 
TEST 15/190 [tests/broken_conf_line_empty_string.phpt]
PASS Configuration line with an empty string [tests/broken_conf_line_empty_string.phpt] 
TEST 16/190 [tests/broken_conf_line_no_closing.phpt]
PASS Configuration line without closing parenthese [tests/broken_conf_line_no_closing.phpt] 
TEST 17/190 [tests/broken_conf_local_var_1.phpt]
PASS Broken configuration [tests/broken_conf_local_var_1.phpt] 
TEST 18/190 [tests/broken_conf_local_var_10.phpt]
PASS Broken configuration [tests/broken_conf_local_var_10.phpt] 
TEST 19/190 [tests/broken_conf_local_var_11.phpt]
PASS Broken configuration [tests/broken_conf_local_var_11.phpt] 
TEST 20/190 [tests/broken_conf_local_var_12.phpt]
PASS Broken configuration [tests/broken_conf_local_var_12.phpt] 
TEST 21/190 [tests/broken_conf_local_var_13.phpt]
PASS Broken configuration [tests/broken_conf_local_var_13.phpt] 
TEST 22/190 [tests/broken_conf_local_var_14.phpt]
PASS Broken configuration [tests/broken_conf_local_var_14.phpt] 
TEST 23/190 [tests/broken_conf_local_var_15.phpt]
PASS Broken configuration [tests/broken_conf_local_var_15.phpt] 
TEST 24/190 [tests/broken_conf_local_var_16.phpt]
PASS Broken configuration [tests/broken_conf_local_var_16.phpt] 
TEST 25/190 [tests/broken_conf_local_var_2.phpt]
PASS Broken configuration [tests/broken_conf_local_var_2.phpt] 
TEST 26/190 [tests/broken_conf_local_var_3.phpt]
PASS Broken configuration [tests/broken_conf_local_var_3.phpt] 
TEST 27/190 [tests/broken_conf_local_var_4.phpt]
PASS Broken configuration [tests/broken_conf_local_var_4.phpt] 
TEST 28/190 [tests/broken_conf_local_var_5.phpt]
PASS Broken configuration [tests/broken_conf_local_var_5.phpt] 
TEST 29/190 [tests/broken_conf_local_var_6.phpt]
PASS Broken configuration [tests/broken_conf_local_var_6.phpt] 
TEST 30/190 [tests/broken_conf_local_var_7.phpt]
PASS Broken configuration [tests/broken_conf_local_var_7.phpt] 
TEST 31/190 [tests/broken_conf_local_var_8.phpt]
PASS Broken configuration [tests/broken_conf_local_var_8.phpt] 
TEST 32/190 [tests/broken_conf_local_var_9.phpt]
PASS Broken configuration [tests/broken_conf_local_var_9.phpt] 
TEST 33/190 [tests/broken_conf_lots_of_quotes.phpt]
PASS Configuration line with too many quotes [tests/broken_conf_lots_of_quotes.phpt] 
TEST 34/190 [tests/broken_conf_missing_script.phpt]
PASS Invalid configuration file for upload [tests/broken_conf_missing_script.phpt] 
TEST 35/190 [tests/broken_conf_mutually_exclusive.phpt]
PASS Broken configuration [tests/broken_conf_mutually_exclusive.phpt] 
TEST 36/190 [tests/broken_conf_mutually_exclusive2.phpt]
PASS Broken configuration [tests/broken_conf_mutually_exclusive2.phpt] 
TEST 37/190 [tests/broken_conf_mutually_exclusive3.phpt]
PASS Broken configuration [tests/broken_conf_mutually_exclusive3.phpt] 
TEST 38/190 [tests/broken_conf_mutually_exclusive4.phpt]
PASS Broken configuration [tests/broken_conf_mutually_exclusive4.phpt] 
TEST 39/190 [tests/broken_conf_mutually_exclusive5.phpt]
PASS Broken configuration [tests/broken_conf_mutually_exclusive5.phpt] 
TEST 40/190 [tests/broken_conf_mutually_exclusive6.phpt]
PASS Broken configuration [tests/broken_conf_mutually_exclusive6.phpt] 
TEST 41/190 [tests/broken_conf_mutually_exclusive7.phpt]
PASS Broken configuration [tests/broken_conf_mutually_exclusive7.phpt] 
TEST 42/190 [tests/broken_conf_mutually_exclusive8.phpt]
PASS Broken configuration [tests/broken_conf_mutually_exclusive8.phpt] 
TEST 43/190 [tests/broken_conf_no_cookie_action.phpt]
PASS Bad config, invalid action. [tests/broken_conf_no_cookie_action.phpt] 
TEST 44/190 [tests/broken_conf_no_cookie_name.phpt]
PASS Borken configuration - encrypted cookie with no name [tests/broken_conf_no_cookie_name.phpt] 
TEST 45/190 [tests/broken_conf_nonexisting_script.phpt]
PASS Invalid configuration file for upload [tests/broken_conf_nonexisting_script.phpt] 
TEST 46/190 [tests/broken_conf_quotes.phpt]
PASS Broken configuration - missing quote [tests/broken_conf_quotes.phpt] 
TEST 47/190 [tests/broken_conf_samesite.phpt]
PASS Bad config, invalid samesite type. [tests/broken_conf_samesite.phpt] 
TEST 48/190 [tests/broken_conf_truncated.phpt]
PASS Bad boolean value in configuration [tests/broken_conf_truncated.phpt] 
TEST 49/190 [tests/broken_conf_upload_validation.phpt]
PASS Invalid configuration file for upload validation [tests/broken_conf_upload_validation.phpt] 
TEST 50/190 [tests/broken_conf_weird_keyword.phpt]
PASS Bad config, unknown keyword [tests/broken_conf_weird_keyword.phpt] 
TEST 51/190 [tests/broken_conf_wrong_quotes.phpt]
PASS Configuration line with too many quotes [tests/broken_conf_wrong_quotes.phpt] 
TEST 52/190 [tests/broken_conf_wrong_type.phpt]
PASS Broken conf with wrong type [tests/broken_conf_wrong_type.phpt] 
TEST 53/190 [tests/broken_invalid_client_ip4.phpt]
PASS Invalid client IP [tests/broken_invalid_client_ip4.phpt] 
TEST 54/190 [tests/broken_regexp.phpt]
PASS Broken regexp [tests/broken_regexp.phpt] 
TEST 55/190 [tests/broken_unmatching_brackets.phpt]
PASS Broken configuration - unmatching brackets [tests/broken_unmatching_brackets.phpt] 
TEST 56/190 [tests/deny_writable_execution.phpt]
PASS Readonly execution attempt [tests/deny_writable_execution.phpt] 
TEST 57/190 [tests/deny_writable_execution_disabled.phpt]
PASS Readonly execution attempt [tests/deny_writable_execution_disabled.phpt] 
TEST 58/190 [tests/deny_writable_execution_simulation.phpt]
SKIP Readonly execution attempt (simulation mode) [tests/deny_writable_execution_simulation.phpt] 
TEST 59/190 [tests/disable_xxe_dom.phpt]
SKIP Disable XXE [tests/disable_xxe_dom.phpt] 
TEST 60/190 [tests/disable_xxe_dom_disabled.phpt]
SKIP Disable XXE [tests/disable_xxe_dom_disabled.phpt] 
TEST 61/190 [tests/disable_xxe_simplexml.phpt]
SKIP Disable XXE [tests/disable_xxe_simplexml.phpt] 
TEST 62/190 [tests/disable_xxe_simplexml_oop.phpt]
SKIP Disable XXE [tests/disable_xxe_simplexml_oop.phpt] 
TEST 63/190 [tests/disable_xxe_xml_parse.phpt]
SKIP Disable XXE in xml_parse [tests/disable_xxe_xml_parse.phpt] reason: because the `xml` extension isn't loaded
TEST 64/190 [tests/disabled_function_local_var.phpt]
PASS Disable functions - match on a local variable [tests/disabled_function_local_var.phpt] 
TEST 65/190 [tests/disabled_function_local_var_10.phpt]
PASS Disable functions - match on a local variable [tests/disabled_function_local_var_10.phpt] 
TEST 66/190 [tests/disabled_function_local_var_2.phpt]
PASS Disable functions - match on a local variable [tests/disabled_function_local_var_2.phpt] 
TEST 67/190 [tests/disabled_function_local_var_3.phpt]
PASS Disable functions - match on a local variable [tests/disabled_function_local_var_3.phpt] 
TEST 68/190 [tests/disabled_function_local_var_4.phpt]
PASS Disable functions - match on a local variable [tests/disabled_function_local_var_4.phpt] 
TEST 69/190 [tests/disabled_function_local_var_5.phpt]
PASS Disable functions - match on a local variable [tests/disabled_function_local_var_5.phpt] 
TEST 70/190 [tests/disabled_function_local_var_6.phpt]
PASS Disable functions - match on a local variable [tests/disabled_function_local_var_6.phpt] 
TEST 71/190 [tests/disabled_function_local_var_7.phpt]
PASS Disable functions - match on a local variable [tests/disabled_function_local_var_7.phpt] 
TEST 72/190 [tests/disabled_function_local_var_8.phpt]
PASS Disable functions - match on a local variable [tests/disabled_function_local_var_8.phpt] 
TEST 73/190 [tests/disabled_function_local_var_9.phpt]
PASS Disable functions - match on a local variable [tests/disabled_function_local_var_9.phpt] 
TEST 74/190 [tests/disabled_function_local_var_const.phpt]
PASS Disable functions - match on a constant [tests/disabled_function_local_var_const.phpt] 
TEST 75/190 [tests/disabled_function_local_var_obj.phpt]
PASS Disable functions - match on a local variable [tests/disabled_function_local_var_obj.phpt] 
TEST 76/190 [tests/disabled_function_super_global_var.phpt]
PASS Disable functions - match on a super global [tests/disabled_function_super_global_var.phpt] 
TEST 77/190 [tests/disabled_functions.phpt]
PASS Disable functions [tests/disabled_functions.phpt] 
TEST 78/190 [tests/disabled_functions_chain.phpt]
PASS Disable functions by matching the calltrace [tests/disabled_functions_chain.phpt] 
TEST 79/190 [tests/disabled_functions_chain_call_user_func.phpt]
PASS Disable functions by matching the calltrace, with call_user_func involved [tests/disabled_functions_chain_call_user_func.phpt] 
TEST 80/190 [tests/disabled_functions_chain_not_matching.phpt]
PASS Disable functions by matching the calltrace [tests/disabled_functions_chain_not_matching.phpt] 
TEST 81/190 [tests/disabled_functions_cidr.phpt]
PASS Disable functions [tests/disabled_functions_cidr.phpt] 
TEST 82/190 [tests/disabled_functions_cidr_6.phpt]
PASS Disable functions [tests/disabled_functions_cidr_6.phpt] 
TEST 83/190 [tests/disabled_functions_die.phpt]
XFAIL Disable functions - die [tests/disabled_functions_die.phpt]   XFAIL REASON: 
TEST 84/190 [tests/disabled_functions_eval.phpt]
PASS Disable functions - eval [tests/disabled_functions_eval.phpt] 
TEST 85/190 [tests/disabled_functions_eval_filename.phpt]
PASS Disable functions - eval [tests/disabled_functions_eval_filename.phpt] 
TEST 86/190 [tests/disabled_functions_eval_simulation.phpt]
PASS Disable functions - eval (simulation) [tests/disabled_functions_eval_simulation.phpt] 
TEST 87/190 [tests/disabled_functions_exit.phpt]
XFAIL Disable functions - exit [tests/disabled_functions_exit.phpt]   XFAIL REASON: 
TEST 88/190 [tests/disabled_functions_filename_r.phpt]
PASS Disable functions - filename regexp [tests/disabled_functions_filename_r.phpt] 
TEST 89/190 [tests/disabled_functions_include_once.phpt]
PASS Disable functions - include_once [tests/disabled_functions_include_once.phpt] 
TEST 90/190 [tests/disabled_functions_include_simulation.phpt]
PASS Disable functions - Include (simulation) [tests/disabled_functions_include_simulation.phpt] 
TEST 91/190 [tests/disabled_functions_local_var_array.phpt]
PASS Disable functions - match on an array value buried in several levels [tests/disabled_functions_local_var_array.phpt] 
TEST 92/190 [tests/disabled_functions_local_var_array_key.phpt]
PASS Disable functions - match on an array value buried in several levels [tests/disabled_functions_local_var_array_key.phpt] 
TEST 93/190 [tests/disabled_functions_mb.phpt]
PASS Disable functions [tests/disabled_functions_mb.phpt] 
TEST 94/190 [tests/disabled_functions_method.phpt]
PASS Disable functions [tests/disabled_functions_method.phpt] 
TEST 95/190 [tests/disabled_functions_name_r.phpt]
PASS Disable functions [tests/disabled_functions_name_r.phpt] 
TEST 96/190 [tests/disabled_functions_name_type.phpt]
PASS Disable functions [tests/disabled_functions_name_type.phpt] 
TEST 97/190 [tests/disabled_functions_namespace.phpt]
PASS Disable functions in namespaces [tests/disabled_functions_namespace.phpt] 
TEST 98/190 [tests/disabled_functions_noconf.phpt]
PASS Disable functions [tests/disabled_functions_noconf.phpt] 
TEST 99/190 [tests/disabled_functions_nul_byte.phpt]
PASS Disable functions with nul byte [tests/disabled_functions_nul_byte.phpt] 
TEST 100/190 [tests/disabled_functions_param.phpt]
PASS Disable functions [tests/disabled_functions_param.phpt] 
TEST 101/190 [tests/disabled_functions_param_alias.phpt]
PASS Disable functions - alias [tests/disabled_functions_param_alias.phpt] 
TEST 102/190 [tests/disabled_functions_param_allow.phpt]
PASS Disable functions - allow [tests/disabled_functions_param_allow.phpt] 
TEST 103/190 [tests/disabled_functions_param_array.phpt]
PASS Disable functions [tests/disabled_functions_param_array.phpt] 
TEST 104/190 [tests/disabled_functions_param_array_deref.phpt]
PASS Disable functions [tests/disabled_functions_param_array_deref.phpt] 
TEST 105/190 [tests/disabled_functions_param_array_no_value.phpt]
PASS Disable functions - matching on an array's variable only [tests/disabled_functions_param_array_no_value.phpt] 
TEST 106/190 [tests/disabled_functions_param_array_r.phpt]
PASS Disable functions - match on an array using regexp [tests/disabled_functions_param_array_r.phpt] 
TEST 107/190 [tests/disabled_functions_param_array_r_keys.phpt]
PASS Disable functions - match on an array using regexp [tests/disabled_functions_param_array_r_keys.phpt] 
TEST 108/190 [tests/disabled_functions_param_array_several_levels.phpt]
PASS Disable functions - match on an array value buried in several levels [tests/disabled_functions_param_array_several_levels.phpt] 
TEST 109/190 [tests/disabled_functions_param_array_several_levels_int.phpt]
PASS Disable functions - match on an array value buried in several levels [tests/disabled_functions_param_array_several_levels_int.phpt] 
TEST 110/190 [tests/disabled_functions_param_array_several_levels_keys.phpt]
PASS Disable functions - match on an array value buried in several levels [tests/disabled_functions_param_array_several_levels_keys.phpt] 
TEST 111/190 [tests/disabled_functions_param_array_several_levels_keys_int.phpt]
PASS Disable functions - match on an array value buried in several levels [tests/disabled_functions_param_array_several_levels_keys_int.phpt] 
TEST 112/190 [tests/disabled_functions_param_broken_line.phpt]
PASS Disable functions - match on a specific line - broken configuration [tests/disabled_functions_param_broken_line.phpt] 
TEST 113/190 [tests/disabled_functions_param_int.phpt]
PASS Disable functions [tests/disabled_functions_param_int.phpt] 
TEST 114/190 [tests/disabled_functions_param_invalid_pos.phpt]
PASS Disable functions - match on argument's position [tests/disabled_functions_param_invalid_pos.phpt] 
TEST 115/190 [tests/disabled_functions_param_line.phpt]
PASS Disable functions - match on a specific line [tests/disabled_functions_param_line.phpt] 
TEST 116/190 [tests/disabled_functions_param_pos.phpt]
PASS Disable functions - match on argument's position [tests/disabled_functions_param_pos.phpt] 
TEST 117/190 [tests/disabled_functions_param_r.phpt]
PASS Disable functions [tests/disabled_functions_param_r.phpt] 
TEST 118/190 [tests/disabled_functions_param_str_representation.phpt]
PASS Disable functions - casting various types to string internally [tests/disabled_functions_param_str_representation.phpt] 
TEST 119/190 [tests/disabled_functions_parse_class.phpt]
PASS Disable functions - Parsing of an Object as a return value of a function [tests/disabled_functions_parse_class.phpt] 
TEST 120/190 [tests/disabled_functions_pos_type.phpt]
PASS Disable functions - match on argument's position [tests/disabled_functions_pos_type.phpt] 
TEST 121/190 [tests/disabled_functions_require.phpt]
PASS Disable functions - Require [tests/disabled_functions_require.phpt] 
TEST 122/190 [tests/disabled_functions_require_allow.phpt]
PASS Disable functions - Require (allow) [tests/disabled_functions_require_allow.phpt] 
TEST 123/190 [tests/disabled_functions_require_once.phpt]
PASS Disable functions - require_once [tests/disabled_functions_require_once.phpt] 
TEST 124/190 [tests/disabled_functions_require_simulation.phpt]
PASS Disable functions - Require (simulation) [tests/disabled_functions_require_simulation.phpt] 
TEST 125/190 [tests/disabled_functions_ret.phpt]
PASS Disable functions check on `ret`. [tests/disabled_functions_ret.phpt] 
TEST 126/190 [tests/disabled_functions_ret2.phpt]
PASS Disable functions check on `ret`. [tests/disabled_functions_ret2.phpt] 
TEST 127/190 [tests/disabled_functions_ret3.phpt]
PASS Disable functions check on `ret`. [tests/disabled_functions_ret3.phpt] 
TEST 128/190 [tests/disabled_functions_ret_allow.phpt]
PASS Disable functions check on `ret`. [tests/disabled_functions_ret_allow.phpt] 
TEST 129/190 [tests/disabled_functions_ret_allow_value.phpt]
PASS Disable functions check on `ret` allowed [tests/disabled_functions_ret_allow_value.phpt] 
TEST 130/190 [tests/disabled_functions_ret_right_hash.phpt]
PASS Disable functions [tests/disabled_functions_ret_right_hash.phpt] 
TEST 131/190 [tests/disabled_functions_ret_simulation.phpt]
PASS Disable functions check on `ret` simulation [tests/disabled_functions_ret_simulation.phpt] 
TEST 132/190 [tests/disabled_functions_ret_type.phpt]
PASS Disable functions check on `ret` by type matching (false) [tests/disabled_functions_ret_type.phpt] 
TEST 133/190 [tests/disabled_functions_ret_type_array.phpt]
PASS Disable functions check on `ret` by type matching (array). [tests/disabled_functions_ret_type_array.phpt] 
TEST 134/190 [tests/disabled_functions_ret_type_double.phpt]
PASS Disable functions check on `ret` by type matching (double). [tests/disabled_functions_ret_type_double.phpt] 
TEST 135/190 [tests/disabled_functions_ret_type_long.phpt]
PASS Disable functions check on `ret` by type matching (long). [tests/disabled_functions_ret_type_long.phpt] 
TEST 136/190 [tests/disabled_functions_ret_type_object.phpt]
XFAIL Disable functions check on `ret` by type matching (object). [tests/disabled_functions_ret_type_object.phpt]   XFAIL REASON: We're not implementing hooking on retval of user functions yet
TEST 137/190 [tests/disabled_functions_ret_type_resource.phpt]
PASS Disable functions check on `ret` by type matching (resource). [tests/disabled_functions_ret_type_resource.phpt] 
TEST 138/190 [tests/disabled_functions_ret_type_str.phpt]
PASS Disable functions check on `ret` by type matching (string). [tests/disabled_functions_ret_type_str.phpt] 
TEST 139/190 [tests/disabled_functions_ret_type_true.phpt]
PASS Disable functions check on `ret` by type matching (true). [tests/disabled_functions_ret_type_true.phpt] 
TEST 140/190 [tests/disabled_functions_ret_val.phpt]
PASS Disable functions ret val [tests/disabled_functions_ret_val.phpt] 
TEST 141/190 [tests/disabled_functions_ret_val_dump.phpt]
PASS Disable functions ret val - dump [tests/disabled_functions_ret_val_dump.phpt] 
TEST 142/190 [tests/disabled_functions_ret_val_rx.phpt]
PASS Disable functions ret val rx [tests/disabled_functions_ret_val_rx.phpt] 
TEST 143/190 [tests/disabled_functions_right_hash.phpt]
PASS Disable functions [tests/disabled_functions_right_hash.phpt] 
TEST 144/190 [tests/disabled_functions_runtime.phpt]
PASS Disable functions - runtime inclusion [tests/disabled_functions_runtime.phpt] 
TEST 145/190 [tests/disabled_functions_upper.phpt]
PASS Disable functions - uppercase [tests/disabled_functions_upper.phpt] 
TEST 146/190 [tests/disabled_functions_zero_cidr.phpt]
PASS Disable functions [tests/disabled_functions_zero_cidr.phpt] 
TEST 147/190 [tests/disabled_option.phpt]
PASS Harden rand [tests/disabled_option.phpt] 
TEST 148/190 [tests/disabled_user_functions.phpt]
PASS Disabled user-created functions [tests/disabled_user_functions.phpt] 
TEST 149/190 [tests/dump_request.phpt]
PASS Dump request [tests/dump_request.phpt] 
TEST 150/190 [tests/dump_request_invalid_folder.phpt]
PASS Dump request - invalid folder. [tests/dump_request_invalid_folder.phpt] 
TEST 151/190 [tests/dump_request_too_big.phpt]
PASS Dump request -- to big, so it's truncated. [tests/dump_request_too_big.phpt] 
TEST 152/190 [tests/dump_segfault1.phpt]
PASS Disable functions check on `ret` with an alias [tests/dump_segfault1.phpt] 
TEST 153/190 [tests/empty_conf.phpt]
PASS Empty configuration [tests/empty_conf.phpt] 
TEST 154/190 [tests/encrypt_cookies.phpt]
PASS Cookie decryption in ipv4 [tests/encrypt_cookies.phpt] 
TEST 155/190 [tests/encrypt_cookies2.phpt]
PASS Cookie encryption in ipv4 [tests/encrypt_cookies2.phpt] 
TEST 156/190 [tests/encrypt_cookies3.phpt]
PASS Cookie decryption with ipv6 [tests/encrypt_cookies3.phpt] 
TEST 157/190 [tests/encrypt_cookies4.phpt]
PASS Cookie encryption in ipv6 [tests/encrypt_cookies4.phpt] 
TEST 158/190 [tests/encrypt_cookies_empty_env.phpt]
PASS Cookie encryption - empty environment variable specified [tests/encrypt_cookies_empty_env.phpt] 
TEST 159/190 [tests/encrypt_cookies_invalid_decryption.phpt]
PASS Cookie encryption [tests/encrypt_cookies_invalid_decryption.phpt] 
TEST 160/190 [tests/encrypt_cookies_invalid_decryption2.phpt]
PASS Cookie encryption [tests/encrypt_cookies_invalid_decryption2.phpt] 
TEST 161/190 [tests/encrypt_cookies_invalid_decryption3.phpt]
PASS Cookie encryption [tests/encrypt_cookies_invalid_decryption3.phpt] 
TEST 162/190 [tests/encrypt_cookies_no_env.phpt]
PASS Cookie encryption - no environment variable specified [tests/encrypt_cookies_no_env.phpt] 
TEST 163/190 [tests/encrypt_cookies_no_key.phpt]
PASS Cookie encryption - no encryption key specified [tests/encrypt_cookies_no_key.phpt] 
TEST 164/190 [tests/encryption_key_only.phpt]
PASS Encryption key only [tests/encryption_key_only.phpt] 
TEST 165/190 [tests/example_configuration.phpt]
PASS Shipped configuration [tests/example_configuration.phpt] 
TEST 166/190 [tests/global_strict.phpt]
PASS Global strict mode [tests/global_strict.phpt] 
TEST 167/190 [tests/global_strict_disabled.phpt]
PASS Global strict mode [tests/global_strict_disabled.phpt] 
TEST 168/190 [tests/harden_mt_rand.phpt]
PASS Harden mt_rand [tests/harden_mt_rand.phpt] 
TEST 169/190 [tests/harden_rand.phpt]
PASS Harden rand [tests/harden_rand.phpt] 
TEST 170/190 [tests/harden_rand_noargs.phpt]
PASS Harden rand without any arguments [tests/harden_rand_noargs.phpt] 
TEST 171/190 [tests/inexistent_conf_file.phpt]
PASS Check for snuffleupagus presence [tests/inexistent_conf_file.phpt] 
TEST 172/190 [tests/inexistent_conf_file_list.phpt]
PASS Non-existent configuration file in a list [tests/inexistent_conf_file_list.phpt] 
TEST 173/190 [tests/loading.phpt]
PASS Check for snuffleupagus presence [tests/loading.phpt] 
TEST 174/190 [tests/multi_config.phpt]
PASS Multiple configuration files [tests/multi_config.phpt] 
TEST 175/190 [tests/noncore_function_hooking.phpt]
PASS Hooking of user-defined functions [tests/noncore_function_hooking.phpt] 
TEST 176/190 [tests/phpinfo_presence.phpt]
PASS Unserialize fail [tests/phpinfo_presence.phpt] 
TEST 177/190 [tests/samesite_cookies.phpt]
PASS Cookie samesite [tests/samesite_cookies.phpt] 
TEST 178/190 [tests/serialize.phpt]
PASS Test serialize hmac [tests/serialize.phpt] 
TEST 179/190 [tests/setcookie.phpt]
PASS Set cookies. [tests/setcookie.phpt] 
TEST 180/190 [tests/shipped_configuration.phpt]
PASS Shipped configuration [tests/shipped_configuration.phpt] 
TEST 181/190 [tests/unserialize.phpt]
PASS Unserialize ok [tests/unserialize.phpt] 
TEST 182/190 [tests/unserialize_fail.phpt]
PASS Unserialize fail [tests/unserialize_fail.phpt] 
TEST 183/190 [tests/unserialize_sim.phpt]
PASS Unserialize ok [tests/unserialize_sim.phpt] 
TEST 184/190 [tests/upload_validation.phpt]
PASS Upload a file, validation ok, no simulation [tests/upload_validation.phpt] 
TEST 185/190 [tests/upload_validation_invalid.phpt]
PASS Upload a file, invalid validation script [tests/upload_validation_invalid.phpt] 
TEST 186/190 [tests/upload_validation_ko.phpt]
PASS Upload a file, validation ko, no simulation [tests/upload_validation_ko.phpt] 
TEST 187/190 [tests/upload_validation_ko_simulation.phpt]
PASS Upload a file, validation ko, simulation [tests/upload_validation_ko_simulation.phpt] 
TEST 188/190 [tests/upload_validation_no_exec.phpt]
PASS Upload a file, validation script not executable [tests/upload_validation_no_exec.phpt] 
TEST 189/190 [tests/upload_validation_nocrash.phpt]
PASS Upload validation isn't crashing [tests/upload_validation_nocrash.phpt] 
TEST 190/190 [tests/upload_validation_ok.phpt]
PASS Upload a file, validation ok, no simulation [tests/upload_validation_ok.phpt] 
=====================================================================
TIME END 2017-12-28 21:36:17

=====================================================================
TEST RESULT SUMMARY
---------------------------------------------------------------------
Exts skipped    :    0
Exts tested     :   10
---------------------------------------------------------------------

Number of tests :  190               184
Tests skipped   :    6 (  3.2%) --------
Tests warned    :    0 (  0.0%) (  0.0%)
Tests failed    :    0 (  0.0%) (  0.0%)
Expected fail   :    3 (  1.6%) (  1.6%)
Tests passed    :  181 ( 95.3%) ( 98.4%)
---------------------------------------------------------------------
Time taken      :    2 seconds
=====================================================================

=====================================================================
EXPECTED FAILED TEST SUMMARY
---------------------------------------------------------------------
Disable functions - die [tests/disabled_functions_die.phpt]  XFAIL REASON: 
Disable functions - exit [tests/disabled_functions_exit.phpt]  XFAIL REASON: 
Disable functions check on `ret` by type matching (object). [tests/disabled_functions_ret_type_object.phpt]  XFAIL REASON: We're not implementing hooking on retval of user functions yet
=====================================================================
fichtner commented 6 years ago

The only out/diff files I could find :)

disabled_functions_die.diff.txt disabled_functions_die.out.txt disabled_functions_exit.diff.txt disabled_functions_exit.out.txt disabled_functions_ret_type_object.diff.txt disabled_functions_ret_type_object.out.txt

jvoisin commented 6 years ago 
Number of tests :  190               184
Tests skipped   :    6 (  3.2%) --------
Tests warned    :    0 (  0.0%) (  0.0%)
Tests failed    :    0 (  0.0%) (  0.0%)
Expected fail   :    3 (  1.6%) (  1.6%)
Tests passed    :  181 ( 95.3%) ( 98.4%)

This looks good to me, although the writable_executionsimulation skip is a bit suspicious. I'll investigate.

jvoisin commented 6 years ago 
PASS Readonly execution attempt [tests/deny_writable_execution.phpt] 
PASS Readonly execution attempt [tests/deny_writable_execution_disabled.phpt] 
PASS Readonly execution attempt (simulation mode) [tests/deny_writable_execution_simulation.phpt]

It's working on my linux box, weird.

jvoisin commented 6 years ago

screenshot_2018-01-02_12-01-06

Apparently, something is really wrong.

jvoisin commented 6 years ago 
root@:~/snuffleupagus/src # cat tests/deny_writable_execution_simulation.out 
[snuffleupagus][0.0.0.0][readonly_exec][simulation] Attempted execution of a writable file (/root/snuffleupagus/src/tests/deny_writable_execution_simulation.php).
[snuffleupagus][0.0.0.0][readonly_exec][simulation] Attempted execution of a writable file (/root/snuffleupagus/src/tests/writable_file.txt).
[snuffleupagus][0.0.0.0][readonly_exec][simulation] Attempted execution of a writable file (/root/snuffleupagus/src/tests/writable_file.txt).
Code execution within a writable file.
[snuffleupagus][0.0.0.0][readonly_exec][simulation] Attempted execution of a writable file (/root/snuffleupagus/src/tests/non_writable_file.txt).
[snuffleupagus][0.0.0.0][readonly_exec][simulation] Attempted execution of a writable file (/root/snuffleupagus/src/tests/non_writable_file.txt).
Code execution within a non-writable file.root@:~/snuffleupagus/src #

Hurray for ssh.

jvoisin commented 6 years ago

This is what you get when you're running the testsuite as root. @fichtner, can you confirm that you're doing this too?

jvoisin commented 6 years ago

The testsuite is failing on a significant number of tests, because the hash module isn't builtin in FreeBSD. This can be fixed by manually adding -d "extension=/usr/local/lib/php/20160303/hash.so" in the generated .sh from the testsuite.

Maybe there is a better way to fix this behaviour, I don't know.

jvoisin commented 6 years ago

I've got a confirmation from @fichtner that he was running the testsuite as root, so everything is fine on FreeBSD from our side, hurray!