Closed jfoug closed 9 years ago
frank-dittrich
commented
9 years ago When I wanted to handle the changed format labels for the first time, I intended to do it in relbench. But Solar wanted to have a separate script for this conversion, and use relbench just for the comparison.
jfoug
commented
9 years ago I think relbench works fine, once bench-unify was updated. Here is a run along with all output, after doing bench-unify on both input's. Both were done with OMP_NUM_THREADS=2 so they would run faster, and so I could more easily use my machine while they were running. NOTE, the 1.7.9-j7 was built with cygwin32 (I do not think I had cygwin64 port yet). There ARE 3 formats seen on 1.7.9-j7 but note on 1.8-bleeding. But they are formats not included any more.
$ run/relbench 179-fixed.log 180-fixed.log
More than one benchmark for NT:Raw in file 1
More than one benchmark for MSCHAPv2, C/R:Many salts in file 2
More than one benchmark for MSCHAPv2, C/R:Only one salt in file 2
More than one benchmark for netntlmv2, NTLMv2 C/R:Many salts in file 2
More than one benchmark for netntlmv2, NTLMv2 C/R:Only one salt in file 2
More than one benchmark for Raw-SHA256:Raw in file 2
More than one benchmark for Raw-SHA512:Raw in file 2
Only in file 1: Raw SHA-1 (pwlen <= 15):Raw
Only in file 1: dynamic_27:Raw
Only in file 1: dynamic_28:Raw
Only in file 2: 7z, 7-Zip:Raw
Only in file 2: Bitcoin:Raw
Only in file 2: Blockchain, My Wallet (x10):Raw
Only in file 2: Citrix_NS10, Netscaler 10:Many salts
Only in file 2: Citrix_NS10, Netscaler 10:Only one salt
Only in file 2: Clipperz, SRP:Raw
Only in file 2: EFS:Raw
Only in file 2: EncFS:Raw
Only in file 2: FormSpring:Raw
Only in file 2: Fortigate, FortiOS:Many salts
Only in file 2: Fortigate, FortiOS:Only one salt
Only in file 2: HAVAL-128-4:Raw
Only in file 2: HAVAL-256-3:Raw
Only in file 2: IKE, PSK:Raw
Only in file 2: LUKS:Raw
Only in file 2: LastPass, sniffed sessions:Raw
Only in file 2: MD2:Raw
Only in file 2: MongoDB, system / network:Raw
Only in file 2: Mozilla, Mozilla key3.db:Many salts
Only in file 2: Mozilla, Mozilla key3.db:Only one salt
Only in file 2: OpenBSD-SoftRAID (8192 iterations):Raw
Only in file 2: OpenVMS, Purdy:Raw
Only in file 2: PBKDF2-HMAC-SHA1:Raw
Only in file 2: PBKDF2-HMAC-SHA256, rounds=12000:Raw
Only in file 2: PBKDF2-HMAC-SHA512, GRUB2 / OS X 10.8+:Raw
Only in file 2: PFX, PKCS12 (.pfx, .p12):Raw
Only in file 2: PST, custom CRC-32:Raw
Only in file 2: Panama:Raw
Only in file 2: PuTTY, Private Key:Raw
Only in file 2: RAKP, IPMI 2.0 RAKP (RMCP+):Many salts
Only in file 2: RAKP, IPMI 2.0 RAKP (RMCP+):Only one salt
Only in file 2: RAR5:Raw
Only in file 2: RAdmin, v2.x:Raw
Only in file 2: Raw-Blake2:Raw
Only in file 2: Raw-Keccak:Raw
Only in file 2: Raw-Keccak-256:Raw
Only in file 2: Raw-SHA1-ng, (pwlen <= 15):Raw
Only in file 2: SSH-ng:Raw
Only in file 2: SSHA512, LDAP:Many salts
Only in file 2: SSHA512, LDAP:Only one salt
Only in file 2: STRIP, Password Manager:Raw
Only in file 2: Siemens-S7:Many salts
Only in file 2: Siemens-S7:Only one salt
Only in file 2: Snefru-128:Raw
Only in file 2: Snefru-256:Raw
Only in file 2: SunMD5:Raw
Only in file 2: Sybase-PROP:Many salts
Only in file 2: Sybase-PROP:Only one salt
Only in file 2: Tiger:Raw
Only in file 2: WoWSRP, Battlenet:Raw
Only in file 2: agilekeychain, 1Password Agile Keychain:Raw
Only in file 2: aix-smd5, AIX LPA {smd5} (modified crypt-md5):Raw
Only in file 2: aix-ssha1, AIX LPA {ssha1}:Raw
Only in file 2: aix-ssha256, AIX LPA {ssha256}:Raw
Only in file 2: aix-ssha512, AIX LPA {ssha512}:Raw
Only in file 2: asa-md5, Cisco ASA:Many salts
Only in file 2: asa-md5, Cisco ASA:Only one salt
Only in file 2: blackberry-es10:Raw
Only in file 2: chap, iSCSI CHAP authentication:Raw
Only in file 2: cloudkeychain, 1Password Cloud Keychain:Raw
Only in file 2: cq, ClearQuest:Raw
Only in file 2: dahua, "MD5 based authentication" Dahua:Many salts
Only in file 2: dahua, "MD5 based authentication" Dahua:Only one salt
Only in file 2: django-scrypt:Raw
Only in file 2: dmg, Apple DMG:Raw
Only in file 2: dynamic_39:Many salts
Only in file 2: dynamic_39:Only one salt
Only in file 2: dynamic_40:Many salts
Only in file 2: dynamic_40:Only one salt
Only in file 2: dynamic_50:Raw
Only in file 2: dynamic_51:Many salts
Only in file 2: dynamic_51:Only one salt
Only in file 2: dynamic_52:Many salts
Only in file 2: dynamic_52:Only one salt
Only in file 2: dynamic_53:Raw
Only in file 2: dynamic_54:Raw
Only in file 2: dynamic_55:Many salts
Only in file 2: dynamic_55:Only one salt
Only in file 2: dynamic_56:Many salts
Only in file 2: dynamic_56:Only one salt
Only in file 2: dynamic_57:Many salts
Only in file 2: dynamic_57:Only one salt
Only in file 2: dynamic_58:Raw
Only in file 2: dynamic_60:Raw
Only in file 2: dynamic_61:Many salts
Only in file 2: dynamic_61:Only one salt
Only in file 2: dynamic_62:Many salts
Only in file 2: dynamic_62:Only one salt
Only in file 2: dynamic_63:Raw
Only in file 2: dynamic_64:Raw
Only in file 2: dynamic_65:Many salts
Only in file 2: dynamic_65:Only one salt
Only in file 2: dynamic_66:Many salts
Only in file 2: dynamic_66:Only one salt
Only in file 2: dynamic_67:Many salts
Only in file 2: dynamic_67:Only one salt
Only in file 2: dynamic_68:Raw
Only in file 2: dynamic_70:Raw
Only in file 2: dynamic_71:Many salts
Only in file 2: dynamic_71:Only one salt
Only in file 2: dynamic_72:Many salts
Only in file 2: dynamic_72:Only one salt
Only in file 2: dynamic_73:Raw
Only in file 2: dynamic_74:Raw
Only in file 2: dynamic_75:Many salts
Only in file 2: dynamic_75:Only one salt
Only in file 2: dynamic_76:Many salts
Only in file 2: dynamic_76:Only one salt
Only in file 2: dynamic_77:Many salts
Only in file 2: dynamic_77:Only one salt
Only in file 2: dynamic_78:Raw
Only in file 2: dynamic_80:Raw
Only in file 2: dynamic_81:Many salts
Only in file 2: dynamic_81:Only one salt
Only in file 2: dynamic_82:Many salts
Only in file 2: dynamic_82:Only one salt
Only in file 2: dynamic_83:Raw
Only in file 2: dynamic_84:Raw
Only in file 2: dynamic_85:Many salts
Only in file 2: dynamic_85:Only one salt
Only in file 2: dynamic_86:Many salts
Only in file 2: dynamic_86:Only one salt
Only in file 2: dynamic_87:Many salts
Only in file 2: dynamic_87:Only one salt
Only in file 2: dynamic_88:Raw
Only in file 2: dynamic_90:Raw
Only in file 2: dynamic_91:Many salts
Only in file 2: dynamic_91:Only one salt
Only in file 2: dynamic_92:Many salts
Only in file 2: dynamic_92:Only one salt
Only in file 2: dynamic_93:Raw
Only in file 2: dynamic_94:Raw
Only in file 2: dynamic_95:Many salts
Only in file 2: dynamic_95:Only one salt
Only in file 2: dynamic_96:Many salts
Only in file 2: dynamic_96:Only one salt
Only in file 2: dynamic_97:Many salts
Only in file 2: dynamic_97:Only one salt
Only in file 2: dynamic_98:Raw
Only in file 2: dynamic_100:Raw
Only in file 2: dynamic_101:Many salts
Only in file 2: dynamic_101:Only one salt
Only in file 2: dynamic_102:Many salts
Only in file 2: dynamic_102:Only one salt
Only in file 2: dynamic_103:Raw
Only in file 2: dynamic_104:Raw
Only in file 2: dynamic_105:Many salts
Only in file 2: dynamic_105:Only one salt
Only in file 2: dynamic_106:Many salts
Only in file 2: dynamic_106:Only one salt
Only in file 2: dynamic_107:Many salts
Only in file 2: dynamic_107:Only one salt
Only in file 2: dynamic_108:Raw
Only in file 2: dynamic_110:Raw
Only in file 2: dynamic_111:Many salts
Only in file 2: dynamic_111:Only one salt
Only in file 2: dynamic_112:Many salts
Only in file 2: dynamic_112:Only one salt
Only in file 2: dynamic_113:Raw
Only in file 2: dynamic_114:Raw
Only in file 2: dynamic_115:Many salts
Only in file 2: dynamic_115:Only one salt
Only in file 2: dynamic_116:Many salts
Only in file 2: dynamic_116:Only one salt
Only in file 2: dynamic_117:Many salts
Only in file 2: dynamic_117:Only one salt
Only in file 2: dynamic_118:Raw
Only in file 2: dynamic_120:Raw
Only in file 2: dynamic_121:Many salts
Only in file 2: dynamic_121:Only one salt
Only in file 2: dynamic_122:Many salts
Only in file 2: dynamic_122:Only one salt
Only in file 2: dynamic_123:Raw
Only in file 2: dynamic_124:Raw
Only in file 2: dynamic_125:Many salts
Only in file 2: dynamic_125:Only one salt
Only in file 2: dynamic_126:Many salts
Only in file 2: dynamic_126:Only one salt
Only in file 2: dynamic_127:Many salts
Only in file 2: dynamic_127:Only one salt
Only in file 2: dynamic_128:Raw
Only in file 2: dynamic_130:Raw
Only in file 2: dynamic_131:Many salts
Only in file 2: dynamic_131:Only one salt
Only in file 2: dynamic_132:Many salts
Only in file 2: dynamic_132:Only one salt
Only in file 2: dynamic_133:Raw
Only in file 2: dynamic_134:Raw
Only in file 2: dynamic_135:Many salts
Only in file 2: dynamic_135:Only one salt
Only in file 2: dynamic_136:Many salts
Only in file 2: dynamic_136:Only one salt
Only in file 2: dynamic_137:Many salts
Only in file 2: dynamic_137:Only one salt
Only in file 2: dynamic_138:Raw
Only in file 2: dynamic_140:Raw
Only in file 2: dynamic_141:Many salts
Only in file 2: dynamic_141:Only one salt
Only in file 2: dynamic_142:Many salts
Only in file 2: dynamic_142:Only one salt
Only in file 2: dynamic_143:Raw
Only in file 2: dynamic_144:Raw
Only in file 2: dynamic_145:Many salts
Only in file 2: dynamic_145:Only one salt
Only in file 2: dynamic_146:Many salts
Only in file 2: dynamic_146:Only one salt
Only in file 2: dynamic_147:Many salts
Only in file 2: dynamic_147:Only one salt
Only in file 2: dynamic_148:Raw
Only in file 2: dynamic_150:Raw
Only in file 2: dynamic_151:Many salts
Only in file 2: dynamic_151:Only one salt
Only in file 2: dynamic_152:Many salts
Only in file 2: dynamic_152:Only one salt
Only in file 2: dynamic_153:Raw
Only in file 2: dynamic_154:Raw
Only in file 2: dynamic_155:Many salts
Only in file 2: dynamic_155:Only one salt
Only in file 2: dynamic_156:Many salts
Only in file 2: dynamic_156:Only one salt
Only in file 2: dynamic_157:Many salts
Only in file 2: dynamic_157:Only one salt
Only in file 2: dynamic_158:Raw
Only in file 2: dynamic_1011:Many salts
Only in file 2: dynamic_1011:Only one salt
Only in file 2: dynamic_1012:Many salts
Only in file 2: dynamic_1012:Only one salt
Only in file 2: dynamic_1013:Many salts
Only in file 2: dynamic_1013:Only one salt
Only in file 2: dynamic_1014:Many salts
Only in file 2: dynamic_1014:Only one salt
Only in file 2: dynamic_1015:Many salts
Only in file 2: dynamic_1015:Only one salt
Only in file 2: dynamic_1016:Many salts
Only in file 2: dynamic_1016:Only one salt
Only in file 2: dynamic_1018:Raw
Only in file 2: dynamic_1019:Raw
Only in file 2: dynamic_1020:Raw
Only in file 2: dynamic_1021:Raw
Only in file 2: dynamic_1022:Raw
Only in file 2: dynamic_1023:Raw
Only in file 2: dynamic_1024:Raw
Only in file 2: dynamic_1025:Raw
Only in file 2: dynamic_1026:Raw
Only in file 2: dynamic_1027:Raw
Only in file 2: dynamic_1028:Raw
Only in file 2: dynamic_1029:Raw
Only in file 2: dynamic_1030:Raw
Only in file 2: dynamic_1031:Raw
Only in file 2: dynamic_1300:Raw
Only in file 2: dynamic_1350:Many salts
Only in file 2: dynamic_1350:Only one salt
Only in file 2: dynamic_1400:Raw
Only in file 2: dynamic_1401:Many salts
Only in file 2: dynamic_1401:Only one salt
Only in file 2: dynamic_1501:Many salts
Only in file 2: dynamic_1501:Only one salt
Only in file 2: dynamic_1502:Many salts
Only in file 2: dynamic_1502:Only one salt
Only in file 2: dynamic_1503:Many salts
Only in file 2: dynamic_1503:Only one salt
Only in file 2: dynamic_1504:Many salts
Only in file 2: dynamic_1504:Only one salt
Only in file 2: dynamic_2000:Raw
Only in file 2: dynamic_2001:Many salts
Only in file 2: dynamic_2001:Only one salt
Only in file 2: dynamic_2002:Raw
Only in file 2: dynamic_2003:Raw
Only in file 2: dynamic_2004:Many salts
Only in file 2: dynamic_2004:Only one salt
Only in file 2: dynamic_2005:Many salts
Only in file 2: dynamic_2005:Only one salt
Only in file 2: dynamic_2006:Many salts
Only in file 2: dynamic_2006:Only one salt
Only in file 2: dynamic_2008:Many salts
Only in file 2: dynamic_2008:Only one salt
Only in file 2: dynamic_2009:Many salts
Only in file 2: dynamic_2009:Only one salt
Only in file 2: dynamic_2010:Many salts
Only in file 2: dynamic_2010:Only one salt
Only in file 2: dynamic_2011:Many salts
Only in file 2: dynamic_2011:Only one salt
Only in file 2: dynamic_2014:Many salts
Only in file 2: dynamic_2014:Only one salt
Only in file 2: eCryptfs:Raw
Only in file 2: eigrp, EIGRP MD5 authentication:Many salts
Only in file 2: eigrp, EIGRP MD5 authentication:Only one salt
Only in file 2: fde, Android FDE:Raw
Only in file 2: gpg, OpenPGP / GnuPG Secret Key:Raw
Only in file 2: hsrp, "MD5 authentication" HSRP, VRRP, GLBP:Many salts
Only in file 2: hsrp, "MD5 authentication" HSRP, VRRP, GLBP:Only one salt
Only in file 2: keyring, GNOME Keyring:Raw
Only in file 2: keystore, Java KeyStore:Raw
Only in file 2: known_hosts, HashKnownHosts HMAC-SHA1:Raw
Only in file 2: krb5-18, Kerberos 5 db etype 18 aes256-cts-hmac-sha1-96:Raw
Only in file 2: krb5pa-sha1, Kerberos 5 AS-REQ Pre-Auth etype 17/18:Raw
Only in file 2: kwallet, KDE KWallet:Raw
Only in file 2: lotus85, Lotus Notes/Domino 8.5:Raw
Only in file 2: lp, LastPass offline:Raw
Only in file 2: mdc2, MDC-2:Many salts
Only in file 2: mdc2, MDC-2:Only one salt
Only in file 2: mssql12, MS SQL 2012/2014:Many salts
Only in file 2: mssql12, MS SQL 2012/2014:Only one salt
Only in file 2: mysqlna, MySQL Network Authentication:Raw
Only in file 2: net-md5, "Keyed MD5" RIPv2, OSPF, BGP, SNMPv2:Many salts
Only in file 2: net-md5, "Keyed MD5" RIPv2, OSPF, BGP, SNMPv2:Only one salt
Only in file 2: net-sha1, "Keyed SHA1" BFD:Many salts
Only in file 2: net-sha1, "Keyed SHA1" BFD:Only one salt
Only in file 2: nk, Nuked-Klan CMS:Raw
Only in file 2: nt2, NT:Raw
Only in file 2: o5logon, Oracle O5LOGON protocol:Raw
Only in file 2: oldoffice, MS Office <= 2003:Many salts
Only in file 2: oldoffice, MS Office <= 2003:Only one salt
Only in file 2: openssl-enc, OpenSSL "enc" encryption:Raw
Only in file 2: postgres, PostgreSQL C/R:Raw
Only in file 2: ripemd-128, RIPEMD 128:Raw
Only in file 2: ripemd-160, RIPEMD 160:Raw
Only in file 2: rsvp, HMAC-MD5 / HMAC-SHA1, RSVP, IS-IS:Many salts
Only in file 2: rsvp, HMAC-MD5 / HMAC-SHA1, RSVP, IS-IS:Only one salt
Only in file 2: scrypt (16384, 8, 1):Raw
Only in file 2: sha1crypt, NetBSD's sha1crypt:Raw
Only in file 2: skein-256, Skein 256:Raw
Only in file 2: skein-512, Skein 512:Raw
Only in file 2: skey, S/Key:Raw
Only in file 2: sxc, StarOffice .sxc:Raw
Only in file 2: tc_ripemd160, TrueCrypt RIPEMD160 AES256_XTS:Raw
Only in file 2: tc_sha512, TrueCrypt SHA512 AES256_XTS:Raw
Only in file 2: tc_whirlpool, TrueCrypt WHIRLPOOL AES256_XTS:Raw
Only in file 2: tcp-md5, TCP MD5 Signatures, BGP:Many salts
Only in file 2: tcp-md5, TCP MD5 Signatures, BGP:Only one salt
Only in file 2: vtp, "MD5 based authentication" VTP:Many salts
Only in file 2: vtp, "MD5 based authentication" VTP:Only one salt
Only in file 2: whirlpool:Raw
Only in file 2: whirlpool0:Raw
Only in file 2: whirlpool1:Raw
Number of benchmarks: 211
Minimum: 0.08299 real, 0.08264 virtual
Maximum: 1969.95192 real, 3300.00000 virtual
Median: 1.57965 real, 1.25835 virtual
Median absolute deviation: 0.36293 real, 0.20025 virtual
Geometric mean: 1.98275 real, 1.55166 virtual
Geometric standard deviation: 2.53488 real, 2.66731 virtual
This script also appears to require update. Possibly we can hide need for any update by changes in the benchmark-unify script, but in any case, both of these scripts should work with the new version.