HMAC_MD5 bad results

[dominicgermain]

First, the documentation state that the default for new is "ALGOS.MD5"... in the code, line 123, there is a mistake... the default is "hashes.md5", should be "hashes.MD5"...

Pretty easy to fix.

Also, trying to use your module with NGINX to build a pop3 proxy... Trying to implement cram-md5 auth.

All Perl examples are able to compute the hash the same way Outlook and other mail clients are... but in LUA, I'm always getting a different result... don't know why...

Sample in Perl :

use Digest::HMAC_MD5;

my $clear = 'mypass';
my $salt = 'mysalt';
my $expected = '0c88761d58df73064d402f6774961030';

print("clear : $clear\n");
print("salt : $salt\n");
print("------------------------------------------------------------------\n");
print("expected : $expected\n");
print("got      : " . Digest::HMAC_MD5::hmac_md5_hex($salt, $clear) . "\n");

Perl output :

clear : mypass
salt : mysalt
------------------------------------------------------------------
expected : 0c88761d58df73064d402f6774961030
got      : 0c88761d58df73064d402f6774961030

LUA script under OpenResty :

local clear = "mypass"
local salt = "mysalt"
local expected = "0c88761d58df73064d402f6774961030"

ngx.say('clear : ', clear)
ngx.say('salt : ', salt)

local hmac = require "resty.hmac"

ngx.say("------------------------------------------------------------------")
ngx.say("expected : ", expected)

--bad result
local hmac_md5 = hmac:new("", hmac.ALGOS.MD5)
ngx.say("got      : ", hmac_md5:final(salt..clear, true))
hmac_md5:reset()

--bad result
local hmac_md5 = hmac:new(salt, hmac.ALGOS.MD5)
ngx.say("got      : ", hmac_md5:final(clear, true))
hmac_md5:reset()

--bad result
local hmac_md5 = hmac:new(salt..clear, hmac.ALGOS.MD5)
ngx.say("got      : ", hmac_md5:final(nil, true))
hmac_md5:reset()

LUA output :

clear : mypass
salt : mysalt
------------------------------------------------------------------
expected : 0c88761d58df73064d402f6774961030
got      : 7e3cedfb7b506b629c1165093753d31f
got      : 0ad5db465c62e82de746ae458a4799b3
got      : b13232a517d6532733f781647130bdbc

This scripts are using a "dummy salt", but I got the same bad result using the "real" salt and password capture from a mail client conversation... What's wrong?

Note that I can easily reproduce your test result when using SHA1... I guest that my environnement is OK. Seems to be MD5 only issue...

My environnement

#resty -v
resty 0.20
nginx version: openresty/1.13.6.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) 
built with OpenSSL 1.0.2k-fips  26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt=-O2 --add-module=../ngx_devel_kit-0.3.0 --add-module=../echo-nginx-module-0.61 --add-module=../xss-nginx-module-0.05 --add-module=../ngx_coolkit-0.2rc3 --add-module=../set-misc-nginx-module-0.31 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.07 --add-module=../srcache-nginx-module-0.31 --add-module=../ngx_lua-0.10.11 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.18 --add-module=../redis2-nginx-module-0.14 --add-module=../redis-nginx-module-0.3.7 --add-module=../rds-json-nginx-module-0.15 --add-module=../rds-csv-nginx-module-0.08 --add-module=../ngx_stream_lua-0.0.3 --with-ld-opt=-Wl,-rpath,/usr/local/openresty/luajit/lib --with-mail --with-mail_ssl_module --with-stream --with-stream_ssl_module --with-http_ssl_module

[jkeys089]

It looks to me like it is probably caused by the order of arguments. The perl module expects hmac_md5_hex($data, $key) but your example code has those swapped.

[dominicgermain]

you're right!!!

doing so give the same result as Perl

local hmac_md5 = hmac:new(clear, hmac.ALGOS.MD5)
ngx.say("got      : ", hmac_md5:final(salt, true))
hmac_md5:reset()

I've try everything about "where to put params", but nothing about the order.

thanks!

[dominicgermain]

FYI, I just implement CRAM-MD5 alongside with APOP support and it works great with Apple Mail and Outlook 👍

Just to give back... part of my LUA code :

-- work with major email clients (Apple Mail and Outlook)

--get params from nginx... 

if authmethod == 'plain' then
    computedpass = clear

elseif authmethod == 'apop' then
    local str = require "resty.string"
    local resty_md5 = require "resty.md5"
    local md5 = resty_md5:new()
    md5:update(authsalt)
    md5:update(clear)
    computedpass = str.to_hex(md5:final())

elseif authmethod == 'cram-md5' then
    local hmac = require "resty.hmac"
    local hmac_md5 = hmac:new(clear, hmac.ALGOS.MD5)
    computedpass = hmac_md5:final(authsalt, true)

else
    ngx.header["Auth-Status"] = string.format("Unsupported authentication method [%s/%s]", authmethod, user)
    return
end

--- compare computedpass to providedpass and take appropriate actions...