PBKDF2 / mosquitto-auth-plug password hash compatibility question

[coldfire84]

I'm looking to migrate my mosquitto/ mosquitto-auth-plug setup, using MongoDB for authentication. Ideally, I want to migrate across without having to get 1200+ users to reset their passwords.

The existing MQTT authentication plugin is very specific about password format, example from mosquitto-auth-plug docs below:

PBKDF2$sha256$901$8ebTR72Pcmjl3cYq$SCVHHfqn9t6Ev9sE6RMTeF3pawvtGqTu
--^--- --^--- -^- ------^--------- -------------^------------------
  |      |     |        |                       |
  |      |     |        |                       +-- : hashed password
  |      |     |        +-------------------------- : salt
  |      |     +----------------------------------- : iterations
  |      +----------------------------------------- : hash function
  +------------------------------------------------ : marker

Password hashing on my back-end is configured to use sha256, with the following additional relevant settings:

saltlen: 12
keylen: 24
iterations: 901
encoding: 'base64' (specifies the encoding the generated salt and hash will be stored in)

I've configured my EMQX docker container as follows:

-e EMQX_AUTH__MONGO__AUTH_QUERY__PASSWORD_HASH="pbkdf2,sha256,901,24"

I'm unsure whether the encoding is causing issues, being set to base64? As per passport-local-mongoose config docs.

To date I cannot get the EMQX broker to authenticate any of my users. A super user account generates the following error in the emqx logs when trying to connect:

2020-01-12 17:57:17.871 [warning] <<"test">>@127.0.0.1:55836 [Channel] Client test (Username: 'username') login failed for not_authorized

A non-super user account causes an error (sanitized with IP/ username/password removed):

2020-01-12 18:12:52.971 [error] <<"test">>@127.0.0.1:57612 [Hooks] Failed to execute {fun emqx_auth_mongo:check/3,
                           [#{authquery =>
                                  {authquery,<<"accounts">>,
                                      [<<"password">>],
                                      {pbkdf2,sha256,901,24},
                                      [{<<"username">>,<<"%u">>}]},
                              superquery =>
                                  {superquery,<<"accounts">>,<<"superuser">>,
                                      [{<<"username">>,<<"%u">>}]}}]}([#{clientid =>
                                                                             <<"test">>,
                                                                         is_bridge =>
                                                                             false,
                                                                         is_superuser =>
                                                                             false,
                                                                         mountpoint =>
                                                                             undefined,
                                                                         password =>
                                                                             <<"password">>,
                                                                         peercert =>
                                                                             nossl,
                                                                         peerhost =>
                                                                             {127,
                                                                              0,
                                                                              0,
                                                                              1},
                                                                         protocol =>
                                                                             mqtt,
                                                                         sockport =>
                                                                             1883,
                                                                         username =>
                                                                             <<"myusername">>,
                                                                         zone =>
                                                                             external},
                                                                       #{anonymous =>
                                                                             false,
                                                                         auth_result =>
                                                                             not_authorized}]): {function_clause,
                                                                                                 [{emqx_passwd,
                                                                                                   hash,
                                                                                                   [{pbkdf2,
                                                                                                     sha256,
                                                                                                     901,
                                                                                                     24},
                                                                                                    <<"password">>],
                                                                                                   [{file,
                                                                                                     "/emqx_rel/_build/emqx/lib/emqx_passwd/src/emqx_passwd.erl"},
                                                                                                    {line,
                                                                                                     53}]},
                                                                                                  {emqx_passwd,
                                                                                                   check_pass,
                                                                                                   2,
                                                                                                   [{file,
                                                                                                     "/emqx_rel/_build/emqx/lib/emqx_passwd/src/emqx_passwd.erl"},
                                                                                                    {line,
                                                                                                     38}]},
                                                                                                  {emqx_auth_mongo,
                                                                                                   check_pass,
                                                                                                   2,
                                                                                                   [{file,
                                                                                                     "/emqx_rel/_build/emqx/lib/emqx_auth_mongo/src/emqx_auth_mongo.erl"},
                                                                                                    {line,
                                                                                                     70}]},
                                                                                                  {emqx_auth_mongo,
                                                                                                   check,
                                                                                                   3,
                                                                                                   [{file,
                                                                                                     "/emqx_rel/_build/emqx/lib/emqx_auth_mongo/src/emqx_auth_mongo.erl"},
                                                                                                    {line,
                                                                                                     52}]},
                                                                                                  {emqx_hooks,
                                                                                                   safe_execute,
                                                                                                   2,
                                                                                                   [{file,
                                                                                                     "/emqx_rel/_build/emqx/lib/emqx/src/emqx_hooks.erl"},
                                                                                                    {line,
                                                                                                     164}]},
                                                                                                  {emqx_hooks,
                                                                                                   do_run_fold,
                                                                                                   3,
                                                                                                   [{file,
                                                                                                     "/emqx_rel/_build/emqx/lib/emqx/src/emqx_hooks.erl"},
                                                                                                    {line,
                                                                                                     143}]},
                                                                                                  {emqx_access_control,
                                                                                                   authenticate,
                                                                                                   1,
                                                                                                   [{file,
                                                                                                     "/emqx_rel/_build/emqx/lib/emqx/src/emqx_access_control.erl"},
                                                                                                    {line,
                                                                                                     82}]},
                                                                                                  {emqx_channel,
                                                                                                   auth_connect,
                                                                                                   2,
                                                                                                   [{file,
                                                                                                     "/emqx_rel/_build/emqx/lib/emqx/src/emqx_channel.erl"},
                                                                                                    {line,
                                                                                                     1022}]}]}

Any help greatly appreciated.

[HJianBo]

Hi, @coldfire84 Thanks for your report. We will try to reproduce it

[coldfire84]

I'm unsure whether the encoding is causing issues, being set to base64? As per passport-local-mongoose config docs.

After extra testing I can confirm that having password hash/ salt stored as 'hex' encoded does not fix the issue. Passport-local mongoose config (note hex encoding is used by default if an alternate is not supplied):

saltlen: 32
keylen: 512
digestAlgorithm: 'sha512'
iterations: 25000

EMQX config:

-e EMQX_AUTH__MONGO__AUTH_QUERY__PASSWORD_HASH="pbkdf2,sha512,25000,512"

[HJianBo]

Hi, @coldfire84 Sorry to tell you that you have to migrate your password data if you switch from mosquitto to emqx

The emqx_auth_mongo saved password format like this:

{"username" : "usera", "password" : "ec7860ccaddf01ab92a6fb65b088fef5b6f8227ac5e00932", "salt" : "ssalt" }

• password field is an encrypted password saved with hex string
• salt field is a plain string only

And the emqx_auth_mongo.conf related options should be:

auth.mongo.auth_query.password_field = password,salt
auth.mongo.auth_query.password_hash = pbkdf2,sha256,901,24

And the MQTT client should using the plain password connecting to the broker, i.g:

mosquitto_sub -u usera -P abcde -t t -d

[coldfire84]

Sorry to tell you that you have to migrate your password data if you switch from mosquitto to emqx

Thanks for confirming.

salt field is a plain string only

I'd missed this :( - I'll have a look at the auth code and see if I can put together a PR for encoded salt based upon an option.

Any ideas why this generated such a large error in the logs?

[HJianBo]

Any ideas why this generated such a large error in the logs?

This error should be caused by emqx_auth_mong.conf options configured error. We will supplement the documentation to show these options

[Ekion-1]

I'm running into the same issue here but with the MySQL authentication. @coldfire84 did you ever figure out a way to do this without having all of the users reset their passwords?

[coldfire84]

@AlexGodbehere , I never figured it out. I adopted a different (and actively developed) Mosquitto Auto Plugin, so didn't have to change brokers in the end.

[ihmgsm]

Just figured out for Postgre SQL SELECT encode(decode(substring(pw, 36, 32), 'base64'), 'hex') AS password_hash, substring(pw, 19, 16) AS salt FROM mqtt_users where username = ${username} LIMIT 1

[FernandoGarcia]

Hi @HJianBo !

I'm facing similar issue when trying to move from mosquitto-auth-plug to emqx.

Could you give me some help to create the SQL query for this purpose?

The hashed password is created in this way using PHP.

define("PBKDF2_HASH_ALGORITHM", "sha256");
define("PBKDF2_ITERATIONS", 901);
define("PBKDF2_SALT_BYTE_SIZE", 12);
define("PBKDF2_HASH_BYTE_SIZE", 24);
define("SEPARATOR", "$");
define("TAG", "PBKDF2");

function create_hash($password) {

    $salt = base64_encode(mcrypt_create_iv(PBKDF2_SALT_BYTE_SIZE, MCRYPT_DEV_URANDOM));
    return TAG . SEPARATOR . PBKDF2_HASH_ALGORITHM . SEPARATOR . PBKDF2_ITERATIONS . SEPARATOR .  $salt . SEPARATOR . 
        base64_encode(pbkdf2(
            PBKDF2_HASH_ALGORITHM,
            $password,
            $salt,
            PBKDF2_ITERATIONS,
            PBKDF2_HASH_BYTE_SIZE,
            true
        ));
}

function pbkdf2($algorithm, $password, $salt, $count, $key_length, $raw_output = false) {
    $algorithm = strtolower($algorithm);
    if(!in_array($algorithm, hash_algos(), true))
        trigger_error('PBKDF2 ERROR: Invalid hash algorithm.', E_USER_ERROR);
    if($count <= 0 || $key_length <= 0)
        trigger_error('PBKDF2 ERROR: Invalid parameters.', E_USER_ERROR);
    if (function_exists("hash_pbkdf2")) {
        // The output length is in NIBBLES (4-bits) if $raw_output is false!
        if (!$raw_output) {
            $key_length = $key_length * 2;
        }
        return hash_pbkdf2($algorithm, $password, $salt, $count, $key_length, $raw_output);
    }
    $hash_length = strlen(hash($algorithm, "", true));
    $block_count = ceil($key_length / $hash_length);
    $output = "";
    for($i = 1; $i <= $block_count; $i++) {
        // $i encoded as 4 bytes, big endian.
        $last = $salt . pack("N", $i);
        // first iteration
        $last = $xorsum = hash_hmac($algorithm, $last, $password, true);
        // perform the other $count - 1 iterations
        for ($j = 1; $j < $count; $j++) {
            $xorsum ^= ($last = hash_hmac($algorithm, $last, $password, true));
        }
        $output .= $xorsum;
    }
    if($raw_output)
        return substr($output, 0, $key_length);
    else
        return bin2hex(substr($output, 0, $key_length));
}

This issues seem related https://github.com/emqx/emqx/issues/1396 https://github.com/emqx/emqx/issues/1394

I tried the SQL query suggested above and in this issues without success.

I'm using MySQL.

Best regards.

[FernandoGarcia]

Hi @zhongwencool! Could you give me some help with this issue? Thanks in advance.