search

processone / ejabberd-contrib

Growing and curated ejabberd contributions repository - PR or ask to join !
http://ejabberd.im
250 stars 137 forks source link

Ejabberd 19.x and 20.01 does not support external HTTP auth #284

Closed adiii717 closed 4 years ago

adiii717 commented 4 years ago

Environment

Configuration

auth_method: http
auth_opts:
  host: "http://localhost:12000"
  connection_pool_size: 10
  connection_opts: []
  basic_auth: ""
  path_prefix: "/"
...

Errors from error.log/crash.log

06:30:30.437 [critical] Failed to start ejabberd application: Invalid value of option auth_method->1: unsupported database: http

Bug description

The ejabberd external HTTP auth module stop working with 19.x and the config not allow me to use auth_method:http, it crashes with the mentioned error, previously we were able to authenticate users using this module. https://github.com/processone/ejabberd-contrib/tree/master/ejabberd_auth_http Thanks

badlop commented 4 years ago

Failed to start ejabberd application: Invalid value of option auth_method->1: unsupported database: http

In general, for contributed modules, the process is: first download and install the module: $ ejabberdctl modules_update_specs $ ejabberdctl module_install ejabberd_auth_http And then enable the module in ejabberd.yml and restart ejabberd.

However, this is not enough for this ejabberd_auth_http contribution, because it is an authentication method, and requires more integration with ejabberd. Now I've noticed it requires different steps to install it, not yet documented in README.txt Please try those steps and comment if they work, then I'll update the documentation.

  1. If not already done: Download the ejabberd-contrib repository: ejabberdctl modules_update_specs
  2. Copy ejabberd_auth_http.erl and scram2.erl from $HOME/.ejabberd-modules/sources/ejabberd-contrib/ejabberd_auth_http/src/ to your ejabberd/src/
  3. Apply this small change: 
    --- a/src/ejabberd_options.erl
+++ b/src/ejabberd_options.erl
@@ -33,6 +33,8 @@
%%% API
%%%===================================================================
-spec opt_type(atom()) -> econf:validator().
+opt_type(auth_opts) ->
+    econf:any();
opt_type(access_rules) ->
 acl:validator(access_rules);
opt_type(acl) ->
@@ -472,6 +474,7 @@ options() ->
  {queue_type, ram},
  {version, ejabberd_config:version()},
  %% Other options
+     {auth_opts, []},
  {acl, []},
  {access_rules, []},
  {acme, #{}},
  4. Run "make options"
  5. Recompile ejabberd as usual, and reinstall it
  6. Now you can configure ejabberd as mentioned in the documentation
  7. Restart ejabberd
adiii717 commented 4 years ago

A already tried the first option which did not worked.


> In general, for contributed modules, the process is: first download and install the module:
> $ ejabberdctl modules_update_specs
> $ ejabberdctl module_install ejabberd_auth_http
> And then enable the module in ejabberd.yml and restart ejabberd.

It will be fine to work with 18.x, but above 18.04 I got below error.

13:35:22.610 [error] Unknown option 'auth_opts'
13:35:22.610 [error] Failed to load configuration file /home/ejabberd/conf/ejabberd.yml

BTW external auth is good to run in a production environment? seems like piping to stdin.

zinid commented 4 years ago

BTW external auth is good to run in a production environment? seems like piping to stdin.

It mostly depends on efficiency of your external authentication script/program: stdin/stdout serialization between processes is not that expensive. For example, Postfix uses the same design and nobody complains.

adiii717 commented 4 years ago

I see thanks for the clarification, but We had a discussion with Jérôme Sautret last year regarding ejabberd server optimization and He suggested us to use HTTP based auth or internal authentication instead of the external script but we can not use internal auth as we are interested to use custom authentication. As we faced some issue in past where external auth script was not responding to the ejabberd during peak time or underload. But Today I have seen some nodejs based script that also able to work with external script. https://www.npmjs.com/package/ejabberd-auth So I am still confused to for external auth or do the above workaround with ejabberd HTTP auth? Thanks

badlop commented 4 years ago

Unknown option 'auth_opts'

That's solved by the method with 6 steps that I mentioned before ;)

zinid commented 4 years ago

So I am still confused to for external auth or do the above workaround with ejabberd HTTP auth?

Then it's probably better to keep using existing solution. As @badlop told already the code can be ported to new ejabberd versions.

adiii717 commented 4 years ago

@badlop seems like the changes does not resolve the error, now getting this error

[critical] Failed to set logging: {error,
                           {handler_not_added,
                               {invalid_config,logger_std_h,
                                   #{file =>
                                         "/home/ejabberd/logs/ejabberd.log"}}}}
zinid commented 4 years ago

This is another problem. You messed with configure/compile process. What's your Erlang version?

adiii717 commented 4 years ago

Erlang/OTP 21 [erts-10.2.1] [source] [64-bit] as I mentioned I am using Docker, so this what I am using for the complication,

I have cloned the repo and did the above mentioned changes. 

COPY ejabberd ejabberd
WORKDIR /ejabberd
COPY vars.config .
COPY rel/*exs rel/
RUN mix deps.get \
 && (cd deps/eimp; ./configure)
zinid commented 4 years ago

Well your ejabberd was compiled as if you were using Erlang/OTP 22, hence this error.

adiii717 commented 4 years ago

So I need to use Erlang/OTP 22?

zinid commented 4 years ago

Yes, you can simply upgrade to the latest Erlang to get rid of this problem fast.

zinid commented 4 years ago

But don't forget to recompile ejabberd after the upgrade.

adiii717 commented 4 years ago

Note with thanks, let me try this.

KiTTYsh commented 4 years ago

With ejabberd 19.09.1 and following the instructions in this comment:

Following the steps in order, make options runs fine, and then a make afterwards results in:

src/ejabberd_option.erl:231: spec for auth_opts/0 already defined
src/ejabberd_option.erl:232: function auth_opts/0 already defined
src/ejabberd_option.erl:234: spec for auth_opts/1 already defined
src/ejabberd_option.erl:235: function auth_opts/1 already defined
src/ejabberd_option.erl:19: Warning: function auth_opts/0 already exported
src/ejabberd_option.erl:19: Warning: function auth_opts/1 already exported
Compiling src/ejabberd_option.erl failed:
ERROR: compile failed while processing /srv/ejabberd/ejabberd-19.09.1: rebar_abort
make: *** [Makefile:112: src] Error 1

Copying the .erl files, running make options, patching src/ejabberd_options.erl, then running make; make install works fine, but upon attempted authentication:

03:57:24.138 [error] gen_server <0.692.0> terminated with reason: no case clause matching true in ejabberd_auth:'-db_check_password/7-fun-0-'/5 line 661
03:57:24.138 [error] CRASH REPORT Process <0.692.0> with 0 neighbours exited with reason: no case clause matching true in ejabberd_auth:'-db_check_password/7-fun-0-'/5 line 661 in p1_server:terminate/7 line 878
03:57:24.139 [error] Supervisor ejabberd_c2s_sup had child undefined started with {ejabberd_c2s,start_link,undefined} at <0.692.0> exit with reason no case clause matching true in ejabberd_auth:'-db_check_password/7-fun-0-'/5 line 661 in context child_terminated

Issues are also present in ejabberd 20.01. Both on Erlang/OTP 22.

badlop commented 4 years ago

You are right, ejabberd considers opt_type only when it is in ejabberd_options, and in fact complains when it's double defined. So I've removed it from ejabberd_auth_http.erl and updated the patch. In summary, the updated steps are:

  1. Download or update the ejabberd-contrib repository: ejabberdctl modules_update_specs
  2. Copy ejabberd_auth_http.erl and scram2.erl from $HOME/.ejabberd-modules/sources/ejabberd-contrib/ejabberd_auth_http/src/ to your ejabberd/src/
  3. Apply this patch to ejabberd source code: 
    diff --git a/src/ejabberd_options.erl b/src/ejabberd_options.erl
index 3af793a91..e8b8cb890 100644
--- a/src/ejabberd_options.erl
+++ b/src/ejabberd_options.erl
@@ -62,6 +62,21 @@ opt_type(auth_cache_size) ->
 econf:pos_int(infinity);
opt_type(auth_method) ->
 econf:list_or_single(econf:db_type(ejabberd_auth));
+opt_type(auth_opts) ->
+    fun(L) when is_list(L) ->
+            lists:map(
+              fun({host, V}) when is_binary(V) ->
+                      {host, V};
+                 ({connection_pool_size, V}) when is_integer(V) ->
+                      {connection_pool_size, V};
+                 ({connection_opts, V}) when is_list(V) ->
+                      {connection_opts, V};
+                 ({basic_auth, V}) when is_binary(V) ->
+                      {basic_auth, V};
+                 ({path_prefix, V}) when is_binary(V) ->
+                      {path_prefix, V}
+              end, L)
+    end;
opt_type(auth_password_format) ->
 econf:enum([plain, scram]);
opt_type(auth_use_cache) ->
@@ -443,6 +458,7 @@ opt_type(jwt_auth_only_rule) ->
        {disable_sasl_mechanisms, [binary()]} |
        {s2s_zlib, boolean()} |
        {loglevel, ejabberd_logger:loglevel()} |
+           {auth_opts, [{any(), any()}]} |
        {listen, [ejabberd_listener:listener()]} |
        {modules, [{module(), gen_mod:opts(), integer()}]} |
        {ldap_uids, [{binary(), binary()}]} |
@@ -493,6 +509,7 @@ options() ->
   fun(Host) -> ejabberd_config:get_option({cache_size, Host}) end},
  {auth_method,
   fun(Host) -> [ejabberd_config:default_db(Host, ejabberd_auth)] end},
+     {auth_opts, []},
  {auth_password_format, plain},
  {auth_use_cache,
   fun(Host) -> ejabberd_config:get_option({use_cache, Host}) end},
  4. Run "make options"
  5. Recompile ejabberd as usual, and reinstall it
  6. Now you can configure ejabberd as mentioned in the documentation
  7. Restart ejabberd
adiii717 commented 4 years ago

I was able to make the above changes and compiled the ejabberd, now its crash with this error @zinid my EJabberd version is 20.01 and Erlang version info 

Erlang/OTP 22 [erts-10.4.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

here is the Error logs

2020-02-06 09:15:35.257134+00:00 [critical] Failed to start ejabberd application: Configuration error: Duplicated option: auth_method

whenever I use auth_method: http

auth_method: http
auth_opts:
  host: "http://localhost:3000"
  connection_pool_size: 10
  connection_opts: []
  basic_auth: ""
  path_prefix: "/"

When I removed ejabberd node modules it throw this err

2020-02-06 09:17:33.088428+00:00 [error] SUPERVISOR REPORT:
    supervisor: {local,ejabberd_http_sup}
    errorContext: child_terminated
    reason: {undef,
                [{cuesport,get_worker,[ejabberd_auth_http_localhost],[]},
                 {ejabberd_auth_http,make_req,5,
                     [{file,"src/ejabberd_auth_http.erl"},{line,203}]},
                 {ejabberd_auth_http,check_password,4,
                     [{file,"src/ejabberd_auth_http.erl"},{line,63}]},
                 {ejabberd_auth,'-db_check_password/7-fun-0-',5,
                     [{file,"src/ejabberd_auth.erl"},{line,661}]},
                 {ets_cache,update,5,
                     [{file,"/ejabberd/deps/cache_tab/src/ets_cache.erl"},
                      {line,208}]},
                 {ejabberd_auth,db_check_password,7,
                     [{file,"src/ejabberd_auth.erl"},{line,658}]},
                 {ejabberd_auth,'-check_password_with_authmodule/6-fun-0-',8,
                     [{file,"src/ejabberd_auth.erl"},{line,243}]},
                 {lists,foldl,3,[{file,"lists.erl"},{line,1263}]}]}
    offender: [{pid,<0.761.0>},
               {id,undefined},
               {mfargs,{ejabberd_http,start_link,undefined}},
               {restart_type,temporary},
               {shutdown,5000}, 
               {child_type,worker}]
adiii717 commented 4 years ago

Failed to start ejabberd application: Invalid value of option auth_method->1: unsupported database: http

In general, for contributed modules, the process is: first download and install the module: $ ejabberdctl modules_update_specs $ ejabberdctl module_install ejabberd_auth_http And then enable the module in ejabberd.yml and restart ejabberd.

However, this is not enough for this ejabberd_auth_http contribution, because it is an authentication method, and requires more integration with ejabberd. Now I've noticed it requires different steps to install it, not yet documented in README.txt Please try those steps and comment if they work, then I'll update the documentation.

  1. If not already done: Download the ejabberd-contrib repository: ejabberdctl modules_update_specs
  2. Copy ejabberd_auth_http.erl and scram2.erl from $HOME/.ejabberd-modules/sources/ejabberd-contrib/ejabberd_auth_http/src/ to your ejabberd/src/
  3. Apply this small change:
--- a/src/ejabberd_options.erl
+++ b/src/ejabberd_options.erl
@@ -33,6 +33,8 @@
 %%% API
 %%%===================================================================
 -spec opt_type(atom()) -> econf:validator().
+opt_type(auth_opts) ->
+    econf:any();
 opt_type(access_rules) ->
     acl:validator(access_rules);
 opt_type(acl) ->
@@ -472,6 +474,7 @@ options() ->
      {queue_type, ram},
      {version, ejabberd_config:version()},
      %% Other options
+     {auth_opts, []},
      {acl, []},
      {access_rules, []},
      {acme, #{}},
  1. Run "make options"
  2. Recompile ejabberd as usual, and reinstall it
  3. Now you can configure ejabberd as mentioned in the documentation
  4. Restart ejabberd

with this option, we also getting this error

[critical] Failed to start ejabberd application: Configuration error: Duplicated option: auth_method
licaon-kter commented 4 years ago

@Adiii717 Do you have multiple definitions in your .yml or includes?

adiii717 commented 4 years ago

No, here is my HTTP auth config

auth_method: http
auth_opts:
  host: "http://localhost:3000"
  connection_pool_size: 10
  connection_opts: []
  basic_auth: ""
  path_prefix: "/"
badlop commented 4 years ago

whenever I use auth_method: http

The error message clearly indicates that the option auth_method is defined twice.

Maybe you define once in ejabberd.yml, and the other definition is in $HOME/.ejabberd-modules/ejabberd_auth_http/conf/ejabberd_auth_http.yml ? You can remove that conf file, as nowadays you configure it directly in the main ejabberd.yml

adiii717 commented 4 years ago

Thanks, but now getting this error when try to login

 [error] SUPERVISOR REPORT:
    supervisor: {local,ejabberd_http_sup}
    errorContext: child_terminated
    reason: {badarg,
                [{ets,lookup,[ejabberd_auth_http_localhost,pool_size],[]},
                 {cuesport,get_worker,1,
                     [{file,
                          "/home/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/ejabberd_auth_http/deps/cuesport/src/cuesport.erl"},
                      {line,55}]},
                 {ejabberd_auth_http,make_req,5,
                     [{file,"src/ejabberd_auth_http.erl"},{line,203}]},
                 {ejabberd_auth_http,check_password,4,
                     [{file,"src/ejabberd_auth_http.erl"},{line,63}]},
                 {ejabberd_auth,'-db_check_password/7-fun-0-',5,
                     [{file,"src/ejabberd_auth.erl"},{line,661}]},
                 {ets_cache,update,5,
                     [{file,"/ejabberd/deps/cache_tab/src/ets_cache.erl"},
                      {line,208}]},
                 {ejabberd_auth,db_check_password,7,
                     [{file,"src/ejabberd_auth.erl"},{line,658}]},
                 {ejabberd_auth,'-check_password_with_authmodule/6-fun-0-',8,
                     [{file,"src/ejabberd_auth.erl"},{line,243}]}]}
    offender: [{pid,<0.842.0>},
               {id,undefined},
               {mfargs,{ejabberd_http,start_link,undefined}},
               {restart_type,temporary},
               {shutdown,5000},
               {child_type,worker}]
badlop commented 4 years ago

Ok, it's taking some time, but it seems this time we're closer to the final instructions. Starting from the beginning:

  1. Download or update the ejabberd-contrib repository: ejabberdctl modules_update_specs
  2. Compile and install the module and its dependencies: ejabberdctl module_install ejabberd_auth_http
  3. Configure the module in your main ejabberd.yml probably with something like this: 
    ## auth_method: internal
auth_method: http
auth_opts:
host: "http://localhost:3000"
connection_pool_size: 10
connection_opts: []
basic_auth: ""
path_prefix: "/"
  4. Remove the small configuration file here, which has duplicated options: $HOME/.ejabberd-modules/ejabberd_auth_http/conf/*
  5. Apply this patch to ejabberd source code: 
    diff --git a/src/ejabberd_options.erl b/src/ejabberd_options.erl
index 3af793a91..e8b8cb890 100644
--- a/src/ejabberd_options.erl
+++ b/src/ejabberd_options.erl
@@ -62,6 +62,21 @@ opt_type(auth_cache_size) ->
 econf:pos_int(infinity);
opt_type(auth_method) ->
 econf:list_or_single(econf:db_type(ejabberd_auth));
+opt_type(auth_opts) ->
+    fun(L) when is_list(L) ->
+            lists:map(
+              fun({host, V}) when is_binary(V) ->
+                      {host, V};
+                 ({connection_pool_size, V}) when is_integer(V) ->
+                      {connection_pool_size, V};
+                 ({connection_opts, V}) when is_list(V) ->
+                      {connection_opts, V};
+                 ({basic_auth, V}) when is_binary(V) ->
+                      {basic_auth, V};
+                 ({path_prefix, V}) when is_binary(V) ->
+                      {path_prefix, V}
+              end, L)
+    end;
opt_type(auth_password_format) ->
 econf:enum([plain, scram]);
opt_type(auth_use_cache) ->
@@ -443,6 +458,7 @@ opt_type(jwt_auth_only_rule) ->
        {disable_sasl_mechanisms, [binary()]} |
        {s2s_zlib, boolean()} |
        {loglevel, ejabberd_logger:loglevel()} |
+           {auth_opts, [{any(), any()}]} |
        {listen, [ejabberd_listener:listener()]} |
        {modules, [{module(), gen_mod:opts(), integer()}]} |
        {ldap_uids, [{binary(), binary()}]} |
@@ -493,6 +509,7 @@ options() ->
   fun(Host) -> ejabberd_config:get_option({cache_size, Host}) end},
  {auth_method,
   fun(Host) -> [ejabberd_config:default_db(Host, ejabberd_auth)] end},
+     {auth_opts, []},
  {auth_password_format, plain},
  {auth_use_cache,
   fun(Host) -> ejabberd_config:get_option({use_cache, Host}) end},
  6. Run "make options"
  7. Recompile ejabberd, and reinstall it
  8. Restart ejabberd
adiii717 commented 4 years ago

again crash during login, but in this case, the auth server at least received the call, but ejabberd crash with this error

2020-02-07 18:50:21.975556+00:00 [info] (<0.886.0>) Accepted connection [::ffff:172.17.0.1]:33436 -> [::ffff:172.17.0.5]:5280
2020-02-07 18:50:21.980539+00:00 [error] CRASH REPORT:
  crasher:
    initial call: ejabberd_http:init/3
    pid: <0.886.0>
    registered_name: []
    exception error: no case clause matching true
      in function  ejabberd_auth:'-db_check_password/7-fun-0-'/5 (src/ejabberd_auth.erl, line 661)
      in call from ets_cache:update/5 (/ejabberd/deps/cache_tab/src/ets_cache.erl, line 208)
      in call from ejabberd_auth:db_check_password/7 (src/ejabberd_auth.erl, line 658)
      in call from ejabberd_auth:'-check_password_with_authmodule/6-fun-0-'/8 (src/ejabberd_auth.erl, line 243)
      in call from lists:foldl/3 (lists.erl, line 1263)
      in call from ejabberd_auth:check_password_with_authmodule/6 (src/ejabberd_auth.erl, line 241)
      in call from ejabberd_auth:check_password/6 (src/ejabberd_auth.erl, line 218)
      in call from ejabberd_web_admin:get_auth_account2/5 (src/ejabberd_web_admin.erl, line 251)
    ancestors: [ejabberd_http_sup,ejabberd_sup,<0.130.0>]
    message_queue_len: 0
    messages: []
    links: [<0.846.0>,#Port<0.45>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 1598
    stack_size: 27
    reductions: 3152
  neighbours:

2020-02-07 18:50:21.983956+00:00 [error] SUPERVISOR REPORT:
    supervisor: {local,ejabberd_http_sup}
    errorContext: child_terminated
    reason: {{case_clause,true},
             [{ejabberd_auth,'-db_check_password/7-fun-0-',5,
                             [{file,"src/ejabberd_auth.erl"},{line,661}]},
              {ets_cache,update,5,
                         [{file,"/ejabberd/deps/cache_tab/src/ets_cache.erl"},
                          {line,208}]},
              {ejabberd_auth,db_check_password,7,
                             [{file,"src/ejabberd_auth.erl"},{line,658}]},
              {ejabberd_auth,'-check_password_with_authmodule/6-fun-0-',8,
                             [{file,"src/ejabberd_auth.erl"},{line,243}]},
              {lists,foldl,3,[{file,"lists.erl"},{line,1263}]},
              {ejabberd_auth,check_password_with_authmodule,6,
                             [{file,"src/ejabberd_auth.erl"},{line,241}]},
              {ejabberd_auth,check_password,6,
                             [{file,"src/ejabberd_auth.erl"},{line,218}]},
              {ejabberd_web_admin,get_auth_account2,5,
                                  [{file,"src/ejabberd_web_admin.erl"},
                                   {line,251}]}]}
    offender: [{pid,<0.886.0>},
               {id,undefined},
               {mfargs,{ejabberd_http,start_link,undefined}},
               {restart_type,temporary},
               {shutdown,5000},
               {child_type,worker}]
ghost commented 4 years ago

I had the same problem
ejabberd version 19.09.1

badlop commented 4 years ago

I ran Dialyzer, which reports inconsistencies between function calls, and it reported several problems in ejabberd_auth_http.erl when used with recent ejabberd. I've fixed most of them and committed to ejabberd-contrib. So, that is another step forward.

Please update the module: remove it reverting step 2, then repeat steps 1, 2, 4 and 8. Let's see what results in your new tests.

The bad news is that I don't have any server where I can test the module in practice...

badlop commented 4 years ago

Thanks to your help, it seems the module now finally can be compiled and installed correctly. So, I've appplied the changes to ejabberd.

From now on, if you find problems in the module, please open a new ticket to investigate them.