search

processone / ejabberd

Robust, Ubiquitous and Massively Scalable Messaging Platform (XMPP, MQTT, SIP Server)
https://www.process-one.net/ejabberd/
Other
6.12k stars 1.51k forks source link

ejabberdctl request_certificate crashes #4267

Closed svbergerem closed 1 month ago

svbergerem commented 3 months ago

Environment

Errors from error.log/crash.log

12:36:38.730 [error] GenServer :ejabberd_acme terminating
** (stop) {:invalid_ejson, {"crv", "P-256"}}
    (jiffy 1.1.2) /build/ejabberd/src/ejabberd-24.07/deps/jiffy/src/jiffy.erl:91: :jiffy.encode/2
    (jose 1.11.10) src/json/jose_json_jiffy.erl:26: :jose_json_jiffy.encode/1
    (jose 1.11.10) src/jws/jose_jws.erl:134: :jose_jws.to_binary/1
    (jose 1.11.10) src/jws/jose_jws.erl:307: :jose_jws.sign/4
    (p1_acme 1.0.23) /build/ejabberd/src/ejabberd-24.07/deps/p1_acme/src/p1_acme.erl:923: :p1_acme.jose_json/3
    (p1_acme 1.0.23) /build/ejabberd/src/ejabberd-24.07/deps/p1_acme/src/p1_acme.erl:286: anonymous fn/1 in :p1_acme.request_new_account/1
    (p1_acme 1.0.23) /build/ejabberd/src/ejabberd-24.07/deps/p1_acme/src/p1_acme.erl:614: :p1_acme.http_request/3
    (p1_acme 1.0.23) /build/ejabberd/src/ejabberd-24.07/deps/p1_acme/src/p1_acme.erl:290: :p1_acme.request_new_account/1
Last message: {:"$gen_cast", {:request, ["conference.example.org", "example.org", "proxy.xmpp.example.org", "pubsub.xmpp.example.org", "upload.example.org"]}}

(replaced my domain by example.org)

Bug description

In addition to the error above, ejabberdctl request_certificate all (as well as ejabberdctl request_certificate "example.org") crashes as well:

Unhandled exception occurred executing the command:
** exception exit: {{{invalid_ejson,{<<"crv">>,<<"P-256">>}},
                   [{jiffy,encode,2,
                        [{file,
                             "/build/ejabberd/src/ejabberd-24.07/deps/jiffy/src/jiffy.erl"},
                         {line,91}]},
                    {jose_json_jiffy,encode,1,
                        [{file,"src/json/jose_json_jiffy.erl"},{line,26}]},
                    {jose_jws,to_binary,1,
                        [{file,"src/jws/jose_jws.erl"},{line,134}]},
                    {jose_jws,sign,4,
                        [{file,"src/jws/jose_jws.erl"},{line,307}]},
                    {p1_acme,jose_json,3,
                        [{file,
                             "/build/ejabberd/src/ejabberd-24.07/deps/p1_acme/src/p1_acme.erl"},
                         {line,923}]},
                    {p1_acme,'-request_new_account/1-fun-0-',1,
                        [{file,
                             "/build/ejabberd/src/ejabberd-24.07/deps/p1_acme/src/p1_acme.erl"},
                         {line,286}]},
                    {p1_acme,http_request,3,
                        [{file,
                             "/build/ejabberd/src/ejabberd-24.07/deps/p1_acme/src/p1_acme.erl"},
                         {line,614}]},
                    {p1_acme,request_new_account,1,
                        [{file,
                             "/build/ejabberd/src/ejabberd-24.07/deps/p1_acme/src/p1_acme.erl"},
                         {line,290}]}]},
                  {gen_server,call,
                      [ejabberd_acme,
                       {request,
                           [<<"example.org">>,<<"proxy.xmpp.example.org">>,
                            <<"pubsub.xmpp.example.org">>,
                            <<"conference.example.org">>,
                            <<"upload.example.org">>]},
                       600000]}}
   in function  gen_server:call/3 (gen_server.erl, line 1126)
   in call from ejabberd_acme:request_certificate/1 (src/ejabberd_acme.erl, line 491)
   in call from ejabberd_ctl:call_command/4 (src/ejabberd_ctl.erl, line 328)
   in call from ejabberd_ctl:try_call_command/4 (src/ejabberd_ctl.erl, line 289)
   in call from ejabberd_ctl:process2/4 (src/ejabberd_ctl.erl, line 229)
   in call from ejabberd_ctl:process/2 (src/ejabberd_ctl.erl, line 207)
   in call from erpc:execute_call/4 (erpc.erl, line 1250)
prefiks commented 3 months ago

I added https://github.com/processone/p1_acme/commit/cd292d02ca12ff9cdcad17732c4b02c289288958 in p1_acme library that i think will help with that error.

knoelli commented 2 months ago

I get the same error as svbergerem reported here. Tried to fix it by recompiling ejabberd with the commit cd292d0 (and ef06ff8) to p1_acme, however, the problem still persists.

Output of ejabberdctl request_certificate all:

Unhandled exception occurred executing the command:
** exception exit: {{{invalid_ejson,{<<"crv">>,<<"P-256">>}},
                   [{jiffy,encode,2,
                        [{file,
                             "/build/ejabberd/src/ejabberd-24.07/deps/jiffy/src/jiffy.erl"},
                         {line,91}]},
                    {jose_json_jiffy,encode,1,
                        [{file,"src/json/jose_json_jiffy.erl"},{line,26}]},
                    {jose_jws,to_binary,1,
                        [{file,"src/jws/jose_jws.erl"},{line,134}]},
                    {jose_jws,sign,4,
                        [{file,"src/jws/jose_jws.erl"},{line,307}]},
                    {p1_acme,jose_json,3,
                        [{file,
                             "/build/ejabberd/src/ejabberd-24.07/deps/p1_acme/src/p1_acme.erl"},
                         {line,923}]},
                    {p1_acme,'-request_new_account/1-fun-0-',1,
                        [{file,
                             "/build/ejabberd/src/ejabberd-24.07/deps/p1_acme/src/p1_acme.erl"},
                         {line,286}]},
                    {p1_acme,http_request,3,
                        [{file,
                             "/build/ejabberd/src/ejabberd-24.07/deps/p1_acme/src/p1_acme.erl"},
                         {line,614}]},
                    {p1_acme,request_new_account,1,
                        [{file,
                             "/build/ejabberd/src/ejabberd-24.07/deps/p1_acme/src/p1_acme.erl"},
                         {line,290}]}]},
                  {gen_server,call,
                      [ejabberd_acme,
                       {request,
                           [<<"conversations.p***.de">>,
                            <<"pubsub.conversations.p***.de">>,
                            <<"conference.conversations.p***.de">>,
                            <<"proxy.conversations.p***.de">>,
                            <<"upload.conversations.p***.de">>]},
                       600000]}}
   in function  gen_server:call/3 (gen_server.erl, line 1126)
   in call from ejabberd_acme:request_certificate/1 (src/ejabberd_acme.erl, line 491)
   in call from ejabberd_ctl:call_command/4 (src/ejabberd_ctl.erl, line 328)
   in call from ejabberd_ctl:try_call_command/4 (src/ejabberd_ctl.erl, line 289)
   in call from ejabberd_ctl:process2/4 (src/ejabberd_ctl.erl, line 229)
   in call from ejabberd_ctl:process/2 (src/ejabberd_ctl.erl, line 207)
   in call from erpc:execute_call/4 (erpc.erl, line 1250)
badlop commented 2 months ago

You could apply this debugging patch. It allows you to check if ejabberd is really using the patched p1_acme binaries, and also prints the arguments that provoke the crash:

```diff diff --git a/src/p1_acme.erl b/src/p1_acme.erl index 8e22352..211d52f 100644 --- a/src/p1_acme.erl +++ b/src/p1_acme.erl @@ -910,6 +910,7 @@ jose_json(#state{account = {Key, AccURL}, nonce = Nonce} = State, Data, URL) -> end, JwsMap0 = #{<<"nonce">> => Nonce, <<"url">> => iolist_to_binary(URL)}, + ?INFO_MSG("JOSE privkey: ~p~npubkey: ~p", [PrivKey, PubKey]), JwsMap = case AccURL of undefined -> {_, BinaryPubKey} = jose_jwk:to_binary(PubKey), @@ -919,7 +920,7 @@ jose_json(#state{account = {Key, AccURL}, nonce = Nonce} = State, Data, URL) -> JwsMap0#{<<"kid">> => iolist_to_binary(AccURL)} end, JwsObj = jose_jws:from(maps:merge(JwsMap, AlgMap)), - ?DEBUG("JOSE payload: ~s~nJOSE protected: ~p", [Data, JwsObj]), + ?INFO_MSG("JOSE payload: ~s~nJOSE protected: ~p", [Data, JwsObj]), {_, JoseJSON} = jose_jws:sign(PrivKey, Data, JwsObj), encode_json(JoseJSON). ```
knoelli commented 2 months ago

Thank you for your input. You are right with your assumption, my test ejabberd was indeed compiled with a p1_acme version tagged 1.0.23 instead of the latest head, although I placed the current version in the deps directory. My mistake. Now with commits cd292d0 and ef06ff8 correctly applied during compilation, certificate renewal works as expected.

a4r7gc4ttfwsqcey commented 1 month ago

How did you resolve it? I don't understand how to compile it against p1_acme HEAD. Can someone explain what I need to do? Also on Arch Linux. I hope this is fixed in the next releases soon...

knoelli commented 1 month ago

@a4r7gc4ttfwsqcey You can checkout the current build scripts from arch using: git clone https://gitlab.archlinux.org/archlinux/packaging/packages/ejabberd.git In the resulting directory ejabberd replace the PKGBUILD with the one from this archive, also copy certificate_renew.patch included in this archive to the ejabberd directory.

Make sure to have the devtools package installed (pacman -S devtools), then compile ejabberd with the patch using a clean chroot environment: extra-x86_64-build (or the equivalent if building for another architecture).

a4r7gc4ttfwsqcey commented 1 month ago

@knoelli Thank you so much man, just did that and it worked perfectly!

Neustradamus commented 1 month ago

@svbergerem: Have you solved your problem with @prefiks, @badlop comments?

svbergerem commented 1 month ago

@Neustradamus No, I temporarily “fixed” the problem by renewing the certificates manually using certbot. I won't build the package myself and rather wait for the next release. However, since others already confirmed that the bug has been fixed, I guess this issue can be closed.