search

processone / ejabberd

Robust, Ubiquitous and Massively Scalable Messaging Platform (XMPP, MQTT, SIP Server)
https://www.process-one.net/en/ejabberd/
Other
6.1k stars 1.51k forks source link

mod_pubsub crashes after adding hosts configuration #3545

Open the-rocinante opened 3 years ago

the-rocinante commented 3 years ago

Environment

Note

All real IP addresses, hostnames, usernames and passwords have been modified to maintain privacy.

Bug description

After adding the hosts section to mod_pubsub as per the following...

  mod_pubsub:
    access_createnode: pubsub_createnode
    plugins:
      - flat
      - pep
    force_node_config:
      storage:bookmarks:
        access_model: whitelist
    hosts:
      - pubsub.HOSTNAME.TLD

... and restarting ejabberd, the following crash/error is generated. If the above hosts section is removed or commented out, ejabberd starts normally and operates without error.

Errors from error.log

2021-03-05 17:23:26.094 [error] <0.564.0>@mod_pubsub:-init/1-fun-0-:272 CRASH REPORT Process <0.564.0> with 0 neighbours crashed with reason: bad argument in call to ets:new('config_pubsub.HOSTNAME.TLD', [set,named_table]) in mod_pubsub:'-init/1-fun-0-'/12 line 272
2021-03-05 17:23:26.098 [critical] <0.358.0>@gen_mod:start_module:169 Failed to start module mod_pubsub:
{error,
    {{badarg,
         [{ets,new,['config_pubsub.HOSTNAME.TLD',[set,named_table]],[]},
          {mod_pubsub,'-init/1-fun-0-',12,
              [{file,"src/mod_pubsub.erl"},{line,272}]},
          {lists,foreach,2,[{file,"lists.erl"},{line,1338}]},
          {mod_pubsub,'-init/1-fun-1-',9,
              [{file,"src/mod_pubsub.erl"},{line,268}]},
          {lists,flatmap,2,[{file,"lists.erl"},{line,1250}]},
          {mod_pubsub,init,1,[{file,"src/mod_pubsub.erl"},{line,258}]},
          {gen_server,init_it,2,[{file,"gen_server.erl"},{line,374}]},
          {gen_server,init_it,6,[{file,"gen_server.erl"},{line,342}]}]},
     {child,undefined,'mod_pubsub_OTHERHOSTNAME.TLD',
         {gen_server,start_link,
             [{local,'mod_pubsub_OTHERHOSTNAME.TLD'},
              mod_pubsub,
              [<<"OTHERHOSTNAME.TLD">>,
               #{access_createnode => pubsub_createnode,db_type => sql,
                 default_node_config => [],
                 force_node_config =>
                     [{{re_pattern,1,0,0,
                           <<69,82,67,80,115,0,0,0,16,0,0,0,1,0,0,0,255,255,
                             255,255,255,255,255,255,0,0,0,0,0,0,1,0,0,0,64,0,
                             0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
                             0,0,0,0,131,0,47,27,133,0,39,0,1,29,115,29,116,
                             29,111,29,114,29,97,29,103,29,101,29,58,29,98,29,
                             111,29,111,29,107,29,109,29,97,29,114,29,107,29,
                             115,120,0,39,25,120,0,47,0>>},
                       [{access_model,whitelist}]}],
                 host => <<"pubsub.OTHERHOSTNAME.TLD">>,
                 hosts => [<<"pubsub.HOSTNAME.TLD">>],
                 ignore_pep_from_offline => true,last_item_cache => false,
                 max_items_node => 10,max_nodes_discoitems => 100,
                 max_subscriptions_node => undefined,
                 name => <<"Publish-Subscribe">>,nodetree => <<"tree">>,
                 pep_mapping => [],
                 plugins => [<<"flat">>,<<"pep">>],
                 vcard => undefined}],
              [{max_queue,10000}]]},
         transient,60000,worker,
         [mod_pubsub]}}}
2021-03-05 17:23:26.098 [critical] <0.358.0>@gen_mod:maybe_halt_ejabberd:256 ejabberd initialization was aborted because a module start failed.

Configuration

hosts:
  - HOSTNAME.TLD
  - OTHERHOSTNAME.TLD
captcha_cmd: /opt/ejabberd-21.01/lib/ejabberd-21.01/priv/bin/captcha.sh
captcha_url: https://im.HOSTNAME.TLD:5443/captcha
host_config:
  HOSTNAME.TLD:
    sql_database: "ejabberd_db_0"
    sql_username: "ejabberd_db_0_user"
    sql_password: "super_secret_password_0"
  OTHERHOSTNAME.TLD:
    sql_database: "ejabberd_db_1"
    sql_username: "ejabberd_db_1_user"
    sql_password: "super_secret_password_1"
loglevel: 4
log_rotate_size: 10485760
log_rotate_count: 50
certfiles:
  - "/path/to/cert.pem"
ca_file: "/opt/ejabberd/conf/cacert.pem"
listen:
  -
    port: 5222
    module: ejabberd_c2s
    max_stanza_size: 262144
    shaper: c2s_shaper
    access: c2s
    starttls_required: true
  -
    port: 5223
    module: ejabberd_c2s
    access: c2s
    shaper: c2s_shaper
    tls: true
    max_stanza_size: 65536
  -
    port: 5269
    module: ejabberd_s2s_in
    max_stanza_size: 524288
  -
    port: 5270
    tls: true
    module: ejabberd_s2s_in
    max_stanza_size: 524288
  -
    port: 5443
    module: ejabberd_http
    tls: true
    request_handlers:
      "/admin": ejabberd_web_admin
      "/api": mod_http_api
      "/bosh": mod_bosh
      "/captcha": ejabberd_captcha
      "/upload": mod_http_upload
      "/ws": ejabberd_http_ws
      "/oauth": ejabberd_oauth
  -
    port: 5280
    module: ejabberd_http
    request_handlers:
      "/admin": ejabberd_web_admin
      "/api": mod_http_api
      "/bosh": mod_bosh
      "/captcha": ejabberd_captcha
      "/upload": mod_http_upload
      "/ws": ejabberd_http_ws
      "/oauth": ejabberd_oauth
  -
    port: 1883
    module: mod_mqtt
    backlog: 1000
  -
    port: 3478
    transport: udp
    module: ejabberd_stun
    auth_type: anonymous
    use_turn: true
    turn_min_port: 49152
    turn_max_port: 65535
    turn_ipv4_address: "::"
  -
    port: 5349
    transport: tcp
    module: ejabberd_stun
    auth_type: anonymous
    use_turn: true
    tls: true
    turn_min_port: 49152
    turn_max_port: 65535
    turn_ipv4_address: "::"
s2s_use_starttls: required
acl:
  local:
    user_regexp: ""
  loopback:
    ip:
      - 127.0.0.0/8
      - ::1/128
      - ::FFFF:127.0.0.1/128
  admin:
    user:
      - "admin@HOSTNAME.TLD"
      - "admin@OTHERHOSTNAME.TLD"
access_rules:
  local:
    allow: local
  c2s:
    deny: blocked
    allow: all
  announce:
    allow: admin
  configure:
    allow: admin
  muc_create:
    allow: local
  pubsub_createnode:
    allow: local
  trusted_network:
    allow: loopback
api_permissions:
  "console commands":
    from:
      - ejabberd_ctl
    who: all
    what: "*"
  "admin access":
    who:
      access:
        allow:
          acl: loopback
          acl: admin
      oauth:
        scope: "ejabberd:admin"
        access:
          allow:
            acl: loopback
            acl: admin
    what:
      - "*"
      - "!stop"
      - "!start"
  "public commands":
    who:
      ip: 127.0.0.1/8
    what:
      - status
      - connected_users_number
shaper:
  normal:
    rate: 3000
    burst_size: 20000
  fast: 100000
shaper_rules:
  max_user_sessions: 10
  max_user_offline_messages:
    5000: admin
    500: all
  c2s_shaper:
    none: admin
    normal: all
  s2s_shaper: fast
  soft_upload_quota:
    512: all
  hard_upload_quota:
    1024: all
max_fsm_queue: 10000
acme:
  contact: "mailto:tls-admin@HOSTNAME.TLD"
  ca_url: "https://acme-v02.api.letsencrypt.org/directory"
  auto: false
modules:
  mod_adhoc: {}
  mod_admin_extra: {}
  mod_announce:
    access: announce
  mod_avatar: {}
  mod_blocking: {}
  mod_bosh: {}
  mod_caps: {}
  mod_carboncopy: {}
  mod_client_state: {}
  mod_configure: {}
  mod_disco:
    server_info:
      -
        modules: all
        name: "abuse-addresses"
        urls: ["mailto:xmpp-admin@HOSTNAME.TLD"]
  mod_fail2ban: {}
  mod_http_api: {}
  mod_http_upload:
    docroot: /home/happy/ejabberd/upload
    put_url: https://@HOST@:5443/upload
    custom_headers:
      "Access-Control-Allow-Origin": "https://@HOST@"
      "Access-Control-Allow-Methods": "GET,HEAD,PUT,OPTIONS"
      "Access-Control-Allow-Headers": "Content-Type"
  mod_http_upload_quota:
    max_days: 30
  mod_last: {}
  mod_mam:
    assume_mam_usage: true
    default: never
  mod_mqtt: {}
  mod_muc:
    access:
      - allow
    access_admin:
      - allow: admin
    access_create: muc_create
    access_persistent: muc_create
    access_mam:
      - allow
    default_room_options:
      allow_subscription: true
      mam: false
    hosts:
      - conference.HOSTNAME.TLD
  mod_muc_admin: {}
  mod_offline:
    access_max_user_messages: max_user_offline_messages
  mod_ping: {}
  mod_privacy: {}
  mod_private: {}
  mod_muc_log:
    outdir: /home/happy/www/ejabberd/muc_log
    timezone: universal
  mod_proxy65:
    access: local
    max_connections: 5
    hosts:
      - proxy.HOSTNAME.TLD
  mod_pubsub:
    access_createnode: pubsub_createnode
    plugins:
      - flat
      - pep
    force_node_config:
      storage:bookmarks:
        access_model: whitelist
  mod_push: {}
  mod_push_keepalive: {}
  mod_register:
    access_from: c2s
    captcha_protected: true
    password_strength: 32
    registration_watchers: [admin@HOSTNAME.TLD]
  mod_roster:
    versioning: true
  mod_s2s_dialback: {}
  mod_shared_roster: {}
  mod_stream_mgmt:
    resend_on_timeout: if_offline
  mod_vcard: {}
  mod_vcard_xupdate: {}
  mod_version:
    show_os: false
  mod_stun_disco:
    credentials_lifetime: 12h
    services:
      -
        host: "::"
        port: 3478
        type: stun
        transport: udp
        restricted: false
      -
        host: "::"
        port: 3478
        type: turn
        transport: udp
        restricted: true
      -
        host: ice.HOSTNAME.TLD
        port: 5349
        type: stuns
        transport: tcp
        restricted: false
      -
        host: ice.HOSTNAME.TLD
        port: 5349
        type: turns
        transport: tcp
        restricted: true
auth_method: [sql]
default_db: sql
sql_type: mysql
sql_server: "127.0.0.1"
sql_database: "ejabberd_db"
sql_username: "ejabberd_db"
sql_password: "super_secret_password"
sql_pool_size: 5
sql_port: 3306
new_sql_schema: false
default_ram_db: redis
redis_password: "extremely_super_secret_password"
redis_server: "127.0.0.1"
redis_port: 6379
redis_db: 0
prefiks commented 3 years ago

Hello,

It probably will work if you use hosts: pubsub.@HOST@, or if you skip that all together as this is default value. Problem here is that you have two domain, and they both start mod_pubsub module, but they share same name for that service - value you passed in hosts:, and this fails when second domain tries to setup service with same name. I will try to extend config validator to reject config like this with more meaningful error.

badlop commented 3 years ago

If you want to have two XMPP hosts, but only one PubSub service, displayed in the Service Discovery of both hosts:

hosts:
  - hostname.tld
  - otherhostname.tld

modules:
  # mod_disco: ...
  # mod_pubsub: ...
  ...

append_host_config:
  hostname.tld:
    modules:
      mod_pubsub:
        hosts:
          - pubsub.hostname.tld
      mod_disco: {}
  otherhostname.tld:
    modules:
      mod_disco:
        extra_domains:
          - pubsub.hostname.tld
the-rocinante commented 3 years ago

The reason I went through this exercise is that I notice in syslog there are numerous entries like the following

[info] Outbound s2s connection started: hostname.tld -> pubsub.stun.hostname.tld
[info] Closing inbound s2s connection hostname.tld -> pubsub.stun.hostname.tld: Stream closed by local host: host-unknown
[warning] Failed to establish outbound s2s connection hostname.tld -> pubsub.stun.hostname.tld: Stream closed by peer: host-unknown; bouncing for 261 seconds

The only place the stun.hostname.tld is mentioned is in the DNS SRV records for hostname.tld related to the stun port (nothing to do with pubsub). So somehow this SRV record is triggering a spurious request to a nonexistent subdomain. My thought was that if I simply configure the service to explicitly use pubsub.hostname.tld, then the above warning would go away, but instead it crashed ejabberd.

I don't necessarily want only one domain to reference a pubsub. subdomain, since I have multiple different domains which I would like to serve as distinct from each other. Although I could simply have the _stun._udp. and _stuns._udp. SRV records point to hostname.tld rather than stun.hostname.tld, I would prefer to use the subdomain as I may eventually setup a stand-alone stun service on a separate host. What I'm really trying to achieve is the elimination of the requests for non-existant subdomains showing up as warnings in my syslog. I even went so far as to setup CNAME records for pubsub.stun.hostname.tld et al, but that didn't seem to have any affect. Anyway, if you can provide some insight about what I'm doing wrong, or how to identify and eliminate whatever this issue is, I'd appreciate it.

Neustradamus commented 3 years ago

@badlop, @prefiks: What do you think about last @the-rocinante comment?

badlop commented 3 years ago

Reading this thread again, it seems a specific setup that has a specific problem. I don't remember any recent fix or issue related to this one, so it may be an obscure bug somewhere, an incompatibility, a misconfiguration... As I don't see an easy way to reproduce the problem in my workbench, and I know very little about pubsub and stun, I won't investigate it. My opinion is that the best place to investigate this behaviour is in the place where it is happening, until some more light is offered, some clue where more people can engage in investigating it. As I have nothing of interest to offer anymore in this issue, I would prefer to shut up, instead of adding spam comments that add nothing.