mod_spam_filter: ejabberd fails to start

[vanitasvitae]

Hi! I tried enabling mod_spam_filter on my server (ejabberd 20.03) but as soon as I enable the module in my ejabberd.yml, I run into the error below. Is the module compatible with the 20.03 release?

2020-04-29 01:01:14.779 [error] <0.432.0> CRASH REPORT Process <0.432.0> with 0 neighbours exited with reason: config_error in gen_server:init_it/6 line 358
2020-04-29 01:01:14.779 [critical] <0.362.0>@gen_mod:start_module:169 Failed to start module mod_spam_filter: 
{error,
    {config_error,
        {child,undefined,'mod_spam_filter_myserver.tld',
            {gen_server,start_link,
                [{local,'mod_spam_filter_myserver.tld'},
                 mod_spam_filter,
                 [<<"myserver.tld">>,
                  #{access_spam => none,cache_size => 500,
                    spam_dump_file => <<"/var/log/ejabberd/spam.log">>,
                    spam_jids_file => none,
                    spam_urls_file =>
                        <<"/etc/ejabberd/spam-filter/urls.txt">>}],
                 [{max_queue,10000}]]},
            transient,60000,worker,
            [mod_spam_filter]}}}
2020-04-29 01:01:14.780 [critical] <0.362.0>@gen_mod:maybe_halt_ejabberd:256 ejabberd initialization was aborted because a module start failed.

[weiss]

Is the module compatible with the 20.03 release?

It's supposed to be (works for me with ejabberd's current master code).

[badlop]

The module starts correctly also with ejabberd 20.03 compiled from sources. Both when the module is compiled by copying the file to ejabberd/src, and also when the module is installed with "ejabberdctl module_install".

So, it may be a matter of how you installed ejabberd, how you compiled and installed the module...

[vanitasvitae]

I installed ejabberd via the deb package from process-one. The module was installed through ejabberdctl module_install mod_spam_filter.

I'll do some further testing on 20.05 once its released.

[badlop]

as soon as I enable the module in my ejabberd.yml, I run into the error below.

The module was installed through ejabberdctl module_install mod_spam_filter.

Please note that installing a module with "module_install" will also enable immediately its default configuration. You can find the module configuration file in a place like $HOME/.ejabberd-modules/mod_spam_filter/conf/mod_spam_filter.yml So, no need to add the module also in your general ejabberd.yml

[vanitasvitae]

Yeah, but I have a configuration where I have 3 vhosts and each one has its own modules: block, so I believe that the default configuration of the module has no effect as the modules: block is overwritten for each host.

I come to this conclusion, as the module is not loaded when its only activated in /opt/ejabberd/.ejabberd-modules/mod_spam_filter/conf/mod_spam_filter.yml but not in /opt/ejabberd/conf/ejabberd.yml. If I change the module configuration file to something like

host_append_config:
  myserver.tld:
    modules:
      mod_spam_filter:
        ...

I run into the same error. Does it maybe not work with per-host configurations?

[vanitasvitae]

One more thing; when I install mod_spam_filter, I get the following error:

/opt/ejabberd/.ejabberd-modules/sources/ejabberd-contrib/mod_spam_filter/src/mod_spam_filter.erl:30: Warning: undefined callback function mod_doc/0 (behaviour 'gen_mod')

Is this any useful? I definitely get this error in ejabberd 20.04 and I got an error (not sure if this one specific) in ejabberd 20.03.

Edit: The original issue is still present for me in 20.04 btw.

[badlop]

I get the following error: Warning: undefined callback

It's a warning, not an error. Fortunately it's irrelevant, and harmless.

Does it maybe not work with per-host configurations?

I've configured like this, and it starts correctly (I didn't check the feature itself)

hosts:
  - localhost
  - atenea

modules:
  ...

append_host_config:
  localhost:
    modules:
      mod_spam_filter:
        spam_jids_file: "/tmp/eja/localhost.txt"

append_host_config:
  atenea:
    modules:
      mod_spam_filter:
        spam_jids_file: "/tmp/eja/atenea.txt"

Yeah, but I have a configuration where I have 3 vhosts and each.........

Oh, so you had a complex configuration, and you didn't inform about that when reporting a problem with an error message that said config_error. In fact, you haven't shown your configuration yet, which is probably the source of the problem.

If I change the module configuration file to something like host_append_config:

And do you really have this in your configuration file?

2020-05-04 19:51:55.904278+02:00 [critical]
 Failed to start ejabberd application:
 Unknown option: host_append_config.
 Did you mean host_config? 

OK, I've lost patience and interest in this.

[vanitasvitae]

Oh, so you had a complex configuration, and you didn't inform about that when reporting a problem with an error message that said config_error. In fact, you haven't shown your configuration yet, which is probably the source of the problem.

Sorry for not providing my configuration.

###
###              ejabberd configuration file
###
### The parameters used in this configuration file are explained at
###
###       https://docs.ejabberd.im/admin/configuration
###
### The configuration file is written in YAML.
### *******************************************************
### *******           !!! WARNING !!!               *******
### *******     YAML IS INDENTATION SENSITIVE       *******
### ******* MAKE SURE YOU INDENT SECTIONS CORRECTLY *******
### *******************************************************
### Refer to http://en.wikipedia.org/wiki/YAML for the brief description.
### However, ejabberd treats different literals as different types:
###
### - unquoted or single-quoted strings. They are called "atoms".
###   Example: dog, 'Jupiter', '3.14159', YELLOW
###
### - numeric literals. Example: 3, -45.0, .0
###
### - quoted or folded strings.
###   Examples of quoted string: "Lizzard", "orange".
###   Example of folded string:
###   > Art thou not Romeo,
###     and a Montague?
###
#
---
## loglevel: Verbosity of log files generated by ejabberd
## 0: No ejabberd log at all (not recommended)
## 1: Critical
## 2: Error
## 3: Warning
## 4: Info
## 5: Debug
loglevel: 4

## rotation: Disable ejabberd's internal log rotation, as the Debian package
## uses logrotate(8).
log_rotate_count: 0
#log_rotate_date: ""

hosts:
    ## serves as main host, as well as auth.meet.myserver.tld for jitsi focus user
  - "myserver.tld"
    ## serves as anonymous authentication host for jitsi meet.myserver.tld
  - "meet.myserver.tld"
    ## anonymous auth for converse.js chat.myserver.tld
  - "chat.myserver.tld"

certfiles:
  - "/etc/ssl/myserver.tld/cert.pem"
  - "/etc/ssl/myserver.tld/fullchain.pem"
  - "/etc/ssl/myserver.tld/privkey.pem"
## TLS configuration
define_macro:
  'TLS_CIPHERS': "HIGH:!aNULL:!eNULL:!3DES:@STRENGTH"
  'TLS_OPTIONS':
    - "no_sslv2"
    - "no_sslv3"
    - "no_tlsv1"
    - "no_tlsv1_1"
    - "cipher_server_preference"
    - "no_compression"
  'DH_FILE': "/etc/ssl/myserver.tld/dhparams_4096.pem" # generated with: openssl dhparam -out dhparams.pem 4096

captcha_cmd: /opt/ejabberd/tools/captcha.sh
captcha_url: https://myserver.tld/captcha

c2s_ciphers: 'TLS_CIPHERS'
s2s_ciphers: 'TLS_CIPHERS'
c2s_protocol_options: 'TLS_OPTIONS'
s2s_protocol_options: 'TLS_OPTIONS'
c2s_dhfile: 'DH_FILE'
s2s_dhfile: 'DH_FILE'

listen:
  -
    port: 3478
    transport: udp
    module: ejabberd_stun
    use_turn: true
    turn_ip: 123.123.123.132

  -
    port: 5222
    ip: "::"
    module: ejabberd_c2s
    max_stanza_size: 262144
    shaper: c2s_shaper
    access: c2s
    starttls_required: true
    protocol_options: 'TLS_OPTIONS'
  -
##    port: 5223
##    ip: "::"
##    module: ejabberd_c2s
##    max_stanza_size: 262144
##    shaper: c2s_shaper
##    access: c2s
##    tls: true
##    protocol_options: 'TLS_OPTIONS'
##  -
    port: 5269
    ip: "::"
    module: ejabberd_s2s_in
    max_stanza_size: 524288

  -
    port: 5275
    ip: "::"
    module: ejabberd_service
    access: all
    shaper: fast
    hosts: 
      "jitsi-videobridge.myserver.tld": 
        password: "SECRET"

  -
    port: 5280
    ip: "::"
    module: ejabberd_http
    request_handlers:
      "/http-bind": mod_bosh
      "/admin": ejabberd_web_admin
      "/captcha": ejabberd_captcha
      "/upload": mod_http_upload
      "/ws": ejabberd_http_ws
      "/oauth": ejabberd_oauth

  -
    port: 5347
    module: ejabberd_service
    hosts: 
      "focus.myserver.tld": 
        password: "SECRET"

##  -
##    port: 5443
##    ip: "::"
##    module: ejabberd_http
##    request_handlers:
##      "/admin": ejabberd_web_admin
##      "/captcha": ejabberd_captcha
##      "/upload": mod_http_upload
##      "/ws": ejabberd_http_ws
##      "/oauth": ejabberd_oauth
##    tls: true
##    protocol_options: 'TLS_OPTIONS'
##    dhfile: 'DH_FILE'
##    ciphers: 'TLS_CIPHERS'

## Disabling digest-md5 SASL authentication. digest-md5 requires plain-text
## password storage (see auth_password_format option).
disable_sasl_mechanisms:
  - "digest-md5"
  - "X-OAUTH2"

s2s_use_starttls: required

## Store the plain passwords or hashed for SCRAM:
auth_password_format: scram
auth_method: sql

##
## MySQL server:
##
sql_type: mysql
sql_server: "localhost"
sql_database: "ejabberd_db"
sql_username: "ejabberd_db_user"
sql_password: "longandsecurepassword"
default_db: sql

acl:
  admin:
     user:
       - "adminuser@myserver.tld"
       - "focus@myserver.tld"
  bad_servers:
    server:
      ...

  local:
    user_regexp: ""
  loopback:
    ip:
      - 127.0.0.0/8
      - ::1/128
      - ::FFFF:127.0.0.1/128

access_rules:
  local:
    allow: local
  c2s:
    deny: blocked
    allow: all
  s2s:
    deny: bad_servers
    allow: all
  announce:
    allow: admin
  configure:
    allow: admin
  muc_create:
    allow: local
  pubsub_createnode:
    allow: local
  register:
    - deny
  trusted_network:
    allow: loopback

api_permissions:
  "console commands":
    from:
      - ejabberd_ctl
    who: all
    what: "*"
  "admin access":
    who:
      access:
        allow:
          acl: loopback
          acl: admin
      oauth:
        scope: "ejabberd:admin"
        access:
          allow:
            acl: loopback
            acl: admin
    what:
      - "*"
      - "!stop"
      - "!start"
  "public commands":
    who:
      ip: 127.0.0.1/8
    what:
      - status
      - connected_users_number

shaper:
  normal: 1000
  fast: 50000

shaper_rules:
  max_user_sessions: 10
  max_user_offline_messages:
    5000: admin
    100: all
  c2s_shaper:
    none: admin
    normal: all
  s2s_shaper: fast

max_fsm_queue: 10000

## Either include mod_spam_filter like this, or as below in myserver.tld's modules block.
append_host_config:
  myserver.tld:
    modules:
      mod_spam_filter:
        spam_dump_file: "/var/log/ejabberd/spam.log"
        spam_urls_file: "/etc/ejabberd/spam-filter/urls.txt"
        cache_size: 500

host_config:
  myserver.tld:
    s2s_access: s2s
    modules:
      mod_adhoc: {}
      mod_admin_extra: {}
      mod_announce:
        access: announce
      mod_avatar: {}
      mod_blocking: {}
      mod_bosh: {}
      mod_caps: {}
      mod_carboncopy: {}
      mod_client_state: {}
      mod_configure: {}
      mod_disco:
        server_info:
          -
            modules: all
            name: "abuse-addresses"
            urls:
              - "mailto:admin@myserver.tld"
          -
            modules: all
            name: "support-addresses"
            urls:
              - "mailto:admin@myserver.tld"
          -
            modules: all
            name: "admin-addresses"
            urls:
              - "mailto:admin@myserver.tld"
    ##  mod_echo: {}
      mod_fail2ban: {}
    ##  mod_http_api: {}
      mod_http_upload:
        access: local
        docroot: "/var/data/ejabberd/upload"
        put_url: https://@HOST@/upload
        custom_headers:
          "Access-Control-Allow-Origin": "https://myserver.tld"
          "Access-Control-Allow-Methods": "GET,HEAD,PUT,OPTIONS"
          "Access-Control-Allow-Headers": "Content-Type"
      mod_http_upload_quota:
        max_days: 30
      mod_last: {}
      mod_mam:
        db_type: sql
        assume_mam_usage: true 
        default: always
    ##  mod_mqtt: {}
      mod_muc:
        access:
          - allow
        access_admin:
          - allow: admin
        access_create: muc_create
        access_persistent: muc_create
        access_mam:
          - allow
        default_room_options:
          allow_private_messages: true
          mam: true
          persistent: true
      mod_muc_admin: {}
      mod_offline:
        access_max_user_messages: max_user_offline_messages
      mod_ping: {}
      mod_pres_counter:
        count: 5
        interval: 60
      mod_privacy: {}
      mod_private: {}
      mod_proxy65:
        host: "proxy.myserver.tld"
        ip: "0.0.0.0"
        hostname: "proxy.myserver.tld"
        max_connections: 5
      mod_pubsub:
        access_createnode: pubsub_createnode
        plugins:
          - flat
          - pep
        force_node_config:
          "eu.siacs.conversations.axolotl.*":
            access_model: open
          ## Avoid buggy clients to make their bookmarks public
          storage:bookmarks:
            access_model: whitelist
##      mod_push: {}
##      mod_push_keepalive: {}
      ## mod_register:
      ##   ## Only accept registration requests from the "trusted"
      ##   ## network (see access_rules section above).
      ##   ## Think twice before enabling registration from any
      ##   ## address. See the Jabber SPAM Manifesto for details:
      ##   ## https://github.com/ge0rg/jabber-spam-fighting-manifesto
      ##   ip_access: trusted_network
      mod_roster:
        versioning: true
      mod_s2s_dialback: {}
##      mod_shared_roster: {}
      mod_stream_mgmt:
        resend_on_timeout: if_offline
      mod_stun_disco: {}
      mod_vcard: {}
      mod_vcard_xupdate: {}
      mod_version:
        show_os: false
## Right now mod_spam_filter is enabled via append_host_config further above
##      mod_spam_filter:
##        spam_dump_file: "/var/log/ejabberd/spam.log"
##        spam_urls_file: "/etc/ejabberd/spam-filter/urls.txt"
##        cache_size: 500

  meet.myserver.tld:
    s2s_access: none
    auth_method: anonymous
    allow_multiple_connections: true
    anonymous_protocol: both
    modules:
      mod_bosh: {}
      mod_disco: {}
      mod_muc:
        access: all
        access_create: local
        access_persistent: local
        access_admin: admin
      mod_muc_admin: {}
      mod_ping: {}
      mod_pubsub:
        access_createnode: local

  chat.myserver.tld:
    s2s_access: none
    auth_method: anonymous
    allow_multiple_connections: true
    anonymous_protocol: both
    modules:
      mod_bosh: {}
      mod_carboncopy: {}
      mod_disco: {}
      mod_muc:
        access: all
        access_create: local
        access_persistent: local
        access_admin: admin
      mod_muc_admin: {}
      mod_ping: {}
      mod_http_upload:
        access: local
        docroot: "/var/data/ejabberd/upload_anon"
        put_url: https://@HOST@/upload
        custom_headers:
          "Access-Control-Allow-Origin": "https://myserver.tld"
          "Access-Control-Allow-Methods": "GET,HEAD,PUT,OPTIONS"
          "Access-Control-Allow-Headers": "Content-Type"
      mod_http_upload_quota:
        max_days: 7

### Local Variables:
### mode: yaml
### End:
### vim: set filetype=yaml tabstop=8

[vanitasvitae]

So, should I close this issue as unresolved then?

[rickygm]

I have been able to solve this, I can't find this module working?