search

processone / docker-ejabberd

Set of ejabberd Docker images
94 stars 77 forks source link

Mnesia database not being saved/loaded properly #84

Closed hankjordan closed 2 years ago

hankjordan commented 2 years ago

Upon removal and re-creation of a docker container, the Mnesia database is not properly saved/loaded, causing all users, contacts, messages, etc to be lost.

The documentation leads me to believe this database is persistent:

/home/ejabberd/database: Directory containing Mnesia database. You should back up or export the content of the directory to persistent storage (host storage, local storage, any storage plugin)

To my knowledge, this was working fine previously.

Here is my configuration file. Certificates are valid, configuration loads properly:

```yaml ### ### ejabberd configuration file ### ### The parameters used in this configuration file are explained at ### ### https://docs.ejabberd.im/admin/configuration ### ### The configuration file is written in YAML. ### ******************************************************* ### ******* !!! WARNING !!! ******* ### ******* YAML IS INDENTATION SENSITIVE ******* ### ******* MAKE SURE YOU INDENT SECTIONS CORRECTLY ******* ### ******************************************************* ### Refer to http://en.wikipedia.org/wiki/YAML for the brief description. ### hosts: - EXTERNAL_URL loglevel: info ## If you already have certificates, list them here certfiles: - /home/ejabberd/certs/EXTERNAL_URL/fullchain.pem - /home/ejabberd/certs/EXTERNAL_URL/key.pem acme: auto: false listen: - port: 5222 ip: "::" module: ejabberd_c2s max_stanza_size: 262144 shaper: c2s_shaper access: c2s starttls_required: true - port: 5223 ip: "::" tls: true module: ejabberd_c2s max_stanza_size: 262144 shaper: c2s_shaper access: c2s starttls_required: true - port: 5269 ip: "::" module: ejabberd_s2s_in max_stanza_size: 524288 - port: 5443 ip: "::" module: ejabberd_http tls: true request_handlers: /admin: ejabberd_web_admin /api: mod_http_api /bosh: mod_bosh /captcha: ejabberd_captcha /upload: mod_http_upload /ws: ejabberd_http_ws - port: 5280 ip: "::" module: ejabberd_http request_handlers: /admin: ejabberd_web_admin /.well-known/acme-challenge: ejabberd_acme - port: 3478 ip: "::" transport: udp module: ejabberd_stun use_turn: true ## The server's public IPv4 address: # turn_ipv4_address: "203.0.113.3" ## The server's public IPv6 address: # turn_ipv6_address: "2001:db8::3" - port: 1883 ip: "::" module: mod_mqtt backlog: 1000 s2s_use_starttls: optional acl: local: user_regexp: "" loopback: ip: - 127.0.0.0/8 - ::1/128 access_rules: local: allow: local c2s: deny: blocked allow: all announce: allow: admin configure: allow: admin muc_create: allow: local pubsub_createnode: allow: local trusted_network: allow: loopback api_permissions: "console commands": from: - ejabberd_ctl who: all what: "*" "admin access": who: access: allow: - acl: loopback - acl: admin oauth: scope: "ejabberd:admin" access: allow: - acl: loopback - acl: admin what: - "*" - "!stop" - "!start" "public commands": who: ip: 127.0.0.1/8 what: - status - connected_users_number shaper: normal: rate: 3000 burst_size: 20000 fast: 100000 shaper_rules: max_user_sessions: 10 max_user_offline_messages: 5000: admin 100: all c2s_shaper: none: admin normal: all s2s_shaper: fast modules: mod_adhoc: {} mod_admin_extra: {} mod_announce: access: announce mod_avatar: {} mod_blocking: {} mod_bosh: {} mod_caps: {} mod_carboncopy: {} mod_client_state: {} mod_configure: {} mod_disco: {} mod_fail2ban: {} mod_http_api: {} mod_http_upload: put_url: https://@HOST@:5443/upload custom_headers: "Access-Control-Allow-Origin": "https://@HOST@" "Access-Control-Allow-Methods": "GET,HEAD,PUT,OPTIONS" "Access-Control-Allow-Headers": "Content-Type" mod_last: {} mod_mam: ## Mnesia is limited to 2GB, better to use an SQL backend ## For small servers SQLite is a good fit and is very easy ## to configure. Uncomment this when you have SQL configured: ## db_type: sql assume_mam_usage: true default: always mod_mqtt: {} mod_muc: access: - allow access_admin: - allow: admin access_create: muc_create access_persistent: muc_create access_mam: - allow default_room_options: mam: true mod_muc_admin: {} mod_offline: access_max_user_messages: max_user_offline_messages mod_ping: {} mod_privacy: {} mod_private: {} mod_proxy65: access: local max_connections: 5 mod_pubsub: access_createnode: pubsub_createnode plugins: - flat - pep force_node_config: ## Avoid buggy clients to make their bookmarks public storage:bookmarks: access_model: whitelist mod_push: {} mod_push_keepalive: {} mod_register: ## Only accept registration requests from the "trusted" ## network (see access_rules section above). ## Think twice before enabling registration from any ## address. See the Jabber SPAM Manifesto for details: ## https://github.com/ge0rg/jabber-spam-fighting-manifesto ip_access: trusted_network mod_roster: versioning: true mod_s2s_dialback: {} mod_shared_roster: {} mod_stream_mgmt: resend_on_timeout: if_offline mod_stun_disco: {} mod_vcard: {} mod_vcard_xupdate: {} mod_version: show_os: false ### Local Variables: ### mode: yaml ### End: ### vim: set filetype=yaml tabstop=8 ```

This is the docker-compose.yml:

```yaml xmpp: image: ejabberd/ecs restart: always ports: - 5222:5222 - 5269:5269 volumes: - ./xmpp/config:/home/ejabberd/conf - ./xmpp/db:/home/ejabberd/database - ./xmpp/log:/home/ejabberd/logs - ./xmpp/uploads:/home/ejabberd/upload - ./proxy/certs:/home/ejabberd/certs ```

I am running the latest version.

hankjordan commented 2 years ago

Container Startup Log:

``` 2022-04-30 19:25:00.091129+00:00 [info] Loading configuration from /home/ejabberd/conf/ejabberd.yml 2022-04-30 19:25:05.262039+00:00 [info] Configuration loaded successfully args: format: label: 2022-04-30 19:25:05.436937 2022-04-30 19:25:08.140666+00:00 [info] Got no NOTIFY_SOCKET, notifications disabled 2022-04-30 19:25:08.269459+00:00 [info] Building language translation cache 2022-04-30 19:25:12.742191+00:00 [info] Creating Mnesia ram table 'ejabberd_commands' 2022-04-30 19:25:14.533077+00:00 [info] Creating Mnesia ram table 'route' 2022-04-30 19:25:14.779195+00:00 [info] Creating Mnesia ram table 'route_multicast' 2022-04-30 19:25:15.078894+00:00 [info] Creating Mnesia ram table 'session' 2022-04-30 19:25:15.200965+00:00 [info] Creating Mnesia ram table 'session_counter' args: format: label: 2022-04-30 19:25:15.701603 2022-04-30 19:25:16.503419+00:00 [info] Creating Mnesia ram table 's2s' 2022-04-30 19:25:16.605418+00:00 [info] Creating Mnesia ram table 'temporarily_blocked' 2022-04-30 19:25:16.863511+00:00 [info] Loading modules for EXTERNAL_URL 2022-04-30 19:25:16.864015+00:00 [info] Creating Mnesia ram table 'mod_register_ip' 2022-04-30 19:25:17.165172+00:00 [info] Creating Mnesia disc table 'sr_group' 2022-04-30 19:25:17.599302+00:00 [info] Creating Mnesia disc table 'sr_user' 2022-04-30 19:25:17.950382+00:00 [info] Creating Mnesia disc_only table 'privacy' 2022-04-30 19:25:18.195154+00:00 [warning] Mnesia backend for mod_mam is not recommended: it's limited to 2GB and often gets corrupted when reaching this limit. SQL backend is recommended. Namely, for small servers SQLite is a preferred choice because it's very easy to configure. 2022-04-30 19:25:18.195597+00:00 [info] Creating Mnesia disc_only table 'archive_msg' 2022-04-30 19:25:18.205870+00:00 [info] Creating Mnesia disc_only table 'archive_prefs' 2022-04-30 19:25:18.441993+00:00 [info] Creating Mnesia disc table 'muc_room' 2022-04-30 19:25:18.485824+00:00 [info] Creating Mnesia disc table 'muc_registered' 2022-04-30 19:25:18.569112+00:00 [info] Creating Mnesia ram table 'muc_online_room' 2022-04-30 19:25:18.632131+00:00 [info] Creating Mnesia disc_only table 'vcard' 2022-04-30 19:25:18.654548+00:00 [info] Creating Mnesia disc table 'vcard_search' 2022-04-30 19:25:18.738451+00:00 [info] Creating Mnesia disc_only table 'motd' 2022-04-30 19:25:18.795253+00:00 [info] Creating Mnesia disc_only table 'motd_users' 2022-04-30 19:25:18.918595+00:00 [info] Creating Mnesia ram table 'bosh' 2022-04-30 19:25:18.963736+00:00 [info] Creating Mnesia disc_only table 'push_session' 2022-04-30 19:25:19.183940+00:00 [info] Going to offer STUN/TURN service: 172.18.0.12:3478 (udp) 2022-04-30 19:25:19.184540+00:00 [info] Creating Mnesia disc_only table 'roster' 2022-04-30 19:25:19.538445+00:00 [info] Creating Mnesia disc_only table 'roster_version' 2022-04-30 19:25:21.299158+00:00 [info] Creating Mnesia disc_only table 'last_activity' 2022-04-30 19:25:21.743854+00:00 [info] Creating Mnesia disc_only table 'offline_msg' 2022-04-30 19:25:22.640882+00:00 [info] Creating Mnesia disc_only table 'caps_features' 2022-04-30 19:25:22.814487+00:00 [info] Creating Mnesia ram table 'pubsub_last_item' 2022-04-30 19:25:22.991173+00:00 [info] Creating Mnesia disc table 'pubsub_index' args: format: label: 2022-04-30 19:25:23.086331 2022-04-30 19:25:23.162844+00:00 [info] Creating Mnesia disc table 'pubsub_node' 2022-04-30 19:25:23.239204+00:00 [info] Creating Mnesia disc table 'pubsub_state' 2022-04-30 19:25:23.312136+00:00 [info] Creating Mnesia disc_only table 'pubsub_item' 2022-04-30 19:25:23.493959+00:00 [info] Creating Mnesia disc table 'pubsub_orphan' 2022-04-30 19:25:23.546301+00:00 [info] Creating Mnesia disc_only table 'private_storage' 2022-04-30 19:25:24.020729+00:00 [info] Creating Mnesia disc_only table 'mqtt_pub' 2022-04-30 19:25:24.219350+00:00 [info] Creating Mnesia ram table 'mqtt_session' 2022-04-30 19:25:24.537471+00:00 [info] Creating Mnesia ram table 'mqtt_sub' 2022-04-30 19:25:25.628095+00:00 [info] Building MQTT cache for EXTERNAL_URL, this may take a while 2022-04-30 19:25:26.796804+00:00 [info] Creating Mnesia ram table 'bytestream' 2022-04-30 19:25:27.246609+00:00 [info] Creating Mnesia disc_only table 'passwd' 2022-04-30 19:25:27.392274+00:00 [info] Creating Mnesia ram table 'reg_users_counter' 2022-04-30 19:25:27.684289+00:00 [info] Creating Mnesia disc_only table 'oauth_token' 2022-04-30 19:25:27.785936+00:00 [info] Creating Mnesia disc table 'oauth_client' 2022-04-30 19:25:28.492065+00:00 [info] Waiting for Mnesia synchronization to complete args: format: label: 2022-04-30 19:25:28.596073 ```
badlop commented 2 years ago

The problem is that the script bin/ejabberdctl in this Docker image sets as erlang node name:

  ejabberd@$(hostname -s)

When you create a container, it gets a random machine name, and you will see when ejabberd starts:

ejabberd 0.0.0 is started in the node ejabberd@c76956d41c42 in 3.23s

You can also see in ./xmpp/db it creates a directory ejabberd@c76...

When you create another container, it gets another machine name, and ejabberdctl sets a new erlang node name, which means the old Mnesia database is unacceptable, and creates a new one.

Good news! there's a simple solution: add this to your docker-compose.yml so ejabberdctl in all containers set the same erlang node name, and the Mnesia database can be reused:

    environment:
      - ERLANG_NODE_ARG=ejabberd@main

Long term solution: there's a PR that addresses this: https://github.com/processone/docker-ejabberd/pull/73

hankjordan commented 2 years ago
> The problem is that the script `bin/ejabberdctl` in this Docker image sets as erlang node name: > > ``` > ejabberd@$(hostname -s) > ``` > > When you create a container, it gets a random machine name, and you will see when ejabberd starts: > > ``` > ejabberd 0.0.0 is started in the node ejabberd@c76956d41c42 in 3.23s > ``` > > You can also see in `./xmpp/db` it creates a directory `ejabberd@c76...` > > When you create another container, it gets another machine name, and ejabberdctl sets a new erlang node name, which means the old Mnesia database is unacceptable, and creates a new one. > > Good news! there's a simple solution: add this to your docker-compose.yml so ejabberdctl in all containers set the same erlang node name, and the Mnesia database can be reused: > > ```yaml > environment: > - ERLANG_NODE_ARG=ejabberd@main > ``` > > Long term solution: there's a PR that addresses this: #73

This solved my issue. Thank you for being so helpful.