Permission issues with Docker volumes

SkypLabs commented 1 year ago

Description

By default, the official Docker image runs with both UID and GID set to 991. This creates permission issues when using Docker volumes (not bind mounts).

Steps to reproduce

Here is a sample Docker Compose configuration file to reproduce the issue:

version: "3.9"

services:
  synapse:
    image: ${SYNAPSE_IMAGE:-docker.io/matrixdotorg/synapse}:${SYNAPSE_TAG:-v1.75.0}
    restart: unless-stopped
    environment:
      - SYNAPSE_CONFIG_DIR=/etc/synapse
    volumes:
      - ./envs/dev/config/synapse/:/etc/synapse:ro
      - synapse-data:/mnt/data
      - synapse-media:/mnt/media
    ports:
      - "8008:8008"

volumes:
  synapse-data:
  synapse-media:

The following Synapse configuration files must be placed in envs/dev/config/synapse/:

homeserver.yml:

# Configuration file for Synapse.
#
# This is a YAML file: see [1] for a quick introduction. Note in particular
# that *indentation is important*: all the elements of a list or dictionary
# should have the same indentation.
#
# [1] https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html
#
# For more information on how to configure Synapse, including a complete accounting of
# each option, go to docs/usage/configuration/config_documentation.md or
# https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html
server_name: "messenger"

pid_file: /mnt/data/homeserver.pid
log_config: /etc/synapse/log.config
media_store_path: /mnt/media

report_stats: false

listeners:
  - port: 8008
    tls: false
    type: http
    x_forwarded: true
    resources:
      - names: [client]
        compress: false

database:
  name: sqlite3
  args:
    database: /mnt/data/homeserver.db

registration_shared_secret: "zqhEN_rejCguMuuh-DSjlRU&-WouP1gD^NssoM3V&qXVkQMmxl"
macaroon_secret_key: "1N=E2Q#w7qT&mOAAa*E4h+WdJbV1aMKSB&x,&9_*rwu&pUIEvJ"
form_secret: "^0WMtS;:~81k0Qf:&JBf-iC8gUorQ2ZcgZdX68Fms9TOppmtv^"
signing_key_path: "/etc/synapse/signing.key"
trusted_key_servers:
  - server_name: "matrix.org"

# vim:ft=yaml

log.config:

version: 1

formatters:
  precise:
    format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'

handlers:
  console:
    class: logging.StreamHandler
    formatter: precise

loggers:
    synapse.storage.SQL:
        level: DEBUG

root:
    level: DEBUG
    handlers: [console]

disable_existing_loggers: false

signing.key

ed25519 a_cNvX 8LtJf/PmNzDdlX1EOIC8qHyKdItTyf6/b5RL5JsrKfI

Homeserver

Local deployment with Docker

Synapse Version

v1.75.0

Installation Method

Docker (matrixdotorg/synapse)

Database

SQLite

Workers

Single process

Platform

Docker engine running on Debian 11.

Configuration

No response

Relevant log output

Starting synapse with args -m synapse.app.homeserver --config-path /etc/synapse/homeserver.yaml                                                                              
This server is configured to use 'matrix.org' as its trusted key server via the                                                                                              
'trusted_key_servers' config option. 'matrix.org' is a good choice for a key                                                                                                 
server since it is long-lived, stable and trusted. However, some admins may                                                                                                  
wish to use another server for this purpose.                                                                                                                                 

To suppress this warning and continue using 'matrix.org', admins should set                                                                                                  
'suppress_key_server_warning' to 'true' in homeserver.yaml.                                                                                                                  
--------------------------------------------------------------------------------                                                                                             
2023-02-01 11:22:24,515 - root - 344 - WARNING - main - ***** STARTING SERVER *****                                                                                          
2023-02-01 11:22:24,515 - root - 345 - WARNING - main - Server /usr/local/lib/python3.9/site-packages/synapse/app/homeserver.py version 1.75.0                               
2023-02-01 11:22:24,515 - root - 350 - INFO - main - Server hostname: messenger                                                                                              
2023-02-01 11:22:24,515 - root - 351 - INFO - main - Instance name: master                                                                                                   
2023-02-01 11:22:24,516 - root - 352 - INFO - main - Twisted reactor: EPollReactor                                                                                           
2023-02-01 11:22:24,516 - synapse.app.homeserver - 337 - INFO - main - Setting up server                                                                                     
2023-02-01 11:22:24,516 - synapse.server - 308 - INFO - main - Setting up.                                                                                                   
2023-02-01 11:22:24,516 - synapse.app._base - 207 - ERROR - main - Exception during startup                                                                                  
Traceback (most recent call last):                                                                                                                                           
  File "/usr/local/lib/python3.9/site-packages/synapse/app/homeserver.py", line 340, in setup                                                                                
    hs.setup()                                                                                                                                                               
  File "/usr/local/lib/python3.9/site-packages/synapse/server.py", line 310, in setup                                                                                        
    self.datastores = Databases(self.DATASTORE_CLASS, self)                                                                                                                  
  File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/__init__.py", line 65, in __init__                                                                  
    with make_conn(database_config, engine, "startup") as db_conn:
  File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 151, in make_conn
    native_db_conn = engine.module.connect(**db_params)
sqlite3.OperationalError: unable to open database file
********************************************
 Error during initialisation:
    unable to open database file
 There may be more information in the logs. 
********************************************

Anything else that would be useful to know?

This issue seems related to https://github.com/matrix-org/synapse/pull/6389 which fixed a similar problem when generating a Synapse configuration file with the official Docker image.

exitflynn commented 1 year ago

can confirm same behaviour on my arch machine, looking into this and your solution!

CompoTypo commented 1 year ago

Any new developments with this issue? Running into the same "Permission Denied" error

SkypLabs commented 1 year ago

I think the problem comes from the assumption that one would run the generate command of the start.py tool as described here prior to starting the service. The generate command changes the ownership of the data directory whereas Synapse itself does nothing about it when spinning up a new instance.

MadLittleMods commented 1 year ago

Related to:

matrix-org / synapse