`generate` option of docker image lacks a way to specify the data path

[richvdh]

When you use the generate option of the docker image to generate a config file, there is no way to tell it where you want to put the media store etc, and it uses /.

Then, when you try to start synapse using the generated config, you get:

PermissionError: [Errno 13] Permission denied: '/media_store'

We should probably use /data by default (and recommend that a volume is mounted there), with an env var to allow you override the path. We also need to chown it so that synapse can write to it.

A workaround is:

• Add -w /data to the generate commandline, then
• docker run <volume_options> --entrypoint=chown matrixdotorg/synapse:latest 901 /data

[jeffcasavant]

It looks like the docker container wants to use /data for all of the persistent stuff (e.g. generated server keys, certs, and homeserver.yaml). It should probably use /data/media by default (or something like that).

[richvdh]

It should probably use /data/media by default (or something like that).

by default, the media store is put in a media_store subdirectory of the data directory, so if the data directory is /data, the media store will be /data/media_store. Or is that not what you mean?

[jeffcasavant]

That's exactly what I mean, I completely forgot how it works. Disregard 👍

[relink2013]

Is it not possible to just simply allow the user to map /conf to an external volume, and edit the existing homeserver.yaml that is generated inside the container? This is what im doing right now, and it works great (so far anyway).

I even went as far as copying over some lines from the "normal" homeserver.yaml, into the stripped down "docker version" and Synapse picks them up just fine. So far everything works like id expect it to. I'm not sure if there is a danger in doing it this way though. I did pass the volume as read only so the container cant overwrite my file, just being cautious.

[richvdh]

@relink2013 of course that's fine. The point is just that the defaults in the generated file should be sensible and not require you to edit them.

[professor-farnsworth]

Hey everyone, is there a way to make this work with docker-compose? Forgive me if I don't understand.... isn't setting the entrypoint to a chown command overriding it? If I set "entrypoint: chown 901:901 /data", the container just exits with code 0, I think cause it just executes the chown and the container does nothing.

I generated my own homeserver.yaml and corrected the "media_store" path to /data/media_store. How exactly would I fix the permissions for the container while still maintaining the existing "start.py" entrypoint?

And sorry if this is real obvious, but how does overriding the entrypoint help? Can you somehow edit that for the container after it runs and does the chown?

[rolfbachmann]

I tried to use docker-compose to setup, but didn't work out of the box. The following steps worked for me:

• Generate the config
• Create needed folders in docker volume and change ownership:

cd path/to/docker/volume
mkdir {media_store,uploads,logs}
chown 991:991 media_store/ uploads/ logs/

• Update homeserver.yaml

media_store_path: "/data/media_store
uploads_path: "/data/uploads"

• Update log config (<yourdomain>.log.config):

handlers:
    file:
        filename: /data/logs/homeserver.log

Notes:

• 991 is the default uid (see start.py)
• I didn't want to change the owner of the docker volume directly (e.g. synapse/_data) so I moved the logs to an own subfolder.

[professor-farnsworth]

Thanks for the reply! You got the right idea. Trying to modify the entrypoint and/or the docker file is a roundabout way of attempting to fix the permissions. And for some reason this issue and the linked issue have fixes that think the group:user is 901.

Through trial and error, I figured out the same thing:" fix the homeserver.yaml's path references and the log config, and chown the data folder to 991:991.

Back to the original issue, I believe it would be more friendly for the generate command to allow you to specify the data path. The bigger issue is when you run it via docker and it doesn't setup the permissions for you.

[Atomique]

I tried to use docker-compose to setup, but didn't work out of the box. The following steps worked for me:

• Generate the config
• Create needed folders in docker volume and change ownership:

cd path/to/docker/volume
mkdir {media_store,uploads,logs}
chown 991:991 media_store/ uploads/ logs/

• Update homeserver.yaml

media_store_path: "/data/media_store
uploads_path: "/data/uploads"

• Update log config (<yourdomain>.log.config):

handlers:
    file:
        filename: /data/logs/homeserver.log

Notes:

• 991 is the default uid (see start.py)
• I didn't want to change the owner of the docker volume directly (e.g. synapse/_data) so I moved the logs to an own subfolder.

I have the same problem. I can start the server without an persistent configuration-file. If I use the generate-command with the config-environment-variable, it generates a config-file but the container doesnt start anymore. I need this configuration file to change, because I want to connect the matrix-server to my Active Directory. I tried to change the values you told us, but it doesnt help at all.

[richvdh]

fixed by #5563 .

Hey everyone, is there a way to make this work with docker-compose?

This is unrelated to this issue...

[dessalines]

I just ran into this error, it tried to rec_walk my entire filesystem. It should probably be reopened, at least until the docker-compose.yml is updated.

[richvdh]

this issue is unrelated to docker-compose as far as I am aware.

The fix is in the v1.1.0 images.

[shiipou]

To fix that, got into your container with :

> docker ps
CONTAINER ID        IMAGE                               COMMAND                  CREATED             STATUS                          PORTS                     NAMES
c90fcf8192af        vectorim/riot-web:develop           "nginx -g 'daemon of…"   23 hours ago        Up 23 hours                     80/tcp                    chat_web.1.vie63blf7kmvohextk3733ome
5da798b100bd        kamax/mxisd:latest                  "/start.sh"              23 hours ago        Up 23 hours                     8090/tcp                  chat_id.1.garduqx3o0p5cx0ry6a48zoud
469eab54f50e        matrixdotorg/synapse:latest         "/start.py"              23 hours ago        Up 23 hours                     8008-8009/tcp, 8448/tcp   chat_matrix.1.sdkw4te3mryqlv8syktrrhj69

Look at the container ID of your matrixdotorg/synapse container and execute this command to get into it :

docker exec -it 469eab54f50e sh

Now your in the bash of your container, exec this command to change the owner of the directory :

chown -R 991:991 /data/media_store

You can see the user id by using :

> ls -lia /data/*.signing.key
5640776 -rw-r--r--    1 991      991             59 Mar 26 09:03 /data/matrix.chat.nocturlab.fr.signing.key

The Id you look for is 991 and 991

[dharmendrakariya]

@shiipou Your workaround works for me

[BartDrown]

I tried to use docker-compose to setup, but didn't work out of the box. The following steps worked for me:

• Generate the config
• Create needed folders in docker volume and change ownership:

cd path/to/docker/volume
mkdir {media_store,uploads,logs}
chown 991:991 media_store/ uploads/ logs/

• Update homeserver.yaml

media_store_path: "/data/media_store
uploads_path: "/data/uploads"

• Update log config (<yourdomain>.log.config):

handlers:
    file:
        filename: /data/logs/homeserver.log

Notes:

• 991 is the default uid (see start.py)
• I didn't want to change the owner of the docker volume directly (e.g. synapse/_data) so I moved the logs to an own subfolder.

Thanks @rolfbachmann, your solution works great.

Using docker-compose may be unrelated directly to the main issue, but still when using it, the error occurs. I think it's is because image process tries to access files that are not yet created and it fails on try.

[richvdh]

I'm confused about what the problem is. As far as I'm aware the original bug is fixed. If there is a problem with the docker-compose file then I suggest opening a new issue (though note that, as with all things under contrib, it is maintained by the community so won't be fixed unless someone can contribute a PR).