Closed petersmithca closed 7 years ago
You likely need to do sudo chown -R 991:991 public/system
And we gotta add that to the documentation :thinking:
Thank you, I got it. Will close this now. Appreciate the help!
very strange issue actually, since keep in mind I'm also running telegraf on my server. This is user 991 on the host PC in my case. Although I'm running docker images of Mastodon.
@Gargron How about in docker ?
@Gargron is uid 991 mastodon on your system? (yes, confirmed by digital ocean's mastodon documentation, the defacto docker container for mastodon uses uid 991)
edit, found the issue, the systemd unit files were prohibiting access to the mastodon user's working directory at the very least, the systemd unit files need to have this line added to the service section
# /etc/systemd/system/mastodon-*
[Service]
ReadWritePaths=/home/mastodon/live/public /home/mastodon/live/tmp
I made these edits to all of my systemd service files, I don't think added the group was strictly necessary, its value is probably implied anyway.
--- /home/mastodon/live/dist/mastodon-web.service 2021-06-02 19:26:02.000000000 -0700
+++ /etc/systemd/system/mastodon-web.service 2021-07-05 13:22:18.618756714 -0700
@@ -5,10 +5,12 @@
[Service]
Type=simple
User=mastodon
+Group=mastodon
WorkingDirectory=/home/mastodon/live
Environment="RAILS_ENV=production"
Environment="PORT=3000"
ExecStart=/home/mastodon/.rbenv/shims/bundle exec puma -C config/puma.rb
ExecReload=/bin/kill -SIGUSR1 $MAINPID
TimeoutSec=15
Restart=always
@@ -39,6 +41,7 @@
# System Call Filtering
SystemCallArchitectures=native
SystemCallFilter=~@clock @cpu-emulation @debug @keyring @module @mount @obsolete @raw-io @reboot @resources @setuid @swap
+ReadWritePaths=/srv/mastodon/mastodon-v3.4.1/public /srv/mastodon/mastodon-v3.4.1/tmp
[Install]
WantedBy=multi-user.target
I think i have it set correctly and I have the issue still
(mastodon v3.4.1)
$ namei -l /home/mastodon/live/public/system/
f: /home/mastodon/live/public/system/
drwxr-xr-x root root /
drwxr-xr-x root root home
drwxr-xr-x mastodon mastodon mastodon
drwxr-xr-x mastodon mastodon live
drwxr-xr-x mastodon mastodon public
drwxr-xr-x mastodon mastodon system
I was playing whack-a-mole with the error messages doing, and others:
sudo -Hsu mastodon mkdir /home/mastodon/live/public/system/accounts/headers
however the latest complaint looks like it need a dynamic directory not a fixed one, so I am losing hope this is something I can correct with a one-time directory creation and I might instead need to author a fuse driver to generate any directory the app wants
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] [paperclip] Trying to link /tmp/RackMultipart20210704-717-1mo4iqj.jpeg to /tmp/ce92b58b669b9a4a3589081bcff2b65320210704-717-19b02yi.jpeg
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] [paperclip] Trying to link /tmp/ce92b58b669b9a4a3589081bcff2b65320210704-717-19b02yi.jpeg to /tmp/95b1ec61fa62a365b62965e0a91663f520210704-717-2ugmdp.jpeg
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] Command :: file -b --mime '/tmp/95b1ec61fa62a365b62965e0a91663f520210704-717-2ugmdp.jpeg'
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] Command :: identify -format %m '/tmp/ce92b58b669b9a4a3589081bcff2b65320210704-717-19b02yi.jpeg[0]'
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] Command :: convert '/tmp/ce92b58b669b9a4a3589081bcff2b65320210704-717-19b02yi.jpeg[0]' -auto-orient -resize "1149x653>" -strip '/tmp/93306a0fb8056377685fc48b10de3bfc20210704-717-m5m5v5'
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] [paperclip] Trying to link /tmp/93306a0fb8056377685fc48b10de3bfc20210704-717-m5m5v5 to /tmp/0debe706011854f5f4071c4c782aec0b20210704-717-1t5yfuf
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] [paperclip] Trying to link /tmp/0debe706011854f5f4071c4c782aec0b20210704-717-1t5yfuf to /tmp/95b1ec61fa62a365b62965e0a91663f520210704-717-gr6560.jpeg
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] Command :: file -b --mime '/tmp/95b1ec61fa62a365b62965e0a91663f520210704-717-gr6560.jpeg'
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] method=PUT path=/settings/profile format=html controller=Settings::ProfilesController action=update status=500 error='Errno::EROFS: Read-only file system @ dir_s_mkdir - /home/mastodon/live/public/system/accounts/headers/106' duration=178.47 view=0.00 db=2.37
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4]
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] Errno::EROFS (Read-only file system @ dir_s_mkdir - /home/mastodon/live/public/system/accounts/headers/106):
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4]
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] lib/paperclip/attachment_extensions.rb:55:in `block in save'
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] lib/paperclip/attachment_extensions.rb:61:in `save'
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] app/services/update_account_service.rb:8:in `call'
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] app/controllers/settings/profiles_controller.rb:11:in `update'
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] app/controllers/concerns/localized.rb:16:in `block in set_locale'
Jul 04 20:53:16 hostname bundle[439]: [2b23eae2-b66d-4308-a4a1-8945c52ec1a4] app/controllers/concerns/localized.rb:15:in `set_locale'
also when all the directories exist, it still cant write the destination file?
Jul 04 21:06:32 hostname bundle[439]: [c40aa3c7-f2c4-40b6-a7af-6a8ba3026835] Errno::EROFS (Read-only file system @ rb_sysopen - /home/mastodon/live/public/system/accounts/headers/106/525/633/591/399/544/original/a689cde116e794ef.jpeg):
even with folder mode 0o777
$ sudo -Hsu mastodon namei -l /home/mastodon/live/public/system/accounts/headers/106/525/633/591/399/544/original/db887fa9c4991702.jpeg
f: /home/mastodon/live/public/system/accounts/headers/106/525/633/591/399/544/original/db887fa9c4991702.jpeg
drwxr-xr-x root root /
drwxr-xr-x root root home
drwxr-xr-x mastodon mastodon mastodon
drwxr-xr-x mastodon mastodon live
drwxr-xr-x mastodon mastodon public
drwxr-xr-x mastodon mastodon system
drwxr-xr-x mastodon mastodon accounts
drwxr-xr-x mastodon mastodon headers
drwxr-xr-x mastodon mastodon 106
drwxr-xr-x mastodon mastodon 525
drwxr-xr-x mastodon mastodon 633
drwxr-xr-x mastodon mastodon 591
drwxr-xr-x mastodon mastodon 399
drwxr-xr-x mastodon mastodon 544
drwxrwxrwx mastodon mastodon original
db887fa9c4991702.jpeg - No such file or directory
the file-system is certainly writable too
$ df -h /home/mastodon/live/public/system/accounts/headers/106/525/633/591/399/544/original/
Filesystem Size Used Avail Use% Mounted on
/dev/vda2 18G 5.8G 11G 36% /
$ mount | grep /dev/vda2
/dev/vda2 on / type ext4 (rw,relatime,errors=remount-ro)
fairly sure it isn't apparmor or selinux blocking it either, but given the code is running as the same user that I am using to create the directories manually, seems like it shoudln't be an issue. additionally I tried with selinux and apparmor disabled on boot and same issue
For anyone coming here looking for filesystem errors that look similar to this:
Apr 28 07:24:38 alice bundle[2847665]: 2022-04-28T12:24:38.774Z pid=2847665 tid=1nz7h WARN: Errno::EROFS: Read-only file system @ dir_s_mkdir - /var/lib/mastodon/public/system/cache/accounts/avatars/108/209/662
The solution set by @thorsummoner is a good one.
Adding the following lines to BOTH the mastodon-web
and the mastodon-sidekiq
systemd unit files.
In arch linux these files live in /lib/systemd/system if you want to edit them directly instead of overloading the values.
ReadWritePaths=/var/lib/mastodon/tmp /var/lib/mastodon/public/system
Otherwise you can overload directly by using
systemctl edit mastodon-web
systemctl edit mastodon-sidekiq
and adding the following to BOTH
[Service]
ReadWritePaths=/var/lib/mastodon/tmp /var/lib/mastodon/public/system
This worked for me too after changing local storage path with environment variables!
Having this same problem, proposed changes don't appear to help. I used the actual location to my tmp and public dirs (/home/mastodon/live/tmp and /home/mastodon/live/public) instead of what was posted above since I have nothing in /var/lib/mastodon
Just upgraded to 4.0.2 (from 3.x version) and started seeing this. I don't believe it was an issue before. And it only happens with GIFs.
This is all I see in the logs:
Nov 19 21:53:36 pone.social bundle[18390]: [34e87808-c3de-45e3-be24-75ee6a7c040b] [paperclip] Trying to link /tmp/RackMultipart20221119-18476-8r8an5.gif to /tmp/0dd34e375351606664ff3ae6be58565c20221119-18476-xergpj.gif
Nov 19 21:53:36 pone.social bundle[18390]: [34e87808-c3de-45e3-be24-75ee6a7c040b] [paperclip] Trying to link /tmp/0dd34e375351606664ff3ae6be58565c20221119-18476-xergpj.gif to /tmp/1fca4b78667663c2093ecce55f3fdd7920221119-18476-suh9r0.gif
Nov 19 21:53:36 pone.social bundle[18390]: [34e87808-c3de-45e3-be24-75ee6a7c040b] Command :: file -b --mime '/tmp/1fca4b78667663c2093ecce55f3fdd7920221119-18476-suh9r0.gif'
Nov 19 21:53:36 pone.social bundle[18390]: [34e87808-c3de-45e3-be24-75ee6a7c040b] method=POST path=/api/v2/media format=html controller=Api::V2::MediaController action=create status=500 duration=16.78 view=0.44 db=1.8
Solved my issue, there were a few things involved:
After doing those two steps, things seem to be working well again.
Hello, I just installed Mastodon on Ubuntu 22 following the official documentation
Unfortunately I cannot upload images, I get the following error from the server:
$ journalctl -u mastodon-web
[...]
Errno::EACCES (Permission denied @ dir_s_mkdir - /home/mastodon/live/public/system):
[...]
I tried the above solution of changing ownership of public/system
. But this comment is from 5 years ago, and this folder doesn't exist on my server.
Any help on where to look for an answer would be grand. Thanks
He montado la última versión de mastodon con docker y al agregar un imagen a mi perfil me aparece el siguiente error: Ya intenté con cambiar de propietario la carpeta y no funciona.
Errno::EACCES (Permission denied @ dir_s_mkdir - /opt/mastodon/public/system/media_attachments):
I'm also experiencing this. I'm using docker-compose.
web_1 | [a9d7fec0-997f-4466-bca0-7e7798d630cd] Chewy request strategy is `mastodon`
web_1 | [a9d7fec0-997f-4466-bca0-7e7798d630cd] method=GET path=/settings/profile format=html controller=Settings::ProfilesController action=show status=200 duration=469.63 view=344.17 db=75.49
web_1 | [a58b9756-3e6b-4ad5-9c7e-787cb337008d] method=HEAD path=/health format=*/* controller=HealthController action=show status=200 duration=7.57 view=2.18 db=0.00
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] Chewy request strategy is `mastodon`
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] [paperclip] Trying to link /tmp/RackMultipart20230425-19-5qvxoc.jpg to /tmp/25794f3c6d8dc7fce807b88d4583cbfc20230425-19-wzgngr.jpg
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] [paperclip] Trying to link /tmp/25794f3c6d8dc7fce807b88d4583cbfc20230425-19-wzgngr.jpg to /tmp/6ed5c5dc239af799f8da68dd86db6fe520230425-19-enp45q.jpg
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] Command :: file -b --mime '/tmp/6ed5c5dc239af799f8da68dd86db6fe520230425-19-enp45q.jpg'
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] Command :: identify -format %m '/tmp/25794f3c6d8dc7fce807b88d4583cbfc20230425-19-wzgngr.jpg[0]'
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] Command :: convert '/tmp/25794f3c6d8dc7fce807b88d4583cbfc20230425-19-wzgngr.jpg[0]' -auto-orient -resize "400x" -crop "400x400+0+0" +repage +profile "!icc,*" +set modify-date +set create-date '/tmp/d3da67cfd443aab5e9a19a60be838e0d20230425-19-v4f64t'
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] [paperclip] Trying to link /tmp/d3da67cfd443aab5e9a19a60be838e0d20230425-19-v4f64t to /tmp/f27385a355cebed5a2bb1783909fe4a320230425-19-7raecq
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] method=PUT path=/settings/profile format=html controller=Settings::ProfilesController action=update status=500 error='Errno::EACCES: Permission denied @ dir_s_mkdir - /opt/mastodon/public/system/accounts' duration=273.90 view=0.00 db=48.95
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49]
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] Errno::EACCES (Permission denied @ dir_s_mkdir - /opt/mastodon/public/system/accounts):
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49]
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] lib/paperclip/attachment_extensions.rb:87:in `block in save'
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] lib/paperclip/attachment_extensions.rb:93:in `save'
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] app/services/update_account_service.rb:8:in `call'
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] app/controllers/settings/profiles_controller.rb:11:in `update'
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] app/controllers/concerns/localized.rb:11:in `set_locale'
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] lib/mastodon/rack_middleware.rb:9:in `call'
web_1 | [ccffe741-e10c-4586-b0d4-ff581ee68c49] lib/public_file_server_middleware.rb:18:in `call'
version: "3"
services:
db:
image: postgres:14-alpine
shm_size: 256mb
healthcheck:
test: ["CMD", "pg_isready", "-U", "postgres"]
volumes:
- /mnt/social/postgres-mastodon:/var/lib/postgresql/data
environment:
- POSTGRES_USER=mastodon
- POSTGRES_PASSWORD=mastodon
redis:
image: redis:7-alpine
healthcheck:
test: ["CMD", "redis-cli", "ping"]
volumes:
- /mnt/social/redis-mastodon:/data
es:
image: docker.elastic.co/elasticsearch/elasticsearch:7.17.4
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m -Des.enforce.bootstrap.checks=true"
- "xpack.license.self_generated.type=basic"
- "xpack.security.enabled=false"
- "xpack.watcher.enabled=false"
- "xpack.graph.enabled=false"
- "xpack.ml.enabled=false"
- "bootstrap.memory_lock=true"
- "cluster.name=es-mastodon"
- "discovery.type=single-node"
- "thread_pool.write.queue_size=1000"
healthcheck:
test:
[
"CMD-SHELL",
"curl --silent --fail localhost:9200/_cluster/health || exit 1",
]
volumes:
- /mnt/social/mastodon-es:/usr/share/elasticsearch/data
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
web:
image: tootsuite/mastodon:v4.1
env_file: env
command: bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000"
healthcheck:
# prettier-ignore
test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:3000/health || exit 1']
ports:
- "127.0.0.1:3000:3000"
depends_on:
- db
- redis
- es
volumes:
- /mnt/social/mastodon:/mastodon/public/system
streaming:
build: .
image: tootsuite/mastodon:v4.1
env_file: env
command: node ./streaming
healthcheck:
# prettier-ignore
test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:4000/api/v1/streaming/health || exit 1']
ports:
- "127.0.0.1:4000:4000"
depends_on:
- db
- redis
sidekiq:
image: tootsuite/mastodon:v4.1
env_file: env
command: bundle exec sidekiq
depends_on:
- db
- redis
volumes:
- /mnt/social/mastodon:/mastodon/public/system
healthcheck:
test: ["CMD-SHELL", "ps aux | grep '[s]idekiq\ 6' || false"]
I'm sure Im missing something basic, but any help will be appreciated. Attempting to load a profile picture and get the following message in the logs. Sure its a permission thing, but not sure what to change where.
method=PUT path=/settings/profile format=html controller=Settings::ProfilesController action=update status=500 error='Errno::EACCES: Permission denied @ dir_s_mkdir - /mastodon/public/system/accounts' duration=243.94 view=0.00 db=12.91
master
(If you're a user, don't worry about this).