Closed ghost closed 1 year ago
It appears you aren’t using nginx or other reverse proxy. If that is true gunicorn must be enabled.
GUNICORN_MEDIA=1
It appears you aren’t using nginx or other reverse proxy. If that is true gunicorn must be enabled.
GUNICORN_MEDIA=1
Yep it's behind nginx. I'm using SWAG (nginx/letsencrypt) .. also mentioned in the original issue that I'm using the default proxy config (the one provided by SWAG) unmodified from its original template. (/config/nginx/proxy-configs/recipes.subdomain.conf
)
I'm not using any external mountable storage for the media, it's just placed in my /home/lsio/recipes/
directory under a /media/
subdirectory as defined by my docker-compose. The uploaded files appear there correctly, but they 404 in Tandoor.
$ ls /home/lsio/recipes/media/recipes/
795f2e16-19ff-45e2-8dc4-ca04b62ccf6c_1.jpeg
nginx:
## Version 2021/05/18
# make sure that your dns has a cname set for recipes
# make sure to mount /media/ in your swag container to point to your Recipes Media directory
# if using Authelia use this one:
# Doc: https://vabene1111.github.io/recipes/install/docker/#using-proxy-authentication
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name recipes.*;
include /config/nginx/ssl.conf;
client_max_body_size 0;
# enable for ldap auth, fill in ldap details in ldap.conf
#include /config/nginx/ldap.conf;
# enable for Authelia
#include /config/nginx/authelia-server.conf;
# serve media files
location /media/ {
alias /media/;
}
location / {
# enable the next two lines for http auth
#auth_basic "Restricted";
#auth_basic_user_file /config/nginx/.htpasswd;
# enable the next two lines for ldap auth
#auth_request /auth;
#error_page 401 =200 /ldaplogin;
# enable for Authelia
#include /config/nginx/authelia-location.conf;
include /config/nginx/proxy.conf;
include /config/nginx/resolver.conf;
set $upstream_app recipes;
set $upstream_port 8080;
set $upstream_proto http;
proxy_pass $upstream_proto://$upstream_app:$upstream_port;
}
}
Ah got it.
You are serving up /media
but your pictures are located at /home/lsio/recipes/media
I tried that as well :\
404 (tested with and without trailing slash)
location /media/ {
alias /home/lsio/recipes/media/;
}
I also tried updating the MEDIA_URL env var to MEDIA_URL=/home/lsio/recipes/media/
in combination with the above nginx config.
404
location /media/ {
alias /media/;
}
I wasn't entirely sure if the /media/ path was relative to the bind mounts in the compose or if I had to enter the full path.. but neither seems to be working
Are there pictures in that directory?
Yep.. it's there
ls /home/lsio/recipes/media/recipes/
795f2e16-19ff-45e2-8dc4-ca04b62ccf6c_1.jpeg
Once you configure the alias correctly please share the relevant logs.
Here are some fresh logs.
Here are the steps included in this log:
$ docker-compose logs
Attaching to recipes, db_recipes
recipes | Checking configuration...
recipes | Waiting for database to be ready...
recipes | Database is ready
recipes | Migrating database
recipes | Operations to perform:
recipes | Apply all migrations: account, admin, auth, authtoken, contenttypes, cookbook, sessions, sites, socialaccount
recipes | Running migrations:
recipes | No migrations to apply.
recipes | Your models in app(s): 'cookbook' have changes that are not yet reflected in a migration, and so won't be applied.
recipes | Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.
recipes | Generating static files
recipes | js-reverse file written to /opt/recipes/cookbook/static/django_js_reverse
recipes |
recipes | 1 static file copied to '/opt/recipes/staticfiles', 1032 unmodified, 1956 post-processed.
recipes | Done
recipes | [2022-04-24 13:27:44 -0500] [1] [INFO] Starting gunicorn 20.1.0
recipes | [2022-04-24 13:27:44 -0500] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
recipes | [2022-04-24 13:27:44 -0500] [1] [INFO] Using worker: sync
recipes | [2022-04-24 13:27:44 -0500] [11] [INFO] Booting worker with pid: 11
recipes | 172.19.0.3 - - [24/Apr/2022:13:28:55 -0500] "PUT /api/recipe/1/image/ HTTP/1.1" 200 70 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes | 172.19.0.3 - - [24/Apr/2022:13:29:02 -0500] "GET /edit/recipe/internal/1/ HTTP/1.1" 200 17992 "https://recipes.mydomain.com/view/recipe/1" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes | 172.19.0.3 - - [24/Apr/2022:13:29:02 -0500] "GET /static/django_js_reverse/reverse.4bbfcb16b9d1.js HTTP/1.1" 200 0 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /api/recipe/1/ HTTP/1.1" 200 623 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /api/unit/?query=&page=1&page_size=25 HTTP/1.1" 200 52 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /api/food/?query=&page=1&page_size=25 HTTP/1.1" 200 52 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /api/keyword/?query=&page=1&page_size=25 HTTP/1.1" 200 52 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /api/user-file/?query= HTTP/1.1" 200 2 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /api/recipe/?query=&page=1&page_size=25 HTTP/1.1" 200 554 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
recipes | 172.19.0.3 - - [24/Apr/2022:13:29:03 -0500] "GET /service-worker.js HTTP/1.1" 200 54271 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
You’ll need to provide web logs from swag. Tandoor isn’t going to receive the image requests.
swag nginx log
x.x.x.x - - [24/Apr/2022:13:28:55 -0500] "PUT /api/recipe/1/image/ HTTP/2.0" 200 70 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:28:55 -0500] "GET /media/recipes/85a4a9ae-368b-4fbf-8fd0-52fb9191fc49_1.jpeg HTTP/2.0" 404 107 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:02 -0500] "GET /edit/recipe/internal/1/ HTTP/2.0" 200 3604 "https://recipes.mydomain.com/view/recipe/1" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:02 -0500] "GET /static/django_js_reverse/reverse.4bbfcb16b9d1.js HTTP/2.0" 200 4909 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /api/recipe/1/ HTTP/2.0" 200 623 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /api/unit/?query=&page=1&page_size=25 HTTP/2.0" 200 52 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /api/food/?query=&page=1&page_size=25 HTTP/2.0" 200 52 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /api/keyword/?query=&page=1&page_size=25 HTTP/2.0" 200 52 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /api/user-file/?query= HTTP/2.0" 200 2 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /api/recipe/?query=&page=1&page_size=25 HTTP/2.0" 200 554 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /media/recipes/85a4a9ae-368b-4fbf-8fd0-52fb9191fc49_1.jpeg HTTP/2.0" 404 107 "https://recipes.mydomain.com/edit/recipe/internal/1/" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
x.x.x.x - - [24/Apr/2022:13:29:03 -0500] "GET /service-worker.js HTTP/2.0" 200 54271 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0"
Small suggestion while we're chatting about this (thank you for your help by the way)
On the "System" screen in Tandoor, I think maybe this message should include a check to make sure the media files are accessible
That log just shows the 404. You’ll need a deeper level of logging or another source of logs to determine why.
I agree that it would be helpful, but tandoor has no visibility into how those media files are accessed, so I’m not sure how that would even be accomplished.
That log just shows the 404. You’ll need a deeper level of logging or another source of logs to determine why.
I agree that it would be helpful, but tandoor has no visibility into how those media files are accessed, so I’m not sure how that would even be accomplished.
Maybe something like ..
Create a dummy file to /media -> requests.get(https://full.path/to/media/file.jpeg) -> Check response -> Test OK -> remove dummy file
Just spitballing
I will dig more into this and see if I can figure something out. Thank you. I'm sure it's just a simple configuration issue somewhere.
Ah ha.. I think I figured out my problem. Now to figure out the best solution.
I have swag/nginx running under UID 1003 I have Tandoor running under UID 1001
Tandoor's media files are under UID 1001's home folder. Nginx (1003) is trying to access 1001's home folder.
So when nginx tries to alias /home/user1001/recipes/media
it's lacking the permission to access those files.
Thus.. 404 file not found. I believe it is a permission error.
Up until this point, I haven't had trouble keeping the swag/lsio containers separated into different linux users, but this time it's an issue since I need to expose the media directory in nginx from a directory it has no permission to access.
EDIT: Er... maybe not. I tested this theory by moving the media outside of the home dir into the system /media/ dir. Same issue. Writes work fine for saving media, but still 404's. Ugh. If I had stopped to consider the fact that writing worked before, I wouldn't even bothered testing this. Definitely not a permission problem.
So that didn't work.. but I got some better logs.
This has me scratching my head. It says file not found and it's looking for it in the correct path.. but...
2022/04/24 19:03:45 [error] 518#518: *1 open() "/home/lsio/recipes/media/recipes/5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg" failed (2: No such file or directory), client: x.x.x.x, server: recipes.*, request: "GET /media/recipes/5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg HTTP/2.0", host: "recipes.mydomain.com", referrer: "https://recipes.mydomain.com/search/"
2022/04/24 19:04:08 [error] 518#518: *1 open() "/home/lsio/recipes/media/recipes/5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg" failed (2: No such file or directory), client: x.x.x.x, server: recipes.*, request: "GET /media/recipes/5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg HTTP/2.0", host: "recipes.mydomain.com", referrer: "https://recipes.mydomain.com/search/"
2022/04/24 19:04:47 [warn] 518#518: *31 an upstream response is buffered to a temporary file /var/lib/nginx/tmp/proxy/2/00/0000000002 while reading upstream, client: x.x.x.x, server: recipes.*, request: "GET /static/vue/js/chunk-vendors.bb5a09569e79.js HTTP/2.0", upstream: "http://172.19.0.9:8080/static/vue/js/chunk-vendors.bb5a09569e79.js", host: "recipes.mydomain.com", referrer: "https://recipes.mydomain.com/search/"
2022/04/24 19:04:50 [error] 518#518: *31 open() "/home/lsio/recipes/media/recipes/5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg" failed (2: No such file or directory), client: x.x.x.x, server: recipes.*, request: "GET /media/recipes/5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg HTTP/2.0", host: "recipes.mydomain.com", referrer: "https://recipes.mydomain.com/search/"
The file that it says is not found, is clearly there....
# ls -la /home/lsio/recipes/media/recipes/
total 228
drwxr-xr-x 2 root root 4096 Apr 25 00:03 .
drwxr-xr-x 3 root root 4096 Apr 25 00:03 ..
-rw-r--r-- 1 root root 223091 Apr 25 00:03 5a035438-2482-4c18-86dc-b147ab26c67e_1.jpeg
root@server:~/compose/recipes#
What user is running swag? The file is owned by root.
swag
is running swag, lsio
is the UID on my compose (/home/lsio
is where the files go).
I've already tried recursively chowning the /media/ path to both the lsio and swag users.. sadly there's no effect
Here is a put and a get side by side.. same file uploaded and then attempted to be downloaded
2022/04/24 19:37:02 [warn] 517#517: *1 a client request body is buffered to a temporary file /var/lib/nginx/tmp/client_body/0000000002, client: x.x.x.x, server: recipes.*, request: "PUT /api/recipe/1/image/ HTTP/2.0", host: "recipes.mydomain.com", referrer: "https://recipes.mydomain.com/edit/recipe/internal/1/"
2022/04/24 19:37:03 [error] 517#517: *1 open() "/home/lsio/recipes/media/recipes/8cb2467b-7bf4-4afa-96d7-9c1fe26167ec_1.jpeg" failed (2: No such file or directory), client: x.x.x.x, server: recipes.*, request: "GET /media/recipes/8cb2467b-7bf4-4afa-96d7-9c1fe26167ec_1.jpeg HTTP/2.0", host: "recipes.mydomain.com", referrer: "https://recipes.mydomain.com/edit/recipe/internal/1/"
Put is to tandoor, get is from nginx. Not sure that is going to help much. It appears tandoor is accessing the file system as root, which nginx doesn’t won’t have access to. I don’t know enough about docker to help change that behavior.
Thanks, I'll keep digging.
I may have possibly hit on something. I changed the /media/ path in my nginx proxy config to media/
(no leading slash) and it spit out this
514#514: *1 open() "/var/lib/nginx/media/recipes/8cb2467b-7bf4-4afa-96d7-9c1fe26167ec_1.jpeg" failed (2: No such file or directory)
.. which is internal to the container. So I'm wondering if that's why I'm getting a 404 for a file that clearly exists on the filesystem. When it says it's looking for /home/lsio/..../media
, it's actually looking at <container_root>/home/lsio/.../media
which doesn't exist.
Maybe.. possibly. tableflip.gif
Did you get this sorted?
The way I have it working is:
services:
swag:
image: lscr.io/linuxserver/swag
container_name: swag
...
volumes:
- ./volumes/swag:/config
- ./volumes/swag/nginx/conf.d:/etc/nginx/conf.d
- ./volumes/recipes/mediafiles:/recipes/media
...
restart: always
Recipes environment variable "MEDIA_URL"
MEDIA_URL=https://recipes.domain.com/media/
SWAG recipes configuration
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name recipes.*;
include /config/nginx/ssl.conf;
include /config/nginx/geoipblock.conf;
client_max_body_size 0;
# enable for ldap auth, fill in ldap details in ldap.conf
#include /config/nginx/ldap.conf;
# enable for Authelia
include /config/nginx/authelia-server.conf;
# serve media files
location /media {
alias /recipes/media/;
}
location / {
# enable the next two lines for http auth
#auth_basic "Restricted";
#auth_basic_user_file /config/nginx/.htpasswd;
# enable the next two lines for ldap auth
#auth_request /auth;
#error_page 401 =200 /ldaplogin;
# enable for Authelia
#include /config/nginx/authelia-location.conf;
include /config/nginx/proxy.conf;
include /config/nginx/resolver.conf;
set $upstream_app recipes;
set $upstream_port 8080;
set $upstream_proto http;
proxy_pass $upstream_proto://$upstream_app:$upstream_port;
}
location ~ /admin|/system {
# enable the next two lines for http auth
#auth_basic "Restricted";
#auth_basic_user_file /config/nginx/.htpasswd;
# enable the next two lines for ldap auth
#auth_request /auth;
#error_page 401 =200 /ldaplogin;
# enable for Authelia
include /config/nginx/authelia-location.conf;
include /config/nginx/proxy.conf;
resolver 127.0.0.11 valid=30s;
set $upstream_app recipes;
set $upstream_port 8080;
set $upstream_proto http;
proxy_pass $upstream_proto://$upstream_app:$upstream_port;
}
}
Hope this helps :)
I am also getting 404s and I've tried running it multiple ways
this command line doesn't even work for me, cant access it: https://docs.tandoor.dev/install/docker/#docker
plain: https://docs.tandoor.dev/install/docker/#plain
getting 404s without any sort of proxy and with haproxy
You can try to mount additional volume to swag based on this example
In my case:
version: "3"
networks:
lsio:
external:
name: lsio
services:
swag:
image: ghcr.io/linuxserver/swag:latest
container_name: swag
cap_add:
- NET_ADMIN
environment:
- PUID=1000
- PGID=1000
volumes:
- ./config:/config
- ../tandoor/mediafiles:/media
networks:
- lsio
ports:
- 443:443
- 80:80 #optional
restart: unless-stopped
For what it's worth, I managed to get this working by just commenting out this section:
# serve media files
location /media/ {
alias /opt/recipes/mediafiles/;
}
I have serve gunicorn media set to 0 and I'm hosting on unraid with the lsio swag container.
My nginx config looks like this:
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name cook.*;
include /config/nginx/ssl.conf;
client_max_body_size 0;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
# serve media files
#location /media/ {
# alias /opt/recipes/mediafiles/;
#}
location / {
include /config/nginx/proxy.conf;
include /config/nginx/resolver.conf;
set $upstream_app TandoorRecipes;
set $upstream_port 8080;
set $upstream_proto http;
proxy_pass $upstream_proto://$upstream_app:$upstream_port;
}
}
Issue
Hi. I just got Tandoor set up with Linuxserver/SWAG
Images are uploading fine, I get success messages. The system page reports that media files are serving up fine and I don't get any errors in
docker-compose logs
, but my images are not accessible, they 404.The proxy config is default/unchanged from the linuxserver.io template.
Any ideas? Thank you
Tandoor Version
1.1.4
OS Version
Ubuntu 22.04
Setup
Docker / Docker-Compose
Reverse Proxy
SWAG
Other
No response
Environment file
Docker-Compose file
Relevant logs