search

LemmyNet / lemmy-ansible

A docker deploy for ansible
GNU Affero General Public License v3.0
252 stars 93 forks source link

Are the upgrade instructions correct? #103

Closed ross-spencer closed 1 year ago

ross-spencer commented 1 year ago

I have a new instance i setup last Sunday, then pulled yesterday and ran the upgrade instructions per the README.

The output is as follows:

nsible-playbook -i inventory/hosts lemmy.yml -kK                                                                              
SSH password:                                                                                                                                                          
BECOME password[defaults to SSH password]:                                                                                                                             

PLAY [all] ************************************************************************************************************************************************************

TASK [check lemmy_base_dir] *******************************************************************************************************************************************
skipping: [httpreserve@httpreserve.info]                                                                                                                               

TASK [install python for Ansible] *************************************************************************************************************************************
changed: [httpreserve@httpreserve.info]                                                                                                                                

TASK [setup] **********************************************************************************************************************************************************
[DEPRECATION WARNING]: Distribution ubuntu 22.04 on host httpreserve@httpreserve.info should use /usr/bin/python3, but is using /usr/bin/python for backward           
compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See                            
https://docs.ansible.com/ansible/2.10/reference_appendices/interpreter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation  
warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.                                                                                         
ok: [httpreserve@httpreserve.info]                                                                                                                                     

TASK [Install aptitude] ***********************************************************************************************************************************************
ok: [httpreserve@httpreserve.info]                                                                                                                                     

TASK [install dependencies] *******************************************************************************************************************************************
ok: [httpreserve@httpreserve.info]                                                                                                                                     

TASK [Add Docker GPG apt Key] *****************************************************************************************************************************************
ok: [httpreserve@httpreserve.info]                                                                                                                                     

TASK [Add Docker Repository] ******************************************************************************************************************************************
ok: [httpreserve@httpreserve.info]                                                                                                                                     

TASK [Update apt and install docker-ce] *******************************************************************************************************************************
ok: [httpreserve@httpreserve.info]                                                                                                                                     

TASK [Install Docker Module and docker-compose for Python] ************************************************************************************************************
ok: [httpreserve@httpreserve.info]                                                                                                                                     

TASK [copy docker config] *********************************************************************************************************************************************
ok: [httpreserve@httpreserve.info]

TASK [request initial letsencrypt certificate] ************************************************************************************************************************
ok: [httpreserve@httpreserve.info]

TASK [create lemmy folder] ********************************************************************************************************************************************
ok: [httpreserve@httpreserve.info] => (item={'path': '/srv/lemmy/digipres.cafe/', 'owner': 'root'})
ok: [httpreserve@httpreserve.info] => (item={'path': '/srv/lemmy/digipres.cafe/volumes/', 'owner': 'root'})
ok: [httpreserve@httpreserve.info] => (item={'path': '/srv/lemmy/digipres.cafe/volumes/pictrs/', 'owner': '991'})

TASK [set_fact] *******************************************************************************************************************************************************
ok: [httpreserve@httpreserve.info]

TASK [add template files] *********************************************************************************************************************************************
changed: [httpreserve@httpreserve.info] => (item={'src': 'templates/docker-compose.yml', 'dest': '/srv/lemmy/digipres.cafe/docker-compose.yml', 'mode': '0600'})
ok: [httpreserve@httpreserve.info] => (item={'src': 'templates/nginx_internal.conf', 'dest': '/srv/lemmy/digipres.cafe/nginx_internal.conf', 'mode': '0644'})
changed: [httpreserve@httpreserve.info] => (item={'src': 'templates/nginx.conf', 'dest': '/etc/nginx/sites-available/digipres.cafe.conf', 'mode': '0644'})

TASK [gather stats on site enabled config] ****************************************************************************************************************************
ok: [httpreserve@httpreserve.info]

TASK [remove if regular file (legacy) instead of symlink] *************************************************************************************************************
skipping: [httpreserve@httpreserve.info]

TASK [enable nginx site] **********************************************************************************************************************************************
ok: [httpreserve@httpreserve.info]

TASK [add the config.hjson] *******************************************************************************************************************************************
ok: [httpreserve@httpreserve.info]

TASK [add the customPostgresql.conf] **********************************************************************************************************************************
ok: [httpreserve@httpreserve.info]

TASK [enable and start docker service] ********************************************************************************************************************************
ok: [httpreserve@httpreserve.info]

TASK [start docker-compose] *******************************************************************************************************************************************

TASK [start docker-compose] *******************************************************************************************************************************************
changed: [httpreserve@httpreserve.info]

TASK [reload nginx with new config] ***********************************************************************************************************************************
changed: [httpreserve@httpreserve.info]

TASK [certbot renewal cronjob] ****************************************************************************************************************************************
ok: [httpreserve@httpreserve.info]

PLAY RECAP ************************************************************************************************************************************************************
httpreserve@httpreserve.info : ok=21   changed=4    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

But the site has been left in a bit of a mess:

image

I am wondering if I am missing a flag or there are any changes I need to make somewhere?

I think I can reinstall and then restore from a database backup but I thought I'd ask here first what may be the issue.

Connected to #100 Connected to https://github.com/LemmyNet/lemmy/issues/3307

ross-spencer commented 1 year ago

Docker status:

         Name                        Command               State                         Ports                       
---------------------------------------------------------------------------------------------------------------------
digiprescafe_lemmy-ui_1   docker-entrypoint.sh /bin/ ...   Up      1234/tcp                                          
digiprescafe_lemmy_1      /app/lemmy                       Up                                                        
digiprescafe_pictrs_1     /sbin/tini -- /usr/local/b ...   Up      6669/tcp, 8080/tcp                                
digiprescafe_postfix_1    /root/run                        Up      25/tcp                                            
digiprescafe_postgres_1   docker-entrypoint.sh postgres    Up                                                        
digiprescafe_proxy_1      /docker-entrypoint.sh ngin ...   Up      80/tcp, 0.0.0.0:17836->8536/tcp,:::17836->8536/tcp
Freeman8461 commented 1 year ago

Your site is working for me. I would clear browser caches or check from a different browser.

Screenshot from 2023-06-24 12-15-11

ross-spencer commented 1 year ago

Your site is working for me. I would clear browser caches or check from a different browser.

Yeah, that's showing for me too but inconsistently, another refresh and there's a 502, another, and then just the wireframe - plus it's not a "new site" - if you check the developer tools there's a lot of stack tracing too.

Since the upgrade basically, it's able to grab some stuff from the DB but then there's a bunch of stuff not working correctly.

ross-spencer commented 1 year ago

I am probably going to uninstall and restore from backup. For other's reference:

Docker logs when requesting the site:

proxy_1     | 172.22.0.1 - - [25/Jun/2023:06:11:35 +0000] "GET /api/v3/user/unread_count?auth=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjEsImlzcyI6ImRpZ2lwcmVzLmNh
ZmUiLCJpYXQiOjE2ODc2NzMyOTF9.kcr2ygSkcXa7DMVeGfZC5BWo1c1q4Oh3UsgQXm8YiAw HTTP/1.1" 200 47 "https://digipres.cafe/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 
(KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"                                                                                                                    
proxy_1     | 172.22.0.1 - - [25/Jun/2023:06:11:35 +0000] "GET /api/v3/user/report_count?auth=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjEsImlzcyI6ImRpZ2lwcmVzLmNh
ZmUiLCJpYXQiOjE2ODc2NzMyOTF9.kcr2ygSkcXa7DMVeGfZC5BWo1c1q4Oh3UsgQXm8YiAw HTTP/1.1" 200 66 "https://digipres.cafe/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 
(KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"                                                                                                                    
proxy_1     | 172.22.0.1 - - [25/Jun/2023:06:11:35 +0000] "GET /api/v3/ws HTTP/1.1" 404 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) C
hrome/114.0.0.0 Safari/537.36"                                                                                                                                         
proxy_1     | 172.22.0.1 - - [25/Jun/2023:06:11:35 +0000] "GET /api/v3/admin/registration_application/count?auth=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjEsImlzc
yI6ImRpZ2lwcmVzLmNhZmUiLCJpYXQiOjE2ODc2NzMyOTF9.kcr2ygSkcXa7DMVeGfZC5BWo1c1q4Oh3UsgQXm8YiAw HTTP/1.1" 200 31 "https://digipres.cafe/" "Mozilla/5.0 (X11; Linux x86_64) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"                                                                                                 
pictrs_1    | 2023-06-25T06:11:38.577470Z  INFO HTTP request{http.method=GET http.route=/image/original/{filename} http.flavor=1.1 http.scheme=http http.host=pictrs:80
80 http.client_ip=172.22.0.2 http.user_agent=node-fetch/1.0 (+https://github.com/bitinn/node-fetch) http.target=/image/original/1ace5326-4ec9-4421-998b-ac404e57c1fc.pn
g otel.name=HTTP GET /image/original/{filename} otel.kind="server" request_id=c29c11a7-0ca4-49c3-bb39-f47fbe23b747}: tracing_actix_web::root_span_builder: new         
pictrs_1    | 2023-06-25T06:11:38.587600Z  INFO HTTP request{http.method=GET http.route=/image/original/{filename} http.flavor=1.1 http.scheme=http http.host=pictrs:80
80 http.client_ip=172.22.0.2 http.user_agent=node-fetch/1.0 (+https://github.com/bitinn/node-fetch) http.target=/image/original/1ace5326-4ec9-4421-998b-ac404e57c1fc.pn
g otel.name=HTTP GET /image/original/{filename} otel.kind="server" request_id=c29c11a7-0ca4-49c3-bb39-f47fbe23b747 trace_id=00000000000000000000000000000000 exception.
message=Error in store exception.details=                                                                                                                              
pictrs_1    |    0: Error in store                                                                                                                                     
pictrs_1    |    1: Requested file is not found
pictrs_1    | 
pictrs_1    | Location:
pictrs_1    |    /drone/src/src/error.rs:38
pictrs_1    | 
pictrs_1    |   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
pictrs_1    | 
pictrs_1    |    0: pict_rs::Serving file with range=None alias=Path(Serde { inner: Alias { id: Uuid(1ace5326-4ec9-4421-998b-ac404e57c1fc), extension: Some(".png") } }
)
pictrs_1    |       at src/lib.rs:758
.host=pictrs:8080 http.client_ip=172.22.0.2 http.user_agent=node-fetch/1.0 (+https://github.com/bitinn/node-fetch) http.target=/image/original/1ace5326-4ec9-4421-998b-
ac404e57c1fc.png otel.name=HTTP GET /image/original/{filename} otel.kind="server" request_id=c29c11a7-0ca4-49c3-bb39-f47fbe23b747 trace_id=0000000000000000000000000000
0000
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-actix-web-0.7.5/src/root_span_builder.rs:41
pictrs_1    |    2: tokio::task::runtime.spawn with kind=local task.name= task.id=5349 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-
server-2.2.0/src/service.rs" loc.line=74 loc.col=17
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
pictrs_1    |    3: tokio::task::runtime.spawn with kind=local task.name= task.id=16 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-se
rver-2.2.0/src/worker.rs" loc.line=457 loc.col=25
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
pictrs_1    |    4: tokio::task::runtime.spawn with kind=local task.name= task.id=12 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-se
rver-2.2.0/src/worker.rs" loc.line=431 loc.col=21
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
pictrs_1    |    5: tokio::task::runtime.spawn with kind=local task.name= task.id=11 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-rt
-2.8.0/src/arbiter.rs" loc.line=316 loc.col=25
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
pictrs_1    |    6: tokio::task::runtime.spawn with kind=block_on task.name= task.id=10 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix
-rt-2.8.0/src/arbiter.rs" loc.line=144 loc.col=24
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
pictrs_1    | 
pictrs_1    |   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
pictrs_1    |   <empty backtrace>
pictrs_1    | 
pictrs_1    | Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering. http.status_code=500 otel.status_code="ERROR"}: tracing_actix_web::root_s
pan_builder: close time.busy=6.90ms time.idle=3.24ms
proxy_1     | 172.22.0.1 - - [25/Jun/2023:06:11:38 +0000] "GET /pictrs/image/1ace5326-4ec9-4421-998b-ac404e57c1fc.png HTTP/1.1" 500 35 "-" "node-fetch/1.0 (+https://gi
thub.com/bitinn/node-fetch)"
proxy_1     | 172.22.0.1 - - [25/Jun/2023:06:11:38 +0000] "GET / HTTP/1.1" 200 21068 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.
0"
lemmy-ui_1  | node:internal/process/promises:289
lemmy-ui_1  |             triggerUncaughtException(err, true /* fromPromise */);
lemmy-ui_1  |             ^
lemmy-ui_1  | 
lemmy-ui_1  | [Error: Input buffer contains unsupported image format]
lemmy-ui_1  | 
lemmy-ui_1  | Node.js v20.3.1
proxy_1     | 172.22.0.1 - - [25/Jun/2023:06:11:38 +0000] "GET /static/styles/styles.css HTTP/1.1" 502 150 "https://digipres.cafe/" "Mozilla/5.0 (X11; Ubuntu; Linux x8
6_64; rv:109.0) Gecko/20100101 Firefox/113.0"
proxy_1     | 2023/06/25 06:11:38 [error] 30#30: *13827 connect() failed (111: Connection refused) while connecting to upstream, client: 172.22.0.1, server: localhost,
 request: "GET /static/styles/styles.css HTTP/1.1", upstream: "http://172.22.0.2:1234/static/styles/styles.css", host: "digipres.cafe", referrer: "https://digipres.caf
e/"
proxy_1     | 2023/06/25 06:11:38 [error] 30#30: *13831 connect() failed (111: Connection refused) while connecting to upstream, client: 172.22.0.1, server: localhost,
 request: "GET /static/js/client.js HTTP/1.1", upstream: "http://172.22.0.2:1234/static/js/client.js", host: "digipres.cafe", referrer: "https://digipres.cafe/"
proxy_1     | 2023/06/25 06:11:38 [error] 30#30: *13830 connect() failed (111: Connection refused) while connecting to upstream, client: 172.22.0.1, server: localhost,
 request: "GET /css/themes/darkly.css HTTP/1.1", upstream: "http://172.22.0.2:1234/css/themes/darkly.css", host: "digipres.cafe", referrer: "https://digipres.cafe/"
proxy_1     | 2023/06/25 06:11:38 [error] 30#30: *13829 connect() failed (111: Connection refused) while connecting to upstream, client: 172.22.0.1, server: localhost,
 request: "GET /static/assets/symbols.svg HTTP/1.1", upstream: "http://172.22.0.2:1234/static/assets/symbols.svg", host: "digipres.cafe", referrer: "https://digipres.c
afe/"
proxy_1     | 172.22.0.1 - - [25/Jun/2023:06:11:38 +0000] "GET /static/js/client.js HTTP/1.1" 502 150 "https://digipres.cafe/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64;
 rv:109.0) Gecko/20100101 Firefox/113.0"
proxy_1     | 172.22.0.1 - - [25/Jun/2023:06:11:38 +0000] "GET /css/themes/darkly.css HTTP/1.1" 502 150 "https://digipres.cafe/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_6
4; rv:109.0) Gecko/20100101 Firefox/113.0"
proxy_1     | 172.22.0.1 - - [25/Jun/2023:06:11:38 +0000] "GET /static/assets/symbols.svg HTTP/1.1" 502 150 "https://digipres.cafe/" "Mozilla/5.0 (X11; Ubuntu; Linux x
86_64; rv:109.0) Gecko/20100101 Firefox/113.0"
proxy_1     | 172.22.0.1 - - [25/Jun/2023:06:11:38 +0000] "GET /pictrs/image/1ace5326-4ec9-4421-998b-ac404e57c1fc.png?format=webp&thumbnail=96 HTTP/1.1" 500 35 "https:
//digipres.cafe/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0"
pictrs_1    | 2023-06-25T06:11:38.869136Z  INFO HTTP request{http.method=GET http.route=/image/process.{ext} http.flavor=1.1 http.scheme=http http.host=pictrs:8080 htt
p.client_ip=80.187.64.60 http.user_agent=Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0 http.target=/image/process.webp?src=1ace5326-4e
c9-4421-998b-ac404e57c1fc.png&thumbnail=96 otel.name=HTTP GET /image/process.{ext} otel.kind="server" request_id=ce6d6c1d-b93b-4bb0-b451-f44c19a40c9f}: tracing_actix_w
eb::root_span_builder: new
pictrs_1    | 2023-06-25T06:11:38.871051Z  INFO HTTP request{http.method=GET http.route=/image/process.{ext} http.flavor=1.1 http.scheme=http http.host=pictrs:8080 htt
p.client_ip=80.187.64.60 http.user_agent=Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0 http.target=/image/process.webp?src=1ace5326-4e
c9-4421-998b-ac404e57c1fc.png&thumbnail=96 otel.name=HTTP GET /image/process.{ext} otel.kind="server" request_id=ce6d6c1d-b93b-4bb0-b451-f44c19a40c9f trace_id=00000000
000000000000000000000000 exception.message=Error in store exception.details=
pictrs_1    |    0: Error in store
pictrs_1    |    1: Requested file is not found
pictrs_1    | 
pictrs_1    | Location:
pictrs_1    |    /drone/src/src/error.rs:38
pictrs_1    | 
pictrs_1    |   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
pictrs_1    | 
pictrs_1    |    0: pict_rs::Serving processed image with range=None query=Query([("src", "1ace5326-4ec9-4421-998b-ac404e57c1fc.png"), ("thumbnail", "96")]) ext=Path("
webp")
pictrs_1    |       at src/lib.rs:590

pictrs:8080 http.client_ip=80.187.64.60 http.user_agent=Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0 http.target=/image/process.webp?
src=1ace5326-4ec9-4421-998b-ac404e57c1fc.png&thumbnail=96 otel.name=HTTP GET /image/process.{ext} otel.kind="server" request_id=ce6d6c1d-b93b-4bb0-b451-f44c19a40c9f tr
ace_id=00000000000000000000000000000000
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-actix-web-0.7.5/src/root_span_builder.rs:41
pictrs_1    |    2: tokio::task::runtime.spawn with kind=local task.name= task.id=5356 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-
server-2.2.0/src/service.rs" loc.line=74 loc.col=17
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
pictrs_1    |    3: tokio::task::runtime.spawn with kind=local task.name= task.id=16 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-se
rver-2.2.0/src/worker.rs" loc.line=457 loc.col=25
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
pictrs_1    |    4: tokio::task::runtime.spawn with kind=local task.name= task.id=12 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-se
rver-2.2.0/src/worker.rs" loc.line=431 loc.col=21
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
pictrs_1    |    5: tokio::task::runtime.spawn with kind=local task.name= task.id=11 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-rt
-2.8.0/src/arbiter.rs" loc.line=316 loc.col=25
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
pictrs_1    |    6: tokio::task::runtime.spawn with kind=block_on task.name= task.id=10 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix
-rt-2.8.0/src/arbiter.rs" loc.line=144 loc.col=24
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
pictrs_1    | 
pictrs_1    |   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
pictrs_1    |   <empty backtrace>
pictrs_1    | 
pictrs_1    | Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering. http.status_code=500 otel.status_code="ERROR"}: tracing_actix_web::root_s

pictrs_1    | Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering. http.status_code=500 otel.status_code="ERROR"}: tracing_actix_web::root_s
pan_builder: close time.busy=1.86ms time.idle=56.3µs
proxy_1     | 172.22.0.1 - - [25/Jun/2023:06:11:38 +0000] "GET /pictrs/image/1ace5326-4ec9-4421-998b-ac404e57c1fc.png HTTP/1.1" 500 35 "https://digipres.cafe/" "Mozill
a/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0"
pictrs_1    | 2023-06-25T06:11:38.956923Z  INFO HTTP request{http.method=GET http.route=/image/original/{filename} http.flavor=1.1 http.scheme=http http.host=pictrs:80
80 http.client_ip=80.187.64.60 http.user_agent=Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0 http.target=/image/original/1ace5326-4ec9
-4421-998b-ac404e57c1fc.png otel.name=HTTP GET /image/original/{filename} otel.kind="server" request_id=268831d4-c285-444f-9d5e-f0141eccd7c7}: tracing_actix_web::root_
span_builder: new
pictrs_1    | 2023-06-25T06:11:38.958761Z  INFO HTTP request{http.method=GET http.route=/image/original/{filename} http.flavor=1.1 http.scheme=http http.host=pictrs:80
80 http.client_ip=80.187.64.60 http.user_agent=Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0 http.target=/image/original/1ace5326-4ec9
-4421-998b-ac404e57c1fc.png otel.name=HTTP GET /image/original/{filename} otel.kind="server" request_id=268831d4-c285-444f-9d5e-f0141eccd7c7 trace_id=00000000000000000
000000000000000 exception.message=Error in store exception.details=
pictrs_1    |    0: Error in store
pictrs_1    |    1: Requested file is not found
pictrs_1    | 
pictrs_1    | Location:
pictrs_1    |    /drone/src/src/error.rs:38
pictrs_1    | 
pictrs_1    |   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
pictrs_1    | 
pictrs_1    |    0: pict_rs::Serving file with range=None alias=Path(Serde { inner: Alias { id: Uuid(1ace5326-4ec9-4421-998b-ac404e57c1fc), extension: Some(".png") } }
)
pictrs_1    |       at src/lib.rs:758
pictrs_1    |    1: tracing_actix_web::root_span_builder::HTTP request with http.method=GET http.route=/image/original/{filename} http.flavor=1.1 http.scheme=http http
.host=pictrs:8080 http.client_ip=80.187.64.60 http.user_agent=Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0 http.target=/image/origina
l/1ace5326-4ec9-4421-998b-ac404e57c1fc.png otel.name=HTTP GET /image/original/{filename} otel.kind="server" request_id=268831d4-c285-444f-9d5e-f0141eccd7c7 trace_id=00
000000000000000000000000000000
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-actix-web-0.7.5/src/root_span_builder.rs:41
pictrs_1    |    2: tokio::task::runtime.spawn with kind=local task.name= task.id=5361 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-
server-2.2.0/src/service.rs" loc.line=74 loc.col=17
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
pictrs_1    |    3: tokio::task::runtime.spawn with kind=local task.name= task.id=16 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-se
rver-2.2.0/src/worker.rs" loc.line=457 loc.col=25
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
pictrs_1    |    4: tokio::task::runtime.spawn with kind=local task.name= task.id=12 loc.file="/opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/actix-se
rver-2.2.0/src/worker.rs" loc.line=431 loc.col=21
pictrs_1    |       at /opt/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.28.2/src/util/trace.rs:16
[2] 0:bash- 1:[tmux]* 2:bash                                                                                                         "exponentialdecay" 08:1

Console log on requesting the site:

digipres.cafe/:34     GET https://digipres.cafe/static/styles/styles.css 502
digipres.cafe/:37     GET https://digipres.cafe/css/themes/darkly.css net::ERR_ABORTED 502
digipres.cafe/:42 Refused to load the font 'data:font/truetype;charset=utf-8;base64,d09GRgABAAAAALY3ABIAAAABztAAAAAAAAC1QAAAAPcAAAHiAAAAAAAAAABHUE9TAACV9AAAFxIAAHKy/pn970dTVUIAAK0IAAAINQAAFD6g2KReTFRTSAAABtQAAAA6AAACQeO3nq5PUy8yAAACDAAAAFMAAABgZoZye2NtYXAAABxsAAADdwAABTpa8HPyY3Z0IAAAIbAAAAAoAAAAKAhGAbdmcGdtAAAf5AAAAQUAAAFzBpmcN2dhc3AAAJXoAAAADAAAAAwABwAHZ2x5ZgAAJkQAAGT0AADhaCjCChFoZG14AAAHEAAAFVwAADPIhU9AOGhlYWQAAAGUAAAANQAAADYFph12aGhlYQAAAcwAAAAgAAAAJAc2BPtobXR4AAACYAAABHEAAAj0jIVtDGxvY2EAACHYAAAEbAAABHyCc7p8bWF4cAAAAewAAAAgAAAAIARXAjxuYW1lAACLOAA...SCThVMYJIqZCqSu4rDezyzyKpY17tqhUyLsX30L+SQltAyggVZVqVMsLfPk1CNKvWoXtPreoMd2Ymd2YVd2S2+tbRTX92te3Sv7tP9egCjlnZ0oid96c/8LMkyrMgarMP67MDvNkYxV0uGjvOJFo+33umRlTnS48NR0ehv5tf6D/25M7IAAAB4AYVRRXfDMAz+K3q+jBc4DVyPmfGu56iNk9bKc9QO/ny5Y7iJP5DukGCGgtCjUDv2DZWsxsroHvmMA3jsUEM9TnoQx8NmHEMax+sKuqHdULlItRFFljsdCtZhW14qWh2mCiKj286Srwlc9vMROJ8M/Hcsak/njBZ6FrPHXtAKON/k7e/j0OQAb9h+mHVQhtg6Gm/r2TmjLVcvwbVymR6e31sY89p7uwn3o6Nvmx8XAmbUwVBOd48CVrkrwdWA8NYEbn69Ft3Z/AmDvFqGE2/fj78tvTlH0w9MfcpAGK7ZliRjjd2agD20SKpx8c34aPZXMwDpj6h5' because it violates the following Content Security Policy directive: "default-src 'self'". Note that 'font-src' was not explicitly set, so 'default-src' is used as a fallback.

digipres.cafe/:42 Refused to load the font 'data:font/truetype;charset=utf-8;base64, d09GRgABAAAAALM7AA8AAAABq0AAAAAAAACyRAAAAPcAAAHiAAAAAAAAAABHUE9TAACIDAAAIgQAAH82Ol56cUdTVUIAAKoQAAAIMwAAFD6gNKPBT1MvMgAAAdAAAABVAAAAYGbqc4pjbWFwAAAGbAAAA3wAAAU4Vch3gWN2dCAAAAwYAAAAQAAAAEAQPwNiZnBnbQAACegAAAEDAAABcwZZnDdnbHlmAAAQvAAAbJ4AAOZUwSn/MmhlYWQAAAFYAAAANgAAADYGApswaGhlYQAAAZAAAAAgAAAAJAdbBRtobXR4AAACKAAABEIAAAjwrN1ggGxvY2EAAAxYAAAEYgAABHqHyk9obWF4cAAAAbAAAAAgAAAAIARWA2ZuYW1lAAB9XAAAAoEAAAZuOd88j3Bvc3QAAH/gAAAIKgAAEi5ynk2NcHJlcAAACuwAAAEqAAACpAoaNTMAAQAAAAEAAA...STdLpgIn7fZCqSu5rDezyzyKpZz7vqhExLsEP0L+KQltJygoVYTqVM+PxDJaFaVekJvaE39RY7sTO7sCu7sbtQL3VQf92n+/WAHtRDehiopT2d6UU/BrAAS7EsK7Em67IBO/KnjVXM1dKhE3yixfOtb3pmZY70/HBUmjxmflv/BTgtM3AAeAGFUUV3wzAM/it6vowXOA1cj5nxrueojZPWynPUDv58uWO4iT+Q7pBghoLQo1A79g2VrMbK6B75jAN47FBDPU56EMfDZhxDGsfrCrqh3VC5SLURRZY7HQrWYVteKlodpgoio9vOkq8JXPbzETifDPx3LGpP54wWehazx17QCjjf5O3v49DkAG/Yfph1UIbYOhpv69k5oy1XL8G1cpkent9bGPPae7sJ96Ojb5sfFwJm1MFQTnePAla5K8HVgPDWBG5+vRbd2fwJg7xahhNv34+/Lb05R9MPTH3KQBiu2ZYkY43dmoA9tEiqcfHN+Gj2VzMA6Y+oeQ==' because it violates the following Content Security Policy directive: "default-src 'self'". Note that 'font-src' was not explicitly set, so 'default-src' is used as a fallback.

digipres.cafe/:42 Refused to load the font 'data:font/truetype;charset=utf-8;base64, d09GRgABAAAAAMjjABIAAAAB2mAAAAAAAADH7AAAAPcAAAHiAAAAAAAAAABHUE9TAACdfAAAIjcAAH80VqR1REdTVUIAAL+0AAAINwAAFD6g7KTPTFRTSAAABwQAAABHAAACQYV/Ri1PUy8yAAACDAAAAFQAAABgZ050kmNtYXAAAB1cAAAChQAAA/wdE0d/Y3Z0IAAAIdQAAAA0AAAANAq+BC1mcGdtAAAf5AAAAQUAAAFzBpmcN2dhc3AAAJ1wAAAADAAAAAwABwAHZ2x5ZgAAJnAAAGxKAADg/GpVnLBoZG14AAAHTAAAFhAAADPIAPyiAmhlYWQAAAGUAAAANQAAADYF/aZQaGhlYQAAAcwAAAAgAAAAJAdxBTBobXR4AAACYAAABKIAAAj0wQRTzWxvY2EAACIIAAAEZwAABHxHMIAQbWF4cAAAAewAAAAgAAAAIARXAs1uYW1lAACSvA...2hawUjGKWSTHlyN3B4j2cWs3I2964qIdNK7Bn7l3NIq2gNwdKsrkImONznSahSZTpEh+owvcW+7Mf+HMCBHCTUSS3UXS/oRb2kl/WKXkVU0IxWdKQr3VmSlVmNtdmIzdiSffjdhinO1apRR/hEi8+3rumZFRzp+eEo1XnM/Kz/ABXXNDcAeAGFUUV3wzAM/it6vowXOA1cj5nxrueojZPWynPUDv58uWO4iT+Q7pBghoLQo1A79g2VrMbK6B75jAN47FBDPU56EMfDZhxDGsfrCrqh3VC5SLURRZY7HQrWYVteKlodpgoio9vOkq8JXPbzETifDPx3LGpP54wWehazx17QCjjf5O3v49DkAG/Yfph1UIbYOhpv69k5oy1XL8G1cpkent9bGPPae7sJ96Ojb5sfFwJm1MFQTnePAla5K8HVgPDWBG5+vRbd2fwJg7xahhNv34+/Lb05R9MPTH3KQBiu2ZYkY43dmoA9tEiqcfHN+Gj2VzMA6Y+oeQ==' because it violates the following Content Security Policy directive: "default-src 'self'". Note that 'font-src' was not explicitly set, so 'default-src' is used as a fallback.

digipres.cafe/:61     GET https://digipres.cafe/pictrs/image/1ace5326-4ec9-4421-998b-ac404e57c1fc.png?format=webp&thumbnail=96 500
digipres.cafe/:58     GET https://digipres.cafe/static/js/client.js net::ERR_ABORTED 502
digipres.cafe/:48     GET https://digipres.cafe/static/assets/symbols.svg net::ERR_ABORTED 502
manifest.webmanifest:1     GET https://digipres.cafe/manifest.webmanifest 502
manifest.webmanifest:1 Manifest: Line: 1, column: 1, Syntax error.
1ace5326-4ec9-4421-998b-ac404e57c1fc.png:1     GET https://digipres.cafe/pictrs/image/1ace5326-4ec9-4421-998b-ac404e57c1fc.png 500

Volumes folder:

tree volumes/
volumes/
├── lemmy-ui
│   └── extra_themes
├── pictrs
│   ├── files
│   │   ├── 000
│   │   │   └── 2b0a7228-7369-43d8-b8d2-749876a5fe3d.png
│   │   └── 001
│   │       ├── 000
│   │       │   └── a3b5aaf7-60b7-4fa6-81b5-128d31e27ac6.png
│   │       ├── 002
│   │       │   └── a3b5aaf7-60b7-4fa6-81b5-128d31e27ac6.png
│   │       ├── 003
│   │       │   └── a3b5aaf7-60b7-4fa6-81b5-128d31e27ac6.png
│   │       ├── 004
│   │       │   └── a3b5aaf7-60b7-4fa6-81b5-128d31e27ac6.png
│   │       ├── 005
│   │       │   └── b5345e42-a104-41fc-ad57-ab8d8bd8f5dd.jpeg
│   │       ├── 007
│   │       │   └── b5345e42-a104-41fc-ad57-ab8d8bd8f5dd.jpeg
│   │       └── 008
│   │           └── b5345e42-a104-41fc-ad57-ab8d8bd8f5dd.jpeg
│   ├── sled
│   │   └── db-0.34
│   │       ├── blobs
│   │       ├── conf
│   │       ├── db
│   │       └── snap.0000000000001982
│   └── sled-repo
│       └── v0.4.0-alpha.1
│           ├── blobs
│           ├── conf
│           ├── db
│           └── snap.0000000000001DD1
└── postgres

I'll close this issue as it's a bit scatter-gun and unspecific. Logs included if they help others find something in the future.