Problems with image caching (IMAGE_PROXY_CACHE_EXTERNAL)

[cowsay1]

I tried to use image caching and it doesn't work for me

- IMAGE_PROXY_CACHE_EXTERNAL=true
- IMAGE_PROXY_HTTP2HTTPS=true

Remark logs:

remark42  | 2023/04/28 01:14:07.922 [WARN]  {rest/httperrors.go:81 rest.SendErrorJSON} can't load picture from the comment - can't get image file for cached_images/e38d5bf8e071f20c8e6bb07ba5925359c84506be-c65ec743cf62a946d2c1ed2f64350d6961024e84: can't get image stats for cached_images/e38d5bf8e071f20c8e6bb07ba5925359c84506be-c65ec743cf62a946d2c1ed2f64350d6961024e84: stat var/pictures.staging/cached_images/84/e38d5bf8e071f20c8e6bb07ba5925359c84506be-c65ec743cf62a946d2c1ed2f64350d6961024e84: no such file or directory - 400 (20) - myusername/github_myhash - /api/v1/comment?site=mysite - [rest/api/rest_private.go:136 api.(*private).createCommentCtrl]

remark42  | 2023/04/28 01:14:07.922 [INFO]  {logger/logger.go:134 logger.(*Middleware).Handler.func1.1} POST - /api/v1/comment?site=mysite - remark42.mysite.com - 0c8d1dc9c652 - 400 (467) - 358.164µs - {"title":"Comments","text":"![Image of my cat](https://placekitten.com/400/400)","locator":{"site":"mysite","url":"https://mysite.com/mypage.html"}}

Happened with any other images (not just https://placekitten.com/400/400), images from main domain e.g https://mysite.com/1.png, both HTTP (without redirect) and HTTP(s).

My setup is docker with nginx reverse proxy (nginx outside compose), umputun/remark42:latest, also tried building from master (logs from master). All remote features, such as OAuth and Telegram notify, function properly, and normal image upload work too.

Remark didn't even create var/cached_images and var/pictures.staging/cached_images dirs. I tried to create manually with chmod/chown with 1001 (same as other dirs in var/), no luck.

Also it happned both in dev machine, and remote cloud, disable firewalls didn't help.

http logs

``` POST /api/v1/comment?site=mysite HTTP/2 Host: remark42.mysite.com User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:112.0) Gecko/20100101 Firefox/112.0 Accept: */* Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate, br Referer: https://remark42.mysite.com/web/iframe.html?host=https%3A%2F%2Fremark42.mysite.com&site_id=mysite&show_rss_subscription=false&url=https%3A%2F%2Fmysite.com%2Fmypage.html X-XSRF-TOKEN: ... Content-Type: application/json Content-Length: 155 Origin: https://remark42.mysite.com DNT: 1 Connection: keep-alive Cookie: JWT=...; XSRF-TOKEN=... Sec-Fetch-Dest: empty Sec-Fetch-Mode: cors Sec-Fetch-Site: same-origin Pragma: no-cache Cache-Control: no-cache TE: trailers {"title":"Comments","text":"","locator":{"site":"mysite","url":"https://mysite.com/mypage.html"}} HTTP/2 400 Bad Request server: nginx date: Fri, 28 Apr 2023 07:25:11 GMT content-type: application/json; charset=utf-8 content-length: 467 access-control-allow-credentials: true access-control-allow-origin: * access-control-expose-headers: Authorization app-name: remark42 app-version: local-20230427T21:44:23 author: umputun cache-control: no-cache, no-store, no-transform, must-revalidate, private, max-age=0 expires: Wed, 31 Dec 1969 18:00:00 CST pragma: no-cache ratelimit-limit: 1 ratelimit-remaining: 0 ratelimit-reset: 1 vary: Origin x-rate-limit-duration: 1 x-rate-limit-limit: 0.50 x-rate-limit-request-forwarded-for: 127.0.0.1 x-rate-limit-request-remote-addr: 127.0.0.1 X-Firefox-Spdy: h2 {"code":20,"details":"can't load picture from the comment","error":"can't get image file for cached_images/e38d5bf8e071f20c8e6bb07ba5925359c84506be-c65ec743cf62a946d2c1ed2f64350d6961024e84: can't get image stats for cached_images/e38d5bf8e071f20c8e6bb07ba5925359c84506be-c65ec743cf62a946d2c1ed2f64350d6961024e84: stat var/pictures.staging/cached_images/84/e38d5bf8e071f20c8e6bb07ba5925359c84506be-c65ec743cf62a946d2c1ed2f64350d6961024e84: no such file or directory"} ```

docker-compose (cleanup it to almost defaults, doenst help)

``` version: "2" services: remark: build: . image: myhub/etc:remark42-master container_name: "remark42" hostname: "remark42" restart: always logging: driver: json-file options: max-size: "10m" max-file: "5" ports: - "127.0.0.1:8080:8080" environment: - REMARK_URL=https://remark42.mysite.com - SITE=mysite - SECRET= - ADMIN_SHARED_ID= - ADMIN_SHARED_EMAIL= - DEBUG=true - IMAGE_PROXY_CACHE_EXTERNAL=true - IMAGE_PROXY_HTTP2HTTPS=true - AUTH_ANON=true - AUTH_GITHUB_CID= - AUTH_GITHUB_CSEC= volumes: - ./var:/srv/var ```

[umputun]

@paskal - as far as I remember you were the last one dealing with images. Could you please take a look?

[paskal]

I haven't had time to look into it yet, but it's on my list now.

[paskal]

When I was investigating this, I saw a misleading error message. If it's not possible to save the image to the file cache, ResetCleanupTimer throws an error and not the image saving:

remark42-dev  | 2023/07/16 12:00:37.093 [WARN]  {rest/httperrors.go:81 rest.SendErrorJSON} can't renew staged picture cleanup timer - can't get image stats for cached_images/e38d5bf8e071f20c8e6bb07ba5925359c84506be-ad90132fb4e9d5524f84c086ccc71a4f0dd703e4: stat var/pictures.staging/cached_images/93/e38d5bf8e071f20c8e6bb07ba5925359c84506be-ad90132fb4e9d5524f84c086ccc71a4f0dd703e4: no such file or directory - 400 (20) - dev_user/dev_user - /api/v1/preview?site=remark - [rest/api/rest_private.go:97 api.(*private).previewCommentCtrl]

I'll try to address this as well.

[paskal]

The problem was introduced in fe716b0a717a732168e2e78fb97fa98fd3cb3a21. I'll fix it.

[paskal]

The problem affected versions 1.9.0-1.11.3. The fix will be released as 1.12.0.