Is this Possible; "Remote Imgproxy and Remote Image files" - If yes, How?

[akhil-naidu]

Inspiration

After using the Imgproxy Heroku button, I used this link to confirm whether my images in s3 are resizing properly or not. And that worked excellently as mentioned in this Evilmartians blog post in a couple of minutes. So, I started thinking to eliminate services like Cloudinary addons and ImageMagick in my production environment and replace it with Imgproxy.

My present Production configuration

Let me introduce how my production environment work so that it might give you some idea to advise me to move forward. I use two Digital Ocean droplets(Let's call them dB and App), One for databases(Redis and Postgress), and the other is to run my Ruby on Rails workers(Web and Sidekiq). The databases are managed through Caprover and Rails app using Dokku, while the images are uploaded in s3.

What I was planning to do

As Imgproxy has containerisation and Heroku support, I can install Imgproxy in any of these droplets(dB or App). As my Sidekiq worker is already consuming an awful amount of RAM, even after configuring Jemalloc, I decided not to install this in App droplet and install Imgproxy in dB droplet(Which is a remote server to my App droplet).

A small test in Heroku

Before starting to implement this in my production, I thought to run some tests in Heroku. I used the Heroku button to create my Imgproxy instance. Configured the ENV variables (IMGPROXY_SALT and IMGPROXY_KEY). Tested my setup using this in browser and with this command curl -I https://imgproxyleedev.herokuapp.com/unsafe/aHR0cDovL2xvY2Fs/aG9zdDozMDAwL2Fz/c2V0cy8xLnBuZw in my Production(App Droplet). Till now everything went quite well.

Problem I was facing

I have two Heroku instances, within which one is hosting Imgproxy, and the other is my development instance. As I want to configure a remote Imgproxy with Remote images, I read almost half of the documentation to understand how Imgproxy works well in my case. Then I came across these environment variables to make this possible. I configured all these variables in both of my Heroku Instances(Imgproxy and Development), as I'm not sure, "how to pinpointedly distribute those ENV variables among these two Heroku instances"(I tried but failed).

Here are my environment variables; $ indicates as placeholders and if no $, I'm using the exact same value.

ENV Variable	Value
IMGPROXY_USE_S3	true
IMGPROXY_S3_REGION	us-east-1
IMGPROXY_S3_ENDPOINT	leewardslope-us.s3.amazonaws.com
AWS_ACCESS_KEY_ID	$(my ID)
AWS_SECRET_ACCESS_KEY	$(my KEY)
IMGPROXY_KEY	$(generated string)
IMGPROXY_SALT	$(generated string)
IMGPROXY_ENDPOINT	imgproxyleedev.herokuapp.com

Now, I restarted and redeployed both my Heroku instances(making sure it is not the issue), but I'm facing few problems.

• My development app now have different image URLs but can't load images. I even tried creating new articles and reupload profile pictures; It's not working too!

I need help

If anyone out there who can resolve this issue or guide me through configuration, please let me know! I'm eagerly waiting to use this tool—thank you in advance.

[DarthSim]

Hey Akhil!

Thanks for the detailed description, but it misses the most important part: what exactly goes wrong. Try to take a look at your imgproxy logs, it explains the situation in most cases.

[akhil-naidu]

Ok, I might have missed pointing it out. The issue was I was not able to connect my Heroku Development App with Heroku Imgproxy. I was not getting any logs in Heroku Imgproxy.

But the movement I added the ENV variables in my Heroku Development App, the images are not loading. Here is a log of reloading just one image file.

2021-06-14T12:35:27.783706+00:00 app[web.1]: 2021-06-14 12:35:27.783 UTC [30] LOG stats: 0 xacts/s, 0 queries/s, in 156 B/s, out 417 B/s, xact 4038 us, query 4038 us, wait 37 us
2021-06-14T12:35:32.487628+00:00 app[web.1]: F, [2021-06-14T12:35:32.487462 #34] FATAL -- : [4beaee19-f77d-41e8-b301-6f39dc0b71fe]   
2021-06-14T12:35:32.487640+00:00 app[web.1]: [4beaee19-f77d-41e8-b301-6f39dc0b71fe] ActionController::RoutingError (No route matches [GET] "/images/-UjH7YZtQMhi-0eLMzSMD-GDwAw-C7uY7aT8IDOhYG8/s:1000:420/mb:500000/aHR0cHM6Ly9sZWV3/YXJkc2xvcGUtdXMu/czMuYW1hem9uYXdz/LmNvbS91cGxvYWRz/L2FydGljbGVzL21x/bHlyc2ljZGYzdmtm/cXpvYTNpLmpwZw"):
2021-06-14T12:35:32.487641+00:00 app[web.1]: [4beaee19-f77d-41e8-b301-6f39dc0b71fe]

What I thought was it might be somehow related to the ENV variable, IMGPROXY_ALLOW_ORIGIN; but I'm not sure it.

[DarthSim]

Well, I see ActionController::RoutingError in the logs and it doesn't look like imgproxy. It seems like you're trying to request the image from your Rails app.

[akhil-naidu]

I'm using the images as a cover image in my rails app. Is that an issue?

[akhil-naidu]

Actually, can you answer me? I might give more information.

If I install Imgproxy in the local server and configure ENV variables, everything will work fine. But in this case, the Heroku Imgproxy is not a local service, so I need to connect these two instances. Till now, I was able to find only one ENV variable for such connections(IMGPROXY_ENDPOINT); Are there any other ENV variables that I need to configure in such cases?

[DarthSim]

I mean, it looks like you use image URLs that lead to your Rails app instead of imgproxy:

<img src="https://leewardslope-dev.herokuapp.com/images/OH870JTVqo-1NlFwaRGa7QYAw1wqBKvrGzmxSNSxpug/rs:fill:90:90/mb:500000/aHR0cHM6Ly9sZWV3/YXJkc2xvcGUtdXMu/czMuYW1hem9uYXdz/LmNvbS91cGxvYWRz/L3VzZXIvcHJvZmls/ZV9pbWFnZS8xLzQ3/MTA4MDk0LTUyNDEt/NDM0OS04OWFiLTli/MmUxMzc5NDM0Zi5q/cGc" alt="morty profile" class="crayons-avatar__image" loading="lazy">

[DarthSim]

You don't need to "connect" instances. Your Rails app should just generate an imgproxy URLs for images.

[akhil-naidu]

I mean, it looks like you use image URLs that lead to your Rails app instead of imgproxy:

This makes a lot of sense, is there a way for me to resolve this. Actually, I'm using an opensource rails app, If you can point out a default location, I will try to locate and change it; if that is the case.

[akhil-naidu]

You don't need to "connect" instances. Your Rails app should just generate an imgproxy URLs for images.

Is there any documentation for me to follow, in this regard?

[akhil-naidu]

What do you think about this?

https://github.com/imgproxy/imgproxy.rb#amazon-s3

I'm also attaching my imgproxy.rb file, this might help you enlighten me, I think so.

[DarthSim]

It is definitely a problem with your Rails app, not with imgproxy, as it generates the wrong URLs or uses them incorrectly. As far as I see, you use Forem, and if you didn't change anything in its code, I think it'd be better for you to ask its maintainers for support. I'm sorry but debugging Rails apps is pretty out of the scope of imgproxy support.

[akhil-naidu]

Sure! But can you have a peek at my imgproxy.rb file, it was attached in my previous comment. If you were not able to suggest anything with it. I will try contacting the Forem team.

[DarthSim]

It is just an unchanged file from Forem. I don't see anything suspicious in its code.