DemocracyClub / yournextrepresentative

👥 A website for crowd-sourcing structured election candidate data
https://candidates.democracyclub.org.uk
GNU Affero General Public License v3.0
21 stars 27 forks source link

Reduce time and errors on photo uploads #2296

Closed VirginiaDooley closed 3 months ago

VirginiaDooley commented 4 months ago

Ref https://app.asana.com/0/1204880927741389/1207101641238151/f These changes are aimed at speeding up the photo upload process.

symroe commented 4 months ago

I don't think this is going to solve anything, and in fact I think it will case more problems.

We can upload images larger than 1mb, and there are many uploads that are larger than this in the system (most, in fact, so I think this basically disables uploads).

The problem is somewhere around converting the image to png or rotating it. It might be a simple timeout in gunicorn. I don't know the details, but the solution will look like doing that work outside of the main upload process I expect.

symroe commented 4 months ago

TBH I don't think this is important enough to work on today. We can ask people to email us if something breaks, I guess?

VirginiaDooley commented 4 months ago

I don't think this is going to solve anything, and in fact I think it will case more problems.

We can upload images larger than 1mb, and there are many uploads that are larger than this in the system (most, in fact, so I think this basically disables uploads).

The problem is somewhere around converting the image to png or rotating it. It might be a simple timeout in gunicorn. I don't know the details, but the solution will look like doing that work outside of the main upload process I expect.

Ok, understood about 1MB, but we aren't limiting upload size at all. I'll look further into the

I don't think this is going to solve anything, and in fact I think it will case more problems.

We can upload images larger than 1mb, and there are many uploads that are larger than this in the system (most, in fact, so I think this basically disables uploads).

The problem is somewhere around converting the image to png or rotating it. It might be a simple timeout in gunicorn. I don't know the details, but the solution will look like doing that work outside of the main upload process I expect.

At the moment, we aren't capping the photo size for uploads at all, so even if just for storage costs, maybe we want to add a max that seems reasonable.

After further testing, you're right - the rotate_image step is taking way too long, even when it's not needed. I see if there is a way to speed it up when rotation isn't needed.