search

boulder-food-rescue / food-rescue-robot

A Rails App for Managing "Just in Time" Food Rescue, Developed by/for Boulder Food Rescue in Boulder, CO, USA
49 stars 56 forks source link

Data Export Times Out on Production #137

Open rylanb opened 7 years ago

rylanb commented 7 years ago

Expected Behavior

Data Export at https://robot.boulderfoodrescue.org/logs/export.csv?utf8=%E2%9C%93&start_date%5Bmonth%5D=8&start_date%5Bday%5D=1&start_date%5Byear%5D=2017&stop_date%5Bmonth%5D=9&stop_date%5Bday%5D=27&stop_date%5Byear%5D=2017&commit=Export

does NOT time-out.

Actual Behavior

It times out!

Steps to Recreate

Log in as a super admin or a region admin and select a large data range and try to export. It will work fine locally, but on web server Heroku will kill it after 30 seconds.

Look for N+1 queries and ways to clean up / generate more cleanly.

Dania02525 commented 7 years ago

Considered moving to async exports and storing them on s3? I've got some classes that can be used to upload exports into an s3 bucket with object expiry, then listing the recent exports scoped to user or anything really for download

Edit: I see that sidekiq is not currently a dependency. I'll create a PR to address the sql injection issues in the log controller + speed up the queries, then on top of that one, one to bring in sidekiq + s3 async reporting so we can weigh whether it will be worth doing.