Move org storage recalculation into background job

webrecorder / browsertrix

Browsertrix is the hosted, high-fidelity, browser-based crawling service from Webrecorder designed to make web archiving easier and more accessible for all!

https://webrecorder.net/browsertrix

GNU Affero General Public License v3.0

201 stars 35 forks source link

Move org storage recalculation into background job #2138

Closed tw4l closed 2 days ago

tw4l commented 1 week ago

Fixes #2112

Moves org storage recalculation to background job, modify endpoint to return job id as part of response
Updates crawl + QA backend tests that broke due to https://webrecorder.net website changes

Not moving org import over to a background job yet as we hadn't firmly decided to do that and it seems likely to take more time to figure out mounting the JSON to import into the background job container than I wanted to spend on this for this sprint.