webrecorder / browsertrix

Browsertrix is the hosted, high-fidelity, browser-based crawling service from Webrecorder designed to make web archiving easier and more accessible for all!
https://webrecorder.net/browsertrix
GNU Affero General Public License v3.0
201 stars 35 forks source link

Add nightly integration tests for modifying org storage #2114

Open tw4l opened 1 month ago

tw4l commented 1 month ago

Follow-up to #2093

Related to #578

The backend tests for org storage cover adding and removing a custom storage, as well as setting the primary and replica storage locations for an organization that hasn't yet crawled anything.

When primary and/or replica storage locations are changed in an organization that already has stored crawl, upload, and/or browser profile files, file migrations take place via backend jobs. We should cover this end-to-end process via integration tests, verifying that the migration completes as expected, all files have been successfully transferred, and downloading/replaying files works as expected.