Reopening this to look at the implications and maybe reconsider
Some things I want to tidy up:
Migration tool (I'm changing it to allow a copy of files before the file system is swapped, otherwise dice will take days to migrate)
File edit/save in admin (too slow, might need to ship some things off to workers)
More testing and real world usage in dice
Proper final review (needs a good sense check with a cleared head)
I've had to do a lot of work and make a few compromises to get the site to load in a reasonable amount of time when using s3 (or other remote back end). This includes things such as making local copies of files before working on them, splitting default resizing up into instant (for things admin will need when the page reloads) and worker based (for front end things like small, medium, etc.
I've also added heavy caching layers for things like filesize and imagesize which otherwise require pulling an entire file down to inspect.
I think this is working pretty well now.
DB MIGRATION TOOL
We need to be able to split up the migration, so we can copy large numbers of files without borking a site in between.
I've had a red hot go at this, by adding a tool to prepare / finalise / both at once.
This uses additional *migrated cols on the record. This will need a proper good test
Reopening this to look at the implications and maybe reconsider
Some things I want to tidy up:
I've had to do a lot of work and make a few compromises to get the site to load in a reasonable amount of time when using s3 (or other remote back end). This includes things such as making local copies of files before working on them, splitting default resizing up into instant (for things admin will need when the page reloads) and worker based (for front end things like small, medium, etc.
I've also added heavy caching layers for things like filesize and imagesize which otherwise require pulling an entire file down to inspect.
I think this is working pretty well now.
DB MIGRATION TOOL
We need to be able to split up the migration, so we can copy large numbers of files without borking a site in between. I've had a red hot go at this, by adding a tool to prepare / finalise / both at once.
This uses additional *migrated cols on the record. This will need a proper good test