Migrating datafiles to different file storage locations

[steveandroulakis]

One strategy for managing physical storage is to migrate the data files from one storage location to another, while keeping the metadata in the MyTardis database.

At the low level, the process would consist of the following steps:

• copy a file to its new location (with a new URL)
• check that the file has reached the new location safely by requesting its size and/or hash at the new URL
• update the MyTardis Dataset_File record to the new URL
• remove the old copy of the file.

We should be able to implement the basic migration functionality without changing the MyTardis core model.

original LH ticket

This ticket has 0 attachment(s).

[steveandroulakis]

Migrating datafiles to different file storage locations

by Stephen Crawley

[steveandroulakis]

Migrating datafiles to different file storage locations

This is about half-way implemented in my datafile-migration fork. The low-level stuff (migration of selected datafiles) is implemented and available as a django command. I’m working on the stuff that decides what to migrate ... based on a scoring heuristic. I’m doing this as a MyTardis "app".

by Stephen Crawley

[steveandroulakis]

Migrating datafiles to different file storage locations

I’ve submitted the pull request for a preliminary version.

by Stephen Crawley