Add a DandisetBackup class that takes care of all repository-wide operations for backing up a Dandiset
Primary attributes:
dandiset: RemoteDandiset
ds: AsyncDataset
manager: Manager
Merge all Syncer methods into a single method of this class
Move the following Datasetter methods to this class:
update_dandiset()
ensure_github_remote()
sync_dandiset()
part of update_github_metadata()?
tag_releases()
mkrelease()
backup_zarrs()
Make update_dandiset_metadata() a method of this class
Make async_assets() a method of this class
Make register_s3urls() a method of this class
Add a ZarrBackup class for subsuming ZarrSyncer?
Wrap this block in a function that takes a DandisetBackup, aiterassets() async iterator, tracker, & error_on_change and returns a structure containing Report, dm.last_timestamp, dm.need_add, & dm.zarrs
Move the addurl methods & attributes of Downloader to an Addurl(?) class, an instance of which will be passed to Downloader on instantiation
Add a Superdataset class that takes care of operations on the superdataset
WIP design document for a new code architecture
Add a
DandisetBackup
class that takes care of all repository-wide operations for backing up a Dandisetdandiset: RemoteDandiset
ds: AsyncDataset
manager: Manager
Syncer
methods into a single method of this classDatasetter
methods to this class:update_dandiset()
ensure_github_remote()
sync_dandiset()
update_github_metadata()
?tag_releases()
mkrelease()
backup_zarrs()
update_dandiset_metadata()
a method of this classasync_assets()
a method of this classregister_s3urls()
a method of this classAdd a
ZarrBackup
class for subsumingZarrSyncer
?Wrap this block in a function that takes a
DandisetBackup
,aiterassets()
async iterator,tracker
, &error_on_change
and returns a structure containingReport
,dm.last_timestamp
,dm.need_add
, &dm.zarrs
Move the
addurl
methods & attributes ofDownloader
to anAddurl
(?) class, an instance of which will be passed toDownloader
on instantiationAdd a
Superdataset
class that takes care of operations on the superdataset