search

dandi / dandisets

730 Dandisets, 807.1 TB total. DataLad super-dataset of all Dandisets from https://github.com/dandisets
10 stars 0 forks source link

Separate out backups2datalad into an independent repo/project #375

Closed yarikoptic closed 9 months ago

yarikoptic commented 9 months ago

to facilitate reuse by @kabilar and @aaronkanzer for an independent deployment of DANDI

probably just worth making it into a proper pypi project and just installing from it to not dance with git submodules

jwodder commented 9 months ago

@yarikoptic Do you mean you actually want this released on PyPI, or just that it should be installable with pip?

yarikoptic commented 9 months ago

Probably not worth releasing on pypi indeed and we can just install directly from git with pip. I just meant that we probably do not want to do the dance with submodules like we do/did elsewhere unless you see the benefit in that

jwodder commented 9 months ago

@yarikoptic What should happen to the config file and all of the various ad hoc scripts in tools/? Should they be left here or be moved to the new repository?

yarikoptic commented 9 months ago

@yarikoptic What should happen to the config file and all of the various ad hoc scripts in tools/? Should they be left here or be moved to the new repository?

I guess we could move them as the need comes and so they are implemented as subcommands within that interface, and just point to this tools/ as an example.

jwodder commented 9 months ago

New repository: https://github.com/dandi/backups2datalad. I'll close this issue once the packaging PR is merged after its CI completes.

I extracted backups2datalad with its Git history with the following script:

#!/bin/bash
set -ex -o pipefail

git clone git@github.com:dandi/dandisets.git

cd dandisets

git-filter-repo \
    --replace-refs delete-no-add \
    --path-rename tools/: \
    --path-glob 'tools/backups2datalad/*' \
    --path-glob 'tools/test_backups2datalad/*' \
    --path-glob '.github/*' \
    --path tools/backups2datalad.req.txt \
    --path tools/setup.cfg \
    --path tools/noxfile.py \
    --path tools/mypy.ini \
    --path tools/test_backups2datalad.py \
    --path tools/test_util.py \
    --path tools/backups2datalad.py \
    --path .gitignore \
    --path .pre-commit-config.yaml