search

poldracklab / tacc-openneuro

0 stars 1 forks source link

code/containers sometimes produces errors when being saved #5

Closed jbwexler closed 2 years ago

jbwexler commented 2 years ago

This has happened often enough that I've included a line in my script to, upon rerunning any subjects in a dataset, run `git annex repair --force' inside code/containers before datalad saving. I'm not sure why it happens but the above command fixes it every time. Note that this is dataset is an ephemeral clone. I have noticed that from time to time in the source dataset (the source of the ephemeral clones) I get similar errors when I try to change something manually in the dataset and then datalad save.

CommandError: 'git -c diff.ignoreSubmodules=none annex add --json --json-error-messages -c annex.dotfiles=true -- binds/HOME/.fontconfig/3830d5c3ddfd5cd38a049b759396e72e-le64.cache-7 binds/HOME/.fontconfig/CACHEDIR.TAG' failed with exitcode 1 under /scratch1/03201/jbwexler/openneuro_derivatives/derivatives/fmriprep/ds000002-fmriprep/code/containers [info keys: stdout_json] error: invalid object 100644 9e32eedcce4b18b994a9cc5ee7af362082d56e62 for '152/3f8/MD5E-s5388869663--8c81a812ccf38b91280ec35d609e5bb9.1.sing.log'\015\012fatal: git-write-tree: error building trees\015\012git-annex: failed to read sha from git write-tree\015\012CallStack (from HasCallStack): error, called at ./Git/Sha.hs:23:15 in main:Git.Sha

jbwexler commented 2 years ago

Here's an example of when my source copy of the containers dataset produces somewhat similar errors. What's odd is that I didn't (intentionally) delete either of those two files. I simply edited .datalad/config.

(main) login4.frontera(1067)$ datalad save delete(ok): .zenodo.json (file) delete(ok): licenses/.gitattributes (file) [WARNING] Received an exception CommandError(CommandError: 'git -c diff.ignoreSubmodules=none annex add --json --json-error-messages -c annex.dotfiles=true -- .datalad/config' failed with exitcode 1 under /scratch1/03201/jbwexler/openneuro_derivatives/containers [info keys: stdout_json] [err: 'error: invalid object 100644 9e32eedcce4b18b994a9cc5ee7af362082d56e62 for '152/3f8/MD5E-s5388869663--8c81a812ccf38b91280ec35d609e5bb9.1.sing.log' fatal: git-write-tree: error building trees git-annex: failed to read sha from git write-tree CallStack (from HasCallStack): error, called at ./Git/Sha.hs:23:15 in main:Git.Sha']). Canceling not-yet running jobs and waiting for completion of running. You can force earlier forceful exit by Ctrl-C. [INFO ] Canceled 0 out of 0 jobs. 0 left running. Total: 0%| | 0.00/1.00 [00:00<?, ? datasets/s]CommandError: 'git -c diff.ignoreSubmodules=none annex add --json --json-error-messages -c annex.dotfiles=true -- .datalad/config' failed with exitcode 1 under /scratch1/03201/jbwexler/openneuro_derivatives/containers [info keys: stdout_json] error: invalid object 100644 9e32eedcce4b18b994a9cc5ee7af362082d56e62 for '152/3f8/MD5E-s5388869663--8c81a812ccf38b91280ec35d609e5bb9.1.sing.log' fatal: git-write-tree: error building trees git-annex: failed to read sha from git write-tree CallStack (from HasCallStack): error, called at ./Git/Sha.hs:23:15 in main:Git.Sha

jbwexler commented 2 years ago

This seems to have been resolved. I think it was due to files being purged from the source containers dir from which these subdatasets are being ephemerally cloned.