Closed alexfoias closed 2 years ago
I tried to trace back when did the issue appear, but it was impossible due to the various changes in infrastructure.
you need to trace it back because we need to make sure:
so please, go back in time, look for emails, etc. document as much as possible. This is a very serious issue and we have a responsibility to not disseminate fake results.
- this error did not happen for other datasets
e.g.: the tokyoIngenia values are also suspicious:
sub-tokyoIngenia01 M 25 - - 2019-10-01 tokyo Ingenia the University of Tokyo Philips Ingenia - 5.3.1_5.3.1.1 "K. Kamiya, Y. Suzuki"
sub-tokyoIngenia02 F 32 - - 2019-10-01 tokyo Ingenia the University of Tokyo Philips Ingenia - 5.3.1_5.3.1.1 "K. Kamiya, Y. Suzuki"
sub-tokyoIngenia03 M 36 - - 2019-10-01 tokyo Ingenia the University of Tokyo Philips Ingenia - 5.3.1_5.3.1.1 "K. Kamiya, Y. Suzuki"
sub-tokyoIngenia04 F 29 - - 2019-10-01 tokyo Ingenia the University of Tokyo Philips Ingenia - 5.3.1_5.3.1.1 "K. Kamiya, Y. Suzuki"
sub-tokyoIngenia05 F 28 - - 2019-10-01 tokyo Ingenia the University of Tokyo Philips Ingenia - 5.3.1_5.3.1.1 "K. Kamiya, Y. Suzuki"
sub-tokyoIngenia06 M 24 - - 2019-06-13 tokyo Ingenia the University of Tokyo Philips Ingenia - 5.3.1_5.3.1.1 "K. Kamiya, Y. Suzuki"
sub-tokyoIngenia07 M 35 - - 2019-10-01 tokyo Ingenia the University of Tokyo Philips Ingenia - 5.3.1_5.3.1.1 "K. Kamiya, Y. Suzuki"
I checked in all my emails. I could only find demographics for Juntendo.
I think we received the data already converted to nifti. In the email conversation with @jcohenadad with the subject Re: Spinal cord generic protocol : consent from participants @ Tokyo Univ
sent on Wed, Apr 3, 2019 at 1:40 PM
, I asked if we had the raw data, but I didn't get an answer.
In the changes files, we don't have all the history of changes (the one from openeuro).
I don't know who added the data to the dataset.
This error seemed to be in the initial commit on openneuro https://github.com/OpenNeuroDatasets/ds002902
I checked in all my emails. I could only find demographics for Juntendo.
I think we received the data already converted to nifti. In the email conversation with @jcohenadad with the subject
Re: Spinal cord generic protocol : consent from participants @ Tokyo Univ
sent onWed, Apr 3, 2019 at 1:40 PM
, I asked if we had the raw data, but I didn't get an answer.
sorry for the oversight-- the answer is: no, we don't have the raw data
In the changes files, we don't have all the history of changes (the one from openeuro).
I don't know who added the data to the dataset.
me neither
OK, we know these are the same subjects. So the last sanity check is to verify that the sub01 on tokyoIngenia corresponds to sub01 on tokyoSkyra, etc. @alexfoias can you please do that (look at the T1w MRIs and you will easily find out if these are the same subjects)
I did a visual checkout and the subjects seem to be correctly identified (similar anatomy for all 3 centers).
Hey, I fixed CI, so now we can actually talk about reviewing this properly.
tokyoIngenia
values also need to be synchronized.Also, this is probably worth a different issue, but:
sub-tokyo*01
-> sub-tokyo01
, sub-vuiis*04
-> sub-vuiss04
) and represent the scanner used as a tag in the filenames?Should we delete https://openneuro.org/datasets/ds002902? Since it's wrong?
yes ( i thought we already did it @alexfoias ?)
Should we rearrange the dataset to put all the data from the same physical people under the same
sorry i don't understand your suggestion-- what do you mean by "data from the same physical people"?
I have removed all datasets from OpenNeuro. (This screenshot is take from the generic account used to create the datasets.)
.
I have removed all datasets from OpenNeuro.
:facepalm: you're right. Okay, punting: https://github.com/OpenNeuroOrg/openneuro/issues/2484
(This screenshot is take from the generic account used to create the datasets.)
By the way does anyone else have that credential? Does Julien? Should I? Better yet, can we make a Team on openneuro and all have our own credentials instead? Or are we giving up on OpenNeuro entirely?
Should we delete https://openneuro.org/datasets/ds002902? Since it's wrong?
yes ( i thought we already did it @alexfoias ?)
The older version is off OpenNeuro for good now: https://github.com/OpenNeuroOrg/openneuro/issues/2484
Should we rearrange the dataset to put all the data from the same physical people under the same
sorry i don't understand your suggestion-- what do you mean by "data from the same physical people"?
-> https://github.com/spine-generic/data-multi-subject/issues/102
2. The `tokyoIngenia` values also need to be synchronized.
And all tokyo750w*
too!: 10c5e459ffb494b10e8054191cffeb6e20e281e8
Still TODO: writing a good commit message.
- What is "the information from our collaborators" that triggered this update? Who are they? Names and academic email addresses?
- Let's make sure to write a relatively detailed commit message justifying this change.
@alexfoias could we see the email that inspired this PR?
New CI bug? https://github.com/spine-generic/data-multi-subject/runs/4226501836?check_suite_focus=true#step:11:2598
(recording state in git...) git-annex: get: 3 failed Error: Process completed with exit code 1.
Looks like maybe it's Amazon S3 having a minor outage?
get derivatives/labels/sub-stanford04/anat/sub-stanford04_acq-T1w_MTS_seg-manual.nii.gz (from amazon...)
download failed: Internal Server Error
Unable to access these remotes: amazon
Try making some of these repositories available:
5a5447a8-a9b8-49bc-8276-01a62632b502 -- [amazon]
5cdba4fc-8d50-4e89-bb0c-a3a4f9449666 -- julien@julien-macbook.local:~/code/spine-generic/data-multi-subject
9e4d13f3-30e1-4a29-8b86-670879928606 -- alex@NeuroPoly-MacBook-Pro.local:~/data/data-multi-subject
(Note that these git remotes have annex-ignore set: origin)
failed
get derivatives/labels/sub-tehranS01/anat/sub-tehranS01_T1w_labels-disc-manual.nii.gz (from amazon...)
download failed: Internal Server Error
Unable to access these remotes: amazon
Try making some of these repositories available:
5a5447a8-a9b8-49bc-8276-01a62632b502 -- [amazon]
5cdba4fc-8d50-4e89-bb0c-a3a4f9449666 -- julien@julien-macbook.local:~/code/spine-generic/data-multi-subject
9e4d13f3-30e1-4a29-8b86-670879928606 -- alex@NeuroPoly-MacBook-Pro.local:~/data/data-multi-subject
(Note that these git remotes have annex-ignore set: origin)
failed
get derivatives/labels/sub-tokyoSkyra03/anat/sub-tokyoSkyra03_T2star_rms_gmseg-manual.nii.gz (from amazon...)
download failed: Internal Server Error
Unable to access these remotes: amazon
Try making some of these repositories available:
5a5447a8-a9b8-49bc-8276-01a62632b502 -- [amazon]
5cdba4fc-8d50-4e89-bb0c-a3a4f9449666 -- julien@julien-macbook.local:~/code/spine-generic/data-multi-subject
9e4d13f3-30e1-4a29-8b86-670879928606 -- alex@NeuroPoly-MacBook-Pro.local:~/data/data-multi-subject
(Note that these git remotes have annex-ignore set: origin)
failed
The following run passed: https://github.com/spine-generic/data-multi-subject/runs/4226604983?check_suite_focus=true
@kousu Here is the info from the email:
Subject01 M 36 177cm 70kg Subject02 M 36 168cm 80kg Subject03 M 32 162cm 57kg Subject04 F 34 161cm 65kg Subject05 F 35 154cm 51kg Subject06 F 30 160cm 48kg Subject07 M 29 165cm 60kg
@alexfoias can you please update this PR after https://github.com/spine-generic/data-multi-subject/pull/112
Updated the information based on the information from our collaborators.
I tried to trace back when did the issue appear, but it was impossible due to the various changes in infrastructure.