search

datalad / datalad-crawler

DataLad extension for tracking web resources as datasets
http://datalad.org
Other
5 stars 16 forks source link

Table for all subjects/files of HCP1200 for addurls #58

Closed TobiasKadelka closed 3 years ago

TobiasKadelka commented 4 years ago

Hello, I am now done with generating the complete table for hcp, with a row for everyfile, containing the version-number, file name, aws path/link and subject-ID so it can be used for the datalad addurls command.

Under https://jugit.fz-juelich.de/t.kadelka/hcp_table I have the table and the script for generating the table. If you like, I would be happy to get feedback or ideas about how to get the same information faster (my python script basically just runs "datalad ls" recursively for every subject and reads the information from the string it gets back).

@mih, the jugit-links should be readable from outside the Forschungszentrum?

bpoldrack commented 4 years ago

@TobiasKadelka : That link is invalid. Either that repository is actually named differently or it's not public.

TobiasKadelka commented 4 years ago

Thanks for the information, changed it.

yarikoptic commented 3 years ago

Since HCP datasets were already "done" and without crawler, I will just close this one