search

bids-standard / bids-specification

Brain Imaging Data Structure (BIDS) Specification
https://bids-specification.readthedocs.io/
Creative Commons Attribution 4.0 International
273 stars 156 forks source link

how popular is BIDS? #225

Closed Remi-Gau closed 5 years ago

Remi-Gau commented 5 years ago

I gave a quick intro to BIDS as part of a crash course on code and data management in my department and I was asked how popular is BIDS and I had did not have any numbers.

I am wondering if anyone has some information on that.

sappelhoff commented 5 years ago

@poldrack was planning to collect some data on that, perhaps he can say more.

cc @franklin-feingold, who was also involved in setting up the questionnaire

poldrack commented 5 years ago

we have prepared a poll which we will unleash shortly. here it is if you want to comment: https://docs.google.com/forms/d/e/1FAIpQLSckiZQ5GkM4fSdnBUlM5xnBqlyyE7jTYQYs7zJy9Gl-9E-PUw/viewform?usp=sf_link the only remaining question is whether we should collect emails from everyone (for tracking only) or only from those who are interested in receiving regular news. thoughts appreciated.

Remi-Gau commented 5 years ago

hum... This kind of assumes people answering the poll even know what BIDS is, no?

Remi-Gau commented 5 years ago

Some questions that I was asked yesterday. Not sure if it helps with the poll. How many BIDS data set are there out there? How many studies have their data set as BIDS?

So maybe asking people how many (or what percentage) of their data sets are in a BIDS format?

yarikoptic commented 5 years ago

FWIW, at Dartmouth we have the majority of the collected datasets in the past 3 years converted to BIDS (>40 IIRC)... I think nobody even knows that it could have been otherwise. Lack of "formal" derivatives spec though might be giving them all leeway to format their analyzed data/results any way they want, but I know that many do use fmriprep so at least some are probably "compliant". But overall, % of "raw" datasets is high here, but of "derived" is probably very low.

Remi-Gau commented 5 years ago

Yeah I am just "afraid" and would like to know how much of what we are seeing around us is not just a reflection of the local environment or the echo-chamber we live in.

poldrack commented 5 years ago

we did an informal poll last year and had a list of something like 60+ groups that were actively using BIDS in their labs.

On Thu, May 9, 2019 at 7:57 PM Remi Gau notifications@github.com wrote:

Yeah I am just "afraid" and would like to know how much of what we are seeing around us is not just a reflection of the local environment or the echo-chamber we live in.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bids-standard/bids-specification/issues/225#issuecomment-491004363, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGUVEFWZFBIIUCQIYO6SNDPURQY7ANCNFSM4HL2YD2A .

-- Russell A. Poldrack Albert Ray Lang Professor of Psychology Professor (by courtesy) of Computer Science Bldg. 420, Jordan Hall Stanford University Stanford, CA 94305

poldrack@stanford.edu http://www.poldracklab.org/

yarikoptic commented 5 years ago

what was the total # of datasets reported by mriqc call home "feature"? were IPs/geo-ips collected and nice map plotted similar to our elderly neurodebian worldmap?

poldrack commented 5 years ago

@oesteban can say more...

On Thu, May 9, 2019 at 11:19 PM Yaroslav Halchenko notifications@github.com wrote:

what was the total # of datasets reported by mriqc call home "feature"? were IPs/geo-ips collected and nice map plotted similar to our elderly neurodebian worldmap https://www.dartmouth.edu/~ccn/software/neurodebian_worldmap2.jpg?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bids-standard/bids-specification/issues/225#issuecomment-491070207, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGUVEHB3OVZWJQIACHB4XDPUSIMVANCNFSM4HL2YD2A .

-- Russell A. Poldrack Albert Ray Lang Professor of Psychology Professor (by courtesy) of Computer Science Bldg. 420, Jordan Hall Stanford University Stanford, CA 94305

poldrack@stanford.edu http://www.poldracklab.org/

oesteban commented 5 years ago

what was the total # of datasets reported by mriqc call home "feature"? were IPs/geo-ips collected and nice map plotted similar to our elderly neurodebian worldmap?

I can tell # of different images but, in general, MRIQC does not tell what dataset originated the data nor the geospatial origin of the hits.

We have Google Analytics stats about visits to the documentation of MRIQC and fMRIPrep which can give a hint on the spread of interest across the world.

nicholst commented 5 years ago

@poldrack - Two responses:

The survey look good, but wonder if it's worth getting estimates of "For how many subjects have you created BIDS datasets"? or something to that extent... could be free-form integer or ranges with radio buttons.

On emails... +1 for creating an email roster. As I understand it, for engagement/tracking, we currently have just the google mail list and association with github repos. If I were a least-committed BIDS user (i.e. no interest in developing, but just using), I would like to be on very low-volume email dlist (BIDS-registered-users ??), which would also sever to give an estimate of BIDS users (and would only get sent emails about key updates or critical bugs).

satra commented 5 years ago

@poldrack - perhaps we can add a question on whether they use any BIDS-Apps

"Do you use any BIDS datasets from others or BIDS-aware applications or libraries (e.g., MRIQC, fMRIPrep, pybids, ...)"

or turn it into a multiple select checkbox:

I use:
- [ ] BIDS-aware applications or libraries (e.g., MRIQC, fMRIPrep, pybids)
- [ ] BIDS datasets from public sources or collaborators outside my group

this would help capture people who are familiar with bids, don't generate any datasets, and reuse BIDS datasets.

on a related note we will be enabling telemetry on heudiconv. so we should be able to capture how many folks are converting data to bids. on docker hub, heudiconv has a 100K+ pulls about 2 times more than fmriprep and 10 times more than nipype container (hence not a testing effect - we don't change the code anywhere close to that of fmriprep or nipype). this doesn't mean everyone is converting to bids, which is what we hope to get if we use telemetry (to detect enabled flags such as the bids flag or the reproin heuristic and on how many subjects). but heudiconv can be parallelized, so pulls could still correspond to individual conversions (just don't know how many are to bids).

poldrack commented 5 years ago

ok, have edited the form based on these suggestions - will unleash it Monday unless I hear otherwise from y'all.

On Fri, May 10, 2019 at 4:15 AM Satrajit Ghosh notifications@github.com wrote:

@poldrack https://github.com/poldrack - perhaps we can add a question on whether they use any BIDS-Apps

"Do you use any BIDS-aware applications or libraries (e.g., MRIQC, fMRIPrep, pybids, ...)"

this would help capture people who are familiar with bids, don't generate any datasets, and reuse BIDS datasets.

on a related note we will be enabling telemetry on heudiconv. so we should be able to capture how many folks are converting data to bids. on docker hub, heudiconv has a 100K+ pulls about 2 times more than fmriprep and 10 times more than nipype container (hence not a testing effect - we don't change the code anywhere close to that of fmriprep or nipype). this doesn't mean everyone is converting to bids, which is what we hope to get if we use telemetry (to detect enabled flags such as the bids flag or the reproin heuristic and on how many subjects). but heudiconv can be parallelized, so pulls could still correspond to individual conversions (just don't know how many are to bids).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bids-standard/bids-specification/issues/225#issuecomment-491252932, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGUVEGEFEULWXWAOSAYLADPUVKLTANCNFSM4HL2YD2A .

-- Russell A. Poldrack Albert Ray Lang Professor of Psychology Professor (by courtesy) of Computer Science Bldg. 420, Jordan Hall Stanford University Stanford, CA 94305

poldrack@stanford.edu http://www.poldracklab.org/

sappelhoff commented 5 years ago

What will help in addition to the results from the survey are numbers on

I think we are tracking these things. Perhaps @franklin-feingold would know more about it?

sappelhoff commented 5 years ago

got the info:

We have about 650 monthly visitors to the specification and 1,700 monthly visitors on the bids.neuroimaging.io site

sappelhoff commented 5 years ago

@Remi-Gau see: https://medium.com/stanford-center-for-reproducible-neuroscience/bids-usage-survey-results-72637ff039c4