Possibility to reconsider adding medical image analysis as an area

[marcniethammer]

Dear Emery,

thanks for considering our request for adding medical image analysis as an area to csrankings. Medical image analysis has become a very active subarea of computer science and csrankings has become a very valuable resource for students to assess CS programs and to determine the strength of CS departments in individual subject areas. Thank you very much for putting this great resource together and maintaining it. To allow students in medical image analysis to determine which department may be best for them it would, of course, be great if it could be included in csrankings as well. That there is a strong desire by the community to do so was highlighted by the support over the last days for our petition ( #558 ). Among many others, the petition received strong support from a large number of senior faculty in our field as well as from various department chairs of computer science departments across the country and abroad.

Closing the request for adding medical image analysis as an area due to the fact that our "top conferences (MICCAI and IPMI) just [have] 340 [entries in the database]" while other areas that are included in csrankings have much higher counts is reasonable. Unfortunately, it appears that the 340 count may be due to a miscount in the database as explained in the following.

MICCAI is not listed simply as 'MICCAI' in DBLP. Since MICCAI is a large conference its proceedings are published in multiple volumes in Springer's Lecture Notes on Computer Science (LNCS). Consequentially, the corresponding entries in DBLP are listed as 'MICCAI', 'MICCAI (1)', 'MICCAI (2)', 'MICCAI (3)'. To test the hypothesis that this explains the difference, I downloaded the current DBLP database and ran the csranking scripts (with a faculty list from about a month ago, which may explain slight deviations from your results) including only 'MICCAI' or all MICCAI identifiers (as given above) as well as IPMI.

If I use only 'MICCAI' I get a resulting count of 349 (which seems consistent with your number, so I assume I am doing a similar query to yours, correct?). If I include all identifiers I obtain a count of 1161, which is well within the range of other areas that are included. For example metrics: 1198, mobile: 1223, bio: 1046, bed: 1172, vis: 1176, ecom: 736.

What may also be important to consider is that many medical image analysis faculty members that could be included in csrankings are currently not included. Quite a few senior researchers with strong publication records are still missing. I think this is a somewhat orthogonal issue to our request for adding the area itself. It would be great if we could include these researchers with your help in an organized way that puts limited strain on your time. I think a good way of doing this would be for us to collect people in medical image analysis that are missing and put them all together in one git pull request (instead of creating many individual requests for you). I could then send you the pull request in the next couple of weeks. Would this work? Once these people are included, I am confident that the overall count in your database would increase significantly over the already significant count based on the researchers currently included in the database.

If my reasoning makes sense to you, I would like to request re-considering the addition of medical image analysis in csrankings. I understand that these decisions need to be made with care and truly appreciate your efforts in coming up with a fair solution. If there is any advice you could give us on strengthening our request and how to direct our efforts to clarify our case we would greatly appreciate it.

I attached the raw data and the script I used to create the numbers for replication.

Many thanks in advance for possibly re-considering our request.

Marc

Here are the numbers for all the areas:

plan: 2021 hpc: 2220 log: 1678 soft: 2015 ops: 1752 arch: 2360 act: 6184 comm: 4433 sec: 2410 ai: 5954 mod: 3479 graph: 2210 metrics: 1198 ir: 1903 chi: 2825 nlp: 2470 vision: 6334 mobile: 1223 robotics: 4094 crypt: 1509 bio: 1046 da: 1867 bed: 1172 vis: 1176 ecom: 736 Medical image analysis (IPMI, MICCAI, MICCAI (1), MICCAI (2), MICCAI (3): 1161 Medical image analysis (only MICCAI and IPMI): 349

data.zip

[wentaozhu]

Strongly support!

PhD, UC Irvine Computer Science Department Wentao

[emeryberger]

(0) Thanks for the clarification. (1) Please don't expect a decision on this to be made soon (holidays, final classes, exams, and holidays). (2) Faculty who do not meet the inclusion criteria are not going to be included. (3) I kindly ask that people stop spamming the Github issues.

[marcniethammer]

Dear Emery,

As there have been a lot of messages going back and forth, I thought it may be a good idea to briefly summarize the case for including medical image analysis as a new area in csrankings.org.

In my opinion, medical image analysis (with the main conferences MICCAI, IPMI, and ISBI) meets all your inclusion criteria for the following reasons. Each reason is supported by data.

1) More than 50 computer science departments of R1 institutions have had publications in these conferences in the last 10 years. In fact, this is even the case when only considering MICCAI (see CS_department_MICCAI_publications.pdf for a list of MICCAI papers by different R1 CS departments).

2) Medical image analysis is supported by the MICCAI Society (http://www.miccai.org), which is equivalent in spirit to an ACM Sig. (Similar to computer vision, natural language processing, cryptography, robotics, and visualization, which all have their own respective associations instead of an ACM Sig.)

3) There is very strong community support in computer science for this area. Evidence for this is the original petition letter (csranking_petition.pdf; this letter also includes a detailed reasoning for inclusion), direct supporting statements for this petition (#558) by computer science faculty including various department chairs of computer science departments (in the US and abroad), as well as the support by the MICCAI society.

4) Medical image analysis is a sizable subfield of computer science. Evidence for this are the number of papers recorded by the csrankings system (based on generated-author-info.csv -- with medical image analysis added considering only MICCAI and IPMI; here is the script to allow replication of results results.txt). Medical image analysis ranks 19/26, i.e., 18 areas are larger and 7 smaller. The breakup is as follows (with mia shorthand for medical image analysis using data from this afternoon): vision: 6651; act: 6359; ai: 6284; robotics: 4352; mod: 3632; chi: 3024; nlp: 2733; sec: 2575; arch: 2395; hpc: 2299; graph: 2269; soft: 2136; ir: 2066; plan: 2043; da: 1919; log: 1700; crypt: 1524; comm: 1427; mia: 1326; mobile: 1290; ops: 1279; metrics: 1248; vis: 1232; bed: 1209; bio: 1061; ecom: 772.

5) The acceptance rates for IPMI and MICCAI are similar (around 30%; see original petition letter for details). The acceptance rate for ISBI is higher. Based on previous discussions on github, it seems reasonable to focus on IPMI and MICCAI only as they form an equivalence class.

If there is any additional information you would like me to provide that would further strengthen the case, please let me know.

Many thanks in advance for re-considering this request and good luck with the end of the semester.

Marc

PS: I also very much realize (especially after creating some pull requests to add missing faculty) how labor intensive it is to run a system such as csrankings. Thank you for doing this for the community! Should you decide in favor of including medical image analysis, I would, of course, be very happy to prepare a pull request to help you with the work. Just let me know.

[marcniethammer]

Hi @emeryberger . Any updates on this issue? In particular, is there any additional information you would like me to provide to make the case for the inclusion of medical image analysis?

Also, I had some time over the holidays and created a version of csrankings that includes medical image analysis. If you would like me to, I'd be happy to send you a pull request for this version.

[marcniethammer]

Hi @emeryberger . Any updates on this?

[marcniethammer]

Hi @emeryberger, could you let me know what the current status of this request is?

I think there is a very compelling case that medical image analysis should be included as one of the interdisciplinary areas. This is backed up by the data I gathered. This data also confirms that medical image analysis is well within the equivalence class of areas that are already included in csrankings.

That this area is part of computer science is evident, given publications by computer science faculty in the area (demonstrated by the list of publications I gathered at the beginning of November upon your request). In fact, the acronym for our main conference, MICCAI, stands for Medical Image Computing and Computer Assisted Intervention.

csrankings, in my understanding, aims at providing prospective CS students with the ability to explore different universities with respect to research areas and associated faculty. Including medical image analysis is therefore important to allow students to explore this growing research area of CS and to make informed decisions regarding admission offers.

Could you let me know what your current reservations for inclusion are, if there is any more information I can provide, and/or if you are currently conducting a survey to assess if the area could be included or not?

As offered previously, I would be happy to provide a pull request to help with the work.

Many thanks in advance for providing an update.

[marcniethammer]

Dear @emeryberger , sorry for being so persistent here, but could you give me an update on this issue and let me know what your thoughts are? Many thanks in advance.

[emeryberger]

Rather than make a decision strictly on my own, over the last months, I have spoken to a wide range of people about this topic (at many institutions, across many areas of CS), with all the relevant context. Unfortunately, there is broad consensus that this topic is not widely recognized as a distinct area of CS and so should not be included as a top-level one in CSrankings.

[emeryberger]

Let me say that I appreciate that you have done a lot of work on this and that I personally appreciate your contributions to CSrankings. I know this decision is unwelcome but it was treated with a high degree of discussion and careful thought.

[marcniethammer]

Emery, who have you spoken to? Should this not be revealed in the interest of transparency?

As you requested, I have done everything openly on github. And there has been a huge outpouring of support by our community. Furthermore, the data bears out that there is a substantial activity of the area in CS. So to come back with the argument "I have spoken to people that say it is not part of CS" seems extremely frustrating, arbitrary and opaque to me.

Is the entire premise of csrankings not to make decisions as quantitative as possible? This decision is not founded on data at all. It seems to be founded on personal opinion.

Have you asked people in our community that are in CS? Maybe you have and then I rest my case, but if not, I am not sure what this means? I am certainly not claiming I can judge other people's areas, but I am able to look at quantitative data.

[emeryberger]

I am of course not going to share the content of private discussions or name names. While CSrankings is largely metrics based, there are editorial decisions that occasionally need to be made (esp. with respect to binning of areas, methodology, and so on), and this is one. (Personally, I would rather not have to make them, but that's the current situation, so I have sought advice and discussion from many people whose opinions I value.) Suffice it to say that this is an editorial decision on my part that reflects a considerable amount of thought and care. I am placing a moratorium on adding new areas to CSrankings through at least 2018.

[marcniethammer]

Maybe a compromise could be to add MICCAI to an existing area. The area that immediately comes to mind is 'computational biology and bioinformatics'.

It seems like a good fit for the following reasons:

1. Thematically it would be a good fit, in particular, as these areas become more interdisciplinary and multimodal (think about imaging-genetics for example).
2. No new main area would need to be added.
3. It currently only has two conferences listed, so adding a third would not break your rules.
4. You would not need to make an editorial decision regarding adding a new area and your moratorium could stay in place.
5. It would strengthen this subarea in terms of quantitative measures, as it currently appears to be the second smallest. With the addition of MICCAI it would end up roughly in the middle of the pack.
6. csmetrics also includes IPMI and MICCAI. If I understand your concerns properly, the issue is less if medical image analysis is an area of CS (I think it is), but rather if it should be its own main area.

Thoughts? This seems like a reasonable solution to me that would make everybody reasonably happy.

[innat]

@emeryberger @marcniethammer I'm desperately seeking something similar to CSRanking for Medical Image Analysis. The debate here and there is too long to read as a newcomer. So apologies if this comment seems to be spam.

I like to know

1. Is the decision persist, any update?
2. Can it be included as Interdisciplinary Areas, HERE. (There is already an option of Comp. bio & bioinformatics)
3. Is there any other alternative site, that provides the requested features?

[marcniethammer]

As far as I know there have unfortunately not been any updates from @emeryberger . And there are many outstanding requests by others (see here #2234 for an open issue which has been left unaddressed for >2 years now).

There have also been some recent tweets about not including ICLR for example (see some ICLR open issues here #240 #4683 #4442 ), but note that these non-inclusion issues cut across many areas.

A possible alternative site is csmetrics. Though this site does not drill down to individual faculty level.

Unfortunately, csrankings can not be forked and modified due to the chosen license.