search

emeryberger / CSrankings

A web app for ranking computer science departments according to their research output in selective venues, and for finding active faculty across a wide range of areas.
http://csrankings.org
Other
2.69k stars 3.17k forks source link

CSrankings has essentially no representation of scientific computing/numerical analysis #856

Closed andrewcmyers closed 6 years ago

andrewcmyers commented 6 years ago

This is a research area that goes right back to the beginnings of computer science, but it basically does not appear in CSrankings except perhaps via computational biology. So, for example, CSrankings thinks that Cornell's David Bindel is an ML researcher because he happens to have published some NIPS papers. But that is hardly his primary focus. It is absurd that mobile computing counts as a research area but scientific computing does not.

I would guess the problem is that this area of CS is centered around journal publications. But that doesn't make it invalid. Some way should be found to include it.

I suggest expanding the set of venues covered to include journals where there are not enough top conferences in a given area. The decision process for what to include should not be harder than the existing process in which a consensus is sought regarding the top publication venues for each area.

emeryberger commented 6 years ago

There's a moratorium in place for adding new areas of CS through at least 2018.

Scientific computing is represented to some extent in the area of HPC (high-performance computing), at least at large-scale.

With respect to David Bindel, here's his publication record (http://www.cs.cornell.edu/~bindel/pubs.html). His work has recently appeared at NIPS (2015/2017), KDD (2015), ECCV, WWW, etc. He does publish in some SIAM Journals, but this is a small fraction of his output (~ 6 out of 61).

andrewcmyers commented 6 years ago

The SIAM articles may be a small fraction of the total publication count but they are a large fraction of the top-tier publications. I expect that's true for a lot of people in this research area.

fycus-tree commented 6 years ago

@andrewcmyers I tried to look into this for you, and unfortunately, simply not that many people publish in this area. Other conferences (AISTATS, WACV, UAI, WCNC, SECON, etc.) have more R1 publications than anything in numerical computing. SIAM J. Comput. does have > 50 R1 universities publishing in it the last 10 years, but is 57th in the "not currently in CSRankings", as ordered by # of R1 universities publishing there (SIAM J. Scientific Computing is 183rd).

In the process, I did make some discoveries about the under-representation of distributed computing (#423) & data mining (#238) and perhaps the over-representation of other fields (#857)

andrewcmyers commented 6 years ago

It's not clear what is the right way to treat the SIAM journals, since there are a bunch of them and scientific computing is spread across the whole set, as I understand it. If you did the R1 count for all of them together, presumably you'd get a higher number.

fycus-tree commented 6 years ago

@andrewcmyers If you combined all the SIAM journals, it'd be one of the top missing venues, with 84 R1 universities in the last 10 years with my numbers. It'd be right around INFOCOM (88), IPDPS (85), TPDS (85), CIKM (81), ICDM (78), ICDCS (75). Related to my previous comment, ICDCS & IPDPS form a distributed computing group (#423), and ICDM is a data mining conference (#238), while a few of the existing categories (logic, econ, see #857) have 40-something, so about half of any of these venues.

emeryberger commented 6 years ago

There are 18 SIAM journals, many of them explicitly about applied mathematics. They include titles like "Journal on Applied Algebra and Geometry", "Journal on Discrete Mathematics", "Journal on Financial Mathematics", and so on. Some of these definitely have "computational science" in mind, but others are plainly math and I think out of scope for CS.

Not only would the inclusion of journals be a substantial departure, but the creation of some Franken-journal grouping N journals (or even a Franken-conference grouping N conferences) is a wholly unprecedented notion and I think not a productive direction.

emeryberger commented 6 years ago

In any event, shelving until 2019.

fycus-tree commented 6 years ago

I should say that based on my quantitative analysis (#859), the SIAM journals do fall into different fields, and the broad topic to discuss is probably "should journals be included for some fields?"

That's it for SIAM journals with many R1 universities publishing in them.