Research faculty should be included

[dbrumley]

At CMU Research-track faculty can fully and independently advise PhD students. For example, Anupam Datta and Lujo Bauer were both originally research faculty, and as such, advised phd students and published in top-tier venues. Both are top-notch, but no one using csrankings would have seen their names in that position.

Leaving research faculty off the list prevents anyone using it from having a full picture of the security and privacy activity, or for a graduate student, whom they might be able to work with.

I believe research faculty are consistent with the mantra on the first page:

This ranking of top computer science schools is designed to identify institutions and faculty actively engaged in research across a number of areas of computer science, based on the number of publications by faculty that have appeared at the most selective conferences in each area of computer science

As well as your faq:

The primary audience of CSRankings is prospective graduate students who are seeking a postgraduate degree in Computer Science.

Research track at CMU fit both criteria. As a suggestion for a change, I suggest the criteria be formally changed to "Anyone who has a faculty appoint, and can independently advise PhD students in computer science."

[emilioferrara]

I completely agree. The same reasoning holds for USC. Research faculty in the CS department have the same ability of supervising phd students, and represent a significant fraction of the faculty body and the research output of the department (see https://www.cs.usc.edu/people). Not including research faculty in the ranking make it less complete, reliable and relevant.

[emeryberger]

I worry that this would create an incentive to not hire tenure-track faculty.

[emilioferrara]

Your concern is based on your personal view that being on a tenure-track is comparatively better than being on a research track, which is an assumption that I (and many others, for that matter) could respectfully disagree with. I could argument extensively about it, but I think that it would be beyond the scope of this discussion.

If the purpose of CSranking is to portrait top CS department and "identify institutions and faculty actively engaged in research across a number of areas of computer science" (as your mission says), then ignoring that some top CS departments in the US pose a strong emphasis on research faculty, and omitting these faculty, simply biases the ranking.

For example, the USC CS research faculty includes some very well-known top CS scientists. Some faculty missing from your list have hundreds of papers and tens of thousands of citations. The same seems true for spot-checking other departments with similar structure and several CS research faculty (e.g., CMU, UIUC, Brown, Northeastern, etc.).

Furthermore, your omission poses a paradox: research faculty spend 100% of their time on research, and generally manage much larger groups of PhD students, massive amounts of funding sufficient to support many scholarships, etc. This information should be valued by e.g., students looking for an institution where research is highly valued and research opportunities will emerge more easily.

[emeryberger]

It is not a personal view that tenure-track faculty positions represent a potentially permanent commitment by an institution to a faculty member, which is categorically different from a research position.

It is also a meta-goal of CSrankings to be incentive aligned. Undermining institutional incentives to grant tenure and establish tenure lines is not something to be taken lightly, to say the least. There is a reason that tenure is sought after by many and widely considered desirable.

In any event, the fact that tenured faculty have a guaranteed job might also make a difference to graduate students, given the vagaries of funding and other vicissitudes; including non-tenure track faculty without demarcation in the database may potentially mislead students.

[emilioferrara]

Following the same reasoning you should have a demarcation between faculty with tenure and untenured ones, given that some don't get tenure and have to move to other institutions or go to industry. This can also potentially mislead students. Not to say that academics transfer on occasion to other universities, regardless of tenure!

Speaking from what I know: Some of the largest research groups at USC CS are lead by research faculty, and the same holds for most of the largest grants and projects. The same is true for many other institutions.

A ranking that accounts only for arbitrary research outputs and/or faculty is problematic as it skews the public perception of departments and their research capabilities.

In your mission on the home page you state that rankings based on surveys etc can be easily manipulated: your arbitrary rules to include some faculty and not others make your ranking biased as well.

I hope you'll change your position in the future as it sets a dangerous precedent that can be interpreted as if research faculty are second order researchers whose output doesn't matter. Until then I won't feel comfortable to endorse this ranking.

[emeryberger]

Tenured vs. untenured: Given that practically every university routinely tenures its tenure-track faculty, I don't think demarcating untenured from tenured faculty would serve any useful purpose to anyone, and it would add considerable churn and maintenance overhead (and so is not going to happen).

While I am open in principle to including research faculty (though I continue to maintain that there is a potential for confusion among grad students), I would like to hear a convincing counterargument to the risk I observe that incorporating contingent faculty (specifically research faculty) might act as an incentive for institutions to reduce their hiring of tenure-track faculty.

[dbrumley]

Emory: feels like you are conflating politicking and research productivity. Isn't CS rankings about the latter, and things like blog post better for the former? I'm just going back to the goal of the ranking, posted above.

If the goal is to help inform new graduate students, wouldn't it better serve them to have data based on who could advise them today rather than the politics of whether tenure is a good idea or not?

Your argument that i) it might bias and ii) universities routinely tenure seems biased on your experience. When I joined CMU 4 faculty went up for tenure in the college, only 1 got it. Tenure track are often weeded out as a practice before they go up for tenure as well, so counting how many fail the vote has survivor bias.

At CMU, we value research faculty as much as tenure track. Wouldn't your ranking be sending a very specific signal that they should not be valued as much?

As far as evidence on the particulars of bias, the only data I have is at CMU a number of faculty started out as research track and moved to tenure track, and a number who are quite happy as research faculty without moving. The ability to hire research faculty increased the universities ability to do research and advise the next generation. I see tenure track limits, at least at CMU, as an imposition at the trustee level of the number of lifetime commitments they have outstanding, not a statement of who is better.

[stephenrkell]

I just wanted to +1 this issue.

I understand the concern about incentives to commit... but even in the US it's clearly not as simple as "tenure-track good, others bad". Internationally it's even more complex, because there are many shades of grey around the space of "contingent" versus "tenured" (and not just grey... it's far from one-dimensional).

[ahmaurya]

+1

IMHO, it is not necessary to include all faculty in the default, but there perhaps can be an option (checkbox) which allows including non-TT/research faculty, maybe next to the options selecting the geographical region and the year range. If it ultimately comes down to top-tier research publications, not sure if it matters if the faculty is TT or not. If a school provides non-TT faculty their primary affiliation and facilities to do top-notch research, it should be counted at least in a non-default view that prospective doctoral students can select for themselves.

[github-actions[bot]]

Stale issue message