emeryberger / CSrankings

A web app for ranking computer science departments according to their research output in selective venues, and for finding active faculty across a wide range of areas.
http://csrankings.org
Other
2.65k stars 3.04k forks source link

KDD should not be removed, and the current AI area taxonomy should be revised #5397

Open ninghaohello opened 1 year ago

ninghaohello commented 1 year ago

Someone told me he found that KDD is removed (as a default conference) from CSranking. And I was shocked.

Previously KDD was put into the "Machine learning" category (this looked strange to me, I admit), and was recently replaced by some other conference. I understand the developer want to limit the number of conferences to 3 for each category.

However, KDD is clearly a top-tier conference focusing on data mining, which is an important domain in AI. Also, KDD is in the same tier as SIGIR and WWW. There is a significant overlap between people working on "data mining" and those working on "Web & information retrieval". Therefore, KDD should not be simply removed from CSranking. I suggest changing "Web & information retrieval" to "Information retrieval & Data mining", and add KDD into this category.

lxiaorui commented 1 year ago

Thanks for this request.

I agree that adding KDD into the category of "Web & information retrieval" or "Web information retrieval & Data mining" is a great idea!

YingtongDou commented 1 year ago

KDD is ranked by Google Scholar as the top 1 venue in Data Mining & Analysis domain with h5-index=114.

I agree with @ninghaohello that it should not be omitted by CSrankings. Adding it to the "Web information retrieval & Data mining" category is a good proposal.

zhaoxyai commented 1 year ago

KDD is the best conference in Data Mining, which should not be overlooked by CSrankings.

AIhongzhi commented 1 year ago

I agree with @ninghaohello that it should not be removed by CSrankings, which does not make sense. Adding it to the "Web information retrieval & Data mining" category is suitable.

mjiang89 commented 1 year ago

It is unacceptable that CSRankings does not have a Data Mining category. ICDM/WSDM/SDM are not on the list of ~70 CS conferences. And now KDD is removed / put as a low-tier Machine Learning conference. No offense to other venues, but any evidence that the data mining conferences are less competitive? Any evidence that Data Mining does not belong to CS? Any research about Data Science / Data Mining textbooks / courses before ignore their impact?

rguo12 commented 1 year ago

It is probably unfair to exclude the KDD conference from the default ranking score. KDD is one of the best applied ML/DS venues where XGBOOST, LIME, and Node2Vec are published and even in its workshops, papers like Deep Cross Network are published.

Maybe it is also time to consider the domain-specific high-quality venues like TMLR, CleaR, FAacT and LoG etc.

Romanticone commented 1 year ago

KDD is the best conference in Data Mining. It is unreasonable to exclude it from the top conferences.

BuleSky233 commented 1 year ago

I would also agree with moving KDD to AI or The Web & information retrieval.

tonghanghang commented 1 year ago

I cannot understand why CSrankings excluded KDD - the premium conference on data mining. let alone exclude an entire area of data mining in a ranking system for CS.

emeryberger commented 1 year ago

This is not the final state of affairs. Please be patient.

arodriguezca commented 1 year ago

Great recommendation @ninghaohello. And thanks for paying attention to this @emeryberger.

cynthiamingxuan commented 1 year ago

I think it would be a better idea for the CS ranking system to identify the data mining area with top conferences including KDD. Data mining community is one of the largest CS research communities. KDD alone has attracted over 2000 submissions each year since 2020.

lalalandlala commented 1 year ago

Finally KDD is removed! It’s not on the same level as other ML conferences. But my suggestion is replacing IR track with data mining track, and replace SIGIR with KDD, since SIGIR is even worse!

Yujun-Yan commented 1 year ago

Excluding KDD will hurt the data mining community in general. It makes people undermine the value of data mining. I would like to kindly remind the developers that data mining and machine learning are two different fields, and if you must remove one ML conference to give way to ICLR, please consider ML conferences like NeurIPS and ICML (if you believe ICLR has better quality than those two). KDD is the TOP conference in data mining, which is not some secondary ML conference. Some researchers in data mining even don’t submit papers to ML conferences. I would suggest separating the fields of data mining and machine learning, and adding back KDD.

Amanda-Zheng commented 1 year ago

KDD should not be removed by CSrankings, it means a lot for data mining community.

Rachelxuan11 commented 1 year ago

Thanks for this post. I am sorry to see that KDD is removed from top conferences of Machine Learning & Data Mining, which would undermine contributions from the data mining community. If it must be excluded from the current Machine Learning top conferences. At least, I suggest to open a new track for data mining. Or please put it in the Web & Information retrieve track.

sayanranu commented 1 year ago

While merging with IR is better than all together removing KDD, the ideal solution is to have a data mining category and include KDD there (and perhaps WSDM as well).

kno10 commented 1 year ago

KDD was a data mining conference. Now, its mostly deep learning, too. The "data mining community" has ceased to exist. Sorry to say, but this is essentially a choice KDD made by itself, by becoming yet another ML conference.

If you take the completely flawed approach of CSranking and only count the number of publications at the top 3 conferences (ignoring that many are entirely uncited, problems of CSranking have been repeatedly pointed out years ago), KDD likely is out because NeurIPS, ICML and ICLR are more hyped right now. On Google Scholar:

Rank Title h5-Index h5-Median
9. International Conference on Learning Representations 286 533
10. Neural Information Processing Systems 278 436
19. International Conference on Machine Learning 237 421
?? ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 114 196

Now if you also merge ML and CV (because a lot of ML is about CV, and much of CV these days uses deep learning and ML), then

Rank Title h5-Index h5-Median
4 IEEE/CVF Conference on Computer Vision and Pattern Recognition 389 627

clearly would have to replace ICML.

The proposal to split ML and data mining into two within CSRankings was ignored years ago: #1109.

The simpler and better solution is simply to abandon CSrankings.

On the up side for KDD: if really some universities rely on this flawed ranking, this should reduce the number of submissions, and hence improve the review quality and the topic spectrum of KDD again.

LifangHe commented 1 year ago

Echo @ninghaohello's point, KDD should not be removed. Hopefully this concern can be well addressed.

lalalandlala commented 1 year ago

While merging with IR is better than all together removing KDD, the ideal solution is to have a data mining category and include KDD there (and perhaps WSDM as well).

KDD is second-tier, WSDM is third-tier, neither of them should be included.

YangLiangwei commented 1 year ago

KDD should not be removed from CSRankings. It is the top-tire conference in Data Mining.

plantc commented 1 year ago

KDD certainly is the best publication venue in data mining research. This community is unique, active and not dead! Data mining tends to focus more on unsupervsied learning challenges that have strong relationships to practical applications Compared to other research communities, like e.g, NLP data mining has always been underrepresented in CSRankings - so this must be at least corrected again. Claudia Plant, current area chair of KDD

KeSyren commented 1 year ago

Stanford U is building a new building for data science. I cannot accept KDD is being removed from CSRankings.

strongaccept commented 1 year ago

KDD is the best CS conference I have attended. It has significant impact in data science, machine learning, and many interdisciplinary subjects. It definitely should be included by default in CSRanking.

Romanticone commented 1 year ago

@emeryberger It has been 5 months since the KDD was removed from the CSRankings list. It seems the CSRankings eliminates the entire data mining community now. I do not think it is fair and right.

kno10 commented 1 year ago

Well, csrankings was never "fair and right", but elitist in a way that is harmful to the community, as it causes certain authors to concentrate completely on a few overrun venues.

ninghaohello commented 10 months ago

This is not the final state of affairs. Please be patient.

@emeryberger May I know if there is any update on this issue? Your team works hard in tackling all types of issues and we appreciate that. But what about this big one.