First, thank you for opening this for public opinion!
Second, one concern that I have about random selection is potential topic bias for the finalized program. As we know, the number of submissions for certain topics (like anything related to AI) is much more than other topics. Think about this: We have 1,000 papers that are accepted and there are five papers on a certain topic (this is common for certain areas; for example, there are six smart contract papers in 2023's USENIX program). We want to choose 100 papers as the final program. Then, the probability of a program without any papers on that topic is (995 choose 100) / (1000 choose 100) = 58.98%. This is not a small probability. And it will become larger if we have more such small areas.
That said, given that many areas have fewer papers compared with others, we may expect that the final program of longer talks will be full of certain topics if the majority of papers are selected randomly. Some may argue that this reflects the reality of the community. But I feel that it will be an unhealthy program because it is hard to argue one topic is more important than the other. It would be nice to balance the program based on topics. Here are some possible solutions:
(1) Like CCS, we divide the conference into tracks and each track has a fixed amount of slots and its (random) selection mechanism,
(2) We have a topic weight for the selection of longer talks.
Third, another concern that I have for random selection is that even if we claim that the selection is random, we are still creating a tiered system. People outside our community may not know that this is random because other conferences (like AI conferences) are selecting oral presentations based on merits. Also, there is nothing preventing authors from indicating that this is an oral presentation on their CVs/websites. If by any means we are creating a tiered system, we (i.e., the security community) should choose oral papers ourselves.
First, thank you for opening this for public opinion!
Second, one concern that I have about random selection is potential topic bias for the finalized program. As we know, the number of submissions for certain topics (like anything related to AI) is much more than other topics. Think about this: We have 1,000 papers that are accepted and there are five papers on a certain topic (this is common for certain areas; for example, there are six smart contract papers in 2023's USENIX program). We want to choose 100 papers as the final program. Then, the probability of a program without any papers on that topic is (995 choose 100) / (1000 choose 100) = 58.98%. This is not a small probability. And it will become larger if we have more such small areas.
That said, given that many areas have fewer papers compared with others, we may expect that the final program of longer talks will be full of certain topics if the majority of papers are selected randomly. Some may argue that this reflects the reality of the community. But I feel that it will be an unhealthy program because it is hard to argue one topic is more important than the other. It would be nice to balance the program based on topics. Here are some possible solutions:
(1) Like CCS, we divide the conference into tracks and each track has a fixed amount of slots and its (random) selection mechanism,
(2) We have a topic weight for the selection of longer talks.
Third, another concern that I have for random selection is that even if we claim that the selection is random, we are still creating a tiered system. People outside our community may not know that this is random because other conferences (like AI conferences) are selecting oral presentations based on merits. Also, there is nothing preventing authors from indicating that this is an oral presentation on their CVs/websites. If by any means we are creating a tiered system, we (i.e., the security community) should choose oral papers ourselves.