USENIX-Security-2025 / conference-format

14 stars 0 forks source link

Using Artifact Evaluation in long talk decision #8

Open lambdafu opened 2 months ago

lambdafu commented 2 months ago

One factor used in deciding for a long talk (or increasing the chance in a lottery) could be the result of artifact evaluation. Preparing and reviewing artifacts is a lot of work, but also very valuable for the research community. This seems to be an easy way to reward that work.

phani-vadrevu commented 2 weeks ago

Having served as the AEC co-chair for this conference for the past couple of cycles, I can add my two cents here. First, in the AEC, we try to give as many badges as the authors applied for as possible. We keep the reviewing process very bilateral and discussion-based as our main goal is to help the authors improve their artifacts and get all the badges that they applied for. In fact, one of our papers in the last cycle had more than 100+ comments and 50+ file uploads as part of the back and forth between reviewers and authors.

So, if we use artifact evaluation as a factor for deciding long talk slots, I can imagine that this might stimulate a large number of papers to apply for these badges and eventually get them. This of course, is a great thing for our community, but it only means that we still need to include a "luck" factor for deciding long-talk papers.

Second, my only small worry is about papers which might have legitimate reasons for not applying for the AE badges (due to licensing restrictions etc.). These papers will miss out on long-talk slots unless they are distinguished papers.