USENIX-Security-2025 / conference-format

15 stars 0 forks source link

Building more lanes usually makes congestion worse... #5

Open evansuva opened 4 months ago

evansuva commented 4 months ago

Thank you for making this discussion public, and all the work you are doing for the conference and considering its future more broadly.

I largely agree with Manuel's comments in https://github.com/USENIX-Security-2025/conference-format/issues/2, though, that the proposed changes don't start from a clear understanding and statement of the problem to be solved.

Without starting from the underlying problems, we end up with patches to fix immediate symptoms of the problem, and often the result of these patches is to make the problems worse not better (cf: we should learn from the experience over the past ~10 years with all the patches done to the review process, which were motivated by good intentions and mostly done for good reasons, but ultimately didn't solve the problems they were targeting and mostly made things worse - but good to see many of those changes are now finally being reversed).

Any understanding of the problems with our current conference model need to start from the dual-nature of the purpose of these conferences:

  1. To allocate "cookies" that are valuable and important for advancing individual authors' careers.
  2. To provide an event for the community to gather to celebrate successes, share ideas, build connections, and consider the futility of trying to make computing systems secure against adversaries who are willing to spend 3 years on social engineering to build trust to embed a backdoor in a widely use utility maintained by a volunteer hobbyist.

We suffer because a single vehicle is used for both purposes, and most of the changes motivated by purpose number 1 end up having negative impacts on purpose number 2 (what is listed as motivation 1, and the subtext of the other three motivations, largely reflects this). A historical goal of these conferences was to communicate research, but that is largely irrelevant now - the research has been communicated through arxiv, twitter, talks, etc., long before the conference, and the visibility a paper gets at conferences like these is limited.

Our main problem with the cookies, is that we have, over the years, tried to allocate more cookies, motivated by the frustrations people have not getting cookies they think they (and mostly their students) deserve. This has make the cookies less valuable, and increased the pressure people feel to obtain more of them. Many people think to be a credible candidate for the desirable post-PhD positions, a student needs to be averaging around 1 cookie every 3-6 months (to see where things can go off the rails, in ML community, the cookies are smaller, so students need to get > 1 cookie/month to be credible for some positions).

Anything done to increase the number of cookies just reduces the value of each cookie and increases the expectations for how many cookies people try to get --- the way to reduce congestion isn't to build more highway lanes, it is to remove parking spots. In the short run, of course, voters/drivers are happy when more highway lanes are built and complain vociferously if parking becomes harder (and in the short run, this increases congestion with people driving around looking for parking); in the longer run, though, more lanes doesn't solve the congestion problems and less parking does.

Of course, we don't want our students to starve, so if there are less cookies available, we need to provide more and better opportunities to get biscuits and ginger snaps, and maybe even seek ways to offer healthier snack options!

The solution to number 2 is to start by divorcing the cookie-allocation issues from the community value of the conference issues, and focus on them separately. Now that we are in a world where presenting a paper at a conference is not the main, or even a useful in most cases, way for an idea to become visible in the community, we don't need to intertwine the two purposes so closely in ways that make it unlikely to really improve either.

tgianko commented 4 months ago

Thank you for sharing your thought on the RFC. We added a new FAQ item expanding more on the problem that we are trying to solve.

evansuva commented 4 months ago

Thank you for sharing your thought on the RFC. We added a new FAQ item expanding more on the problem that we are trying to solve.

Reading what is there, the problem is there are too many accepted papers - if that's the actual problem this is intended to solve, the solution is to accept fewer papers. As mentioned well in #2, this is totally under control of the conference organizers (and directly under control of the PC chairs), so doesn't seem to be a sufficient motivation for the changes proposed here.