./williams/TheisenWilliams.md

[timm]

Once reviewed by one person, please relabel to 2.ReviewTwo.

Once reviewed by a second person, please relabel to 3.EditorComments.

[pruneson46]

I am confused by the spamming. What am I expected to do and when?

/Per

---

Dr. Per Runeson, Professor in Software Engineering

Dept. of Computer Science LUND UNIVERSITY per.runeson@cs.lth.se
Box 118 phone +46 46 222 93 25 SE-221 00 Lund fax +46 46 13 10 21 Sweden http://cs.lth.se/per_runeson

Head of Department Computer Science http://cs.lth.se Group Leader Software Engineering Research Group http://serg.cs.lth.se @softengresgrp Research Director Industrial Excellence Center EASE http://ease.cs.lth.se

[timm]

dear pers,

we are using 2 free internet services (google groups and github) and they seem to send notifications more often than donald trump changes his hair piece

right now, you have no review assignments. they are coming next week

so sorry for the spam... but as always, a pleasure to talk to you

t

p.s. also, please see my email from today on how to reduce email storms from google groups and github.

[rvprasad]

Title of chapter

There’s Gold in Them Thar Stack Traces

URL to the chapter

https://github.com/ds4se/chapters/blob/master/williams/TheisenWilliams.md

Message?

Stack traces can help approximate the attack surface. [However, I am not sure if this was the intended message.]

Accessible?

Yes, it is accessible.

Size?

It is about the right length.

When I read the title, I thought it would tell me about potential interesting uses of stack traces to understand software. Instead, the chapter was focused on the use of stack traces to approximate attack surfaces. So, it would be better to either retarget the chapter to focus on approximate attack surfaces or revamp the chapter by including other recent interesting uses of stack trace (e.g., StackMine, STAT). Given the book is focused on data science for software engineering, the latter option might be better.

Gotta Mantra?

I didn't get the "Thar" part of the title. I looked it up and I was unclear if it referred to Thar desert or Thar the forgotten realms. So, how about just There's Gold in Them Stack Traces (provided the content is refocused onto stack traces)?

Best Points

The graph images and the concrete numbers from Windows and Mozilla drive the point home. Keep'em.

[timm]

Given that this is the only stack trace paper in the book, perhaps a little more info on other trace usages would be appropraite

i like the the idea that reasoning over stack dump traces is SCALABLE. as we said in our ASA proposal:

[tostrand]

Title of chapter

There’s Gold in Them Thar Stack Traces

URL to the chapter

https://github.com/ds4se/chapters/blob/master/williams/TheisenWilliams.md

Message?

The chapter seems to promote two distinct messages:

1. stack traces are useful for finding faults in code
2. stack traces can help to find the attack surface(s) of a system

The first paragraph describes how stack traces are useful for debugging. Then suddenly attack surfaces are brought in, and the focus of the chapter changes to security issues.

If security and attack surfaces are the main focus, then the title should indicate that.

Accessible?

I understand the purpose of Fig 1 is to illustrate the overall difficulty of understanding the structure of a complex system, but I think the description makes it even more mysterious than it has to be. What features of the system is this complex graph supposed to represent? Is it the static program structure? the calling relations? data flow? dynamic control flow trace?

The most interesting part of the Mozilla result is that 91.6% of the files did NOT appear in any stack trace, so I would emphasize that in addition to the 8.4% that did appear. Of course, it's also extremely useful to know that 72% of the vulnerabilities occurred in the 8.4%.

The sentence beginning "Researchers also explored other metrics ..." is tantalizing, but I'm left wondering what they did with those metrics. It would be nice to include a short description of their use.

One minor grammar fix(remove the comma, make verb agree with subject): Busy security professionals, can uses -> Busy security professionals can use

Gotta Mantra?

The chapter title is a play on an American phrase "there's gold in them thar hills". While this might be cute and catchy for an American, I think it's not not understandable by the general international SE audience, and will only generate confusion.

A couple of suggestions: Stack Traces Reveal Attack Surfaces Attack surfaces leave footprints in stack traces

Best Points

Figures 1 & 2 should be kept, but I'd like to read a description of what the connection structure represents.

[tzimmermsr]

A few more comments:

• The paper puts a lot of focus on "Researchers" in the beginning. This makes it sound like crashes only matter to researcher, but in this is not the case. Crashes have been analyzed for many years in industry, e.g., Debugging in the Very Large: Ten Years of Implementation and Experience (SOSP 2009) http://www.sigops.org/sosp/sosp09/papers/glerum-sosp09.pdf A simple fix is to avoid the use of the word "researchers" in the first paragraphs and also talk about some of the industrial applications (like the SOSP paper)
• What is OWASAP?
• Figure 2 is never discussed

[tzimmermsr]

@theisencr @lauriew Please prepare a new version of your paper by January 13 taking the reviewers' feedback into account.

[tzimmermsr]

@theisencr @lauriew I'm not sure if you are done with the revision, since the issue report is not marked as AuthorHasRevised.

The following two of my earlier comments are must fix:

• [x] What is OWASAP?
• [x] Mention Figure 2 when you discuss that you color the nodes red.

The following is a strongly recommended fix:

• [x] The paper puts a lot of focus on "Researchers" in the beginning. This makes it sound like crashes only matter to researcher, but in this is not the case. Crashes have been analyzed for many years in industry, e.g., Debugging in the Very Large: Ten Years of Implementation and Experience (SOSP 2009) http://www.sigops.org/sosp/sosp09/papers/glerum-sosp09.pdf A simple fix is to avoid the use of the word "researchers" in the first paragraphs and also talk about some of the industrial applications (like the SOSP paper)

[crtheisen]

I went ahead and addressed these comments; I'll leave final author approval to Laurie (or get her blessing in person to flip the switch).