[SR-16102] SwiftSyntax tree visitation exhausts stack space in background thread

[swift-ci]

Previous ID	SR-16102
Radar	None
Original Reporter	weak (JIRA User)
Type	Bug

Attachment: Download

Environment

See attached .ips

Additional Detail from JIRA

| | | |------------------|-----------------| |Votes | 0 | |Component/s | SwiftSyntax | |Labels | Bug | |Assignee | None | |Priority | Medium | md5: 2a9aabfc959b265ab4e2f662e74979ca

Issue Description:

See attached .ips or take a look at realm/SwiftLint/issues/3935.

To reproduce, please rename 'SwiftSyntaxSample.zip.txt' to 'SwiftSyntaxSample.zip', open and run

swift run swiftlint --config ./.swiftlint.yml --path ./

For some reason it's not possible to attach any kind of archive...

[ahoppen]

rdar://95600013

[ahoppen]

For anyone hitting this, https://github.com/apple/swift-format/issues/357#issuecomment-1121238414 contains a workaround to create a Thread with regular stack size:

let work = DispatchWorkItem {
    doFormatting()
}
let thread = Thread {
    work.perform()
}
thread.stackSize = 8 << 20 // 8 MB.
thread.start()
work.wait()

[rdingman]

I'm also hitting this stack overflow problem. I'd love to see a solution to this so I can use Swift Concurrency to process/lint source files in background tasks.

[ahoppen]

Did you try the solution mentioned above of creating a new thread with more stack space?

[rdingman]

Yes, that's a great workaround. Long term, I'd prefer to not have to do my own thread management and have a solution where I can just use Swift Concurrency. Perhaps a non-recursive walking strategy or one that consumes less stack space. Or, if Swift Concurrency provided a way to increase the stack space of the background threads in its thread pool.

[ahoppen]

I think that would be a request for Swift Concurrency to be able to specify the stack size. SwiftSyntax will continue to use recursion and there's no way to avoid the stack usage for nested data structures.

If you are hitting the stack overflow in SwiftParser, an alternative workaround would be to pass the maximumNestingLevel parameter when constructing Parser. This will cause the parser to fail with an error once that nesting level has been reached without crashing. My experience has shown that 25 is sufficiently low to avoid hitting stack overflows on Dispatch background Threads. I assume that Swift Concurrency background Threads have the same stack size.