Parallelize communication with SourceKit to reduce analysis time

[Ockey12]

Currently, the flow of communication using SourceKitClient is as follows:

1. Convert a single source file into an abstract syntax tree using SwiftSyntax.
2. Traversing the abstract syntax tree to extract data objects of interest within a file.
3. Call SourceKitClient serially for each data object extracted from a single file.

[Ockey12]

The process flow needs to be changed as follows:

1. Finish scanning all files and extracting data objects.
2. Execute requests using for await on data objects.

[Ockey12]

I would like to keep the sendCursorInfoRequest currently provided by SourceKitClient as a way to make a single synchronous request.

[Ockey12]

First, modify DeclarationExtractor

[Ockey12]

Currently, DeclarationExtractor.extractDeclarations() operates on one source file.

Implement a method that takes all the files as input and executes the request to SourceKit in bulk.

[Ockey12]

DeclarationExtractor.extractDeclarations() is called in RAGESSReducer as follows:

func extractDeclarations(
    allSourceFiles: [SourceFile],
    buildSettings: [String: String],
    packages: [PackageObject]
) async -> [any DeclarationObject] {
    var declarationObjects: [any DeclarationObject] = []
    let allSourceFilePaths = allSourceFiles.map { $0.path }
    let extractor = DeclarationExtractor()

    for sourceFile in allSourceFiles {
        let declarations = await extractor.extractDeclarations(
            from: sourceFile,
            buildSettings: buildSettings,
            sourceFilePaths: allSourceFilePaths,
            packages: packages
        )

        declarationObjects.append(contentsOf: declarations)
    }

    return declarationObjects
}

Therefore, changes to the caller are likely to be small.

[Ockey12]

A concrete KeyPath cannot be spliced ​​onto an abstract KeyPath.

Therefore, all objects that store data extracted from the source code must be of the same type.

The kind of the extracted type can be determined by the enumeration property rather than the type of the data object.

Below is the code I used to check in the Playground whether the KeyPath could be appended to the modified data object.

struct S {
    let name: String
    var note: String
    var a: [Self]
    var b: [Self]

    init(name: String, a: [Self], b: [Self]) {
        self.name = name
        self.note = name
        self.a = a
        self.b = b
    }
}

let child = S(name: "child", a: [], b: [])
let parent = S(name: "parent", a: [], b: [child])
var root = S(name: "root", a: [parent], b: [])

let parentKeyPath = \S.a[0]
let childKeyPath = \S.b[0]
let keyPathForRoot = parentKeyPath.appending(path: childKeyPath)

root[keyPath: keyPathForRoot].note = "new note"
print(root[keyPath: keyPathForRoot].note)
// "new note"