Incomplete standard output

[nicholascross]

I have come across a scenario when not all standard output is returned. This seems to be happening on a non zero exit code, successful termination may or may not be affected.

Background I was using shellOut to execute fastlane and it was failing for version requirements not being read.

Investigation I did some tinkering and it seems to be related to how this function reads the output.

private extension Process {
    @discardableResult func launchBash(with command: String, outputHandle: FileHandle? = nil, errorHandle: FileHandle? = nil) throws -> String {
        launchPath = "/bin/bash"
        arguments = ["-c", command]

        var outputData = Data()
        var errorData = Data()

        let outputPipe = Pipe()
        standardOutput = outputPipe

        let errorPipe = Pipe()
        standardError = errorPipe

        #if !os(Linux)
        outputPipe.fileHandleForReading.readabilityHandler = { handler in
            let data = handler.availableData
            outputData.append(data)
            outputHandle?.write(data)
        }

        errorPipe.fileHandleForReading.readabilityHandler = { handler in
            let data = handler.availableData
            errorData.append(data)
            errorHandle?.write(data)
        }
        #endif

        launch()

        #if os(Linux)
        outputData = outputPipe.fileHandleForReading.readDataToEndOfFile()
        errorData = errorPipe.fileHandleForReading.readDataToEndOfFile()
        #endif

        waitUntilExit()

        outputHandle?.closeFile()
        errorHandle?.closeFile()

        #if !os(Linux)
        outputPipe.fileHandleForReading.readabilityHandler = nil
        errorPipe.fileHandleForReading.readabilityHandler = nil
        #endif

        if terminationStatus != 0 {
            throw ShellOutError(
                terminationStatus: terminationStatus,
                errorData: errorData,
                outputData: outputData
            )
        }

        return outputData.shellOutput()
    }
}

This is the output when using the default implementation

[19:08:21]: -------------------------------------------------
[19:08:21]: --- Step: Verifying required fastlane version ---
[19:08:21]: -------------------------------------------------

I noticed the linux only variation and thought I would try it out on macos, I hacked the function as follows.

private extension Process {
    @discardableResult func launchBash(with command: String, outputHandle: FileHandle? = nil, errorHandle: FileHandle? = nil) throws -> String {
        launchPath = "/bin/bash"
        arguments = ["-c", command]

        var outputData = Data()
        var errorData = Data()

        let outputPipe = Pipe()
        standardOutput = outputPipe

        let errorPipe = Pipe()
        standardError = errorPipe

//        #if !os(Linux)
//        outputPipe.fileHandleForReading.readabilityHandler = { handler in
//            let data = handler.availableData
//            outputData.append(data)
//            outputHandle?.write(data)
//        }

//        errorPipe.fileHandleForReading.readabilityHandler = { handler in
//            let data = handler.availableData
//            errorData.append(data)
//            errorHandle?.write(data)
//        }
//        #endif

        launch()

//        #if os(Linux)
        outputData = outputPipe.fileHandleForReading.readDataToEndOfFile()
        errorData = errorPipe.fileHandleForReading.readDataToEndOfFile()
//        #endif

        waitUntilExit()

        outputHandle?.closeFile()
        errorHandle?.closeFile()

        #if !os(Linux)
        outputPipe.fileHandleForReading.readabilityHandler = nil
        errorPipe.fileHandleForReading.readabilityHandler = nil
        #endif

        if terminationStatus != 0 {
            throw ShellOutError(
                terminationStatus: terminationStatus,
                errorData: errorData,
                outputData: outputData
            )
        }

        return outputData.shellOutput()
    }
}

Which produces the following output

[19:04:50]: -------------------------------------------------
[19:04:50]: --- Step: Verifying required fastlane version ---
[19:04:50]: -------------------------------------------------

[!] The Fastfile requires a fastlane version of >= 2.47.0. You are on 1.91.0. Please update using `sudo gem update fastlane`.

Assuming I am not using this wrong it looks like the output is getting cut off early.

[nicholascross]

I was able to create a small example that can reproduce the problem.

ShellOutput.zip

The following output will not be observed using the default launchBash implementation but will with the hacked version (and when run from terminal).

[!] The Fastfile requires a fastlane version of >= 5000. You are on 2.5.0.

[JohnSundell]

@nicholascross Awesome, thanks for this! I'll look into this within the next couple of days if no one else beats me to it 👍