Closed mpinjr closed 2 years ago
very interesting, thank you for extensive tests.
This patch is a big step in the right direction. The only thing I don't like is that you fflush all files, including those that are open for input, which is a meaningless operation. I don't know what effect that has on ferror. I suggest that both functions be more careful to only flush output files and pipes. Thanks.
I suggest that both functions be more careful to only flush output files and pipes.
Agreed. I'll make the changes, rerun the tests, and push (sometime in the next few hours).
printstat and awkprintf are very clear: print statement errors are fatal.
In Jan 2020, to prevent fatal print errors from masquerading as fclose warnings, #63 changed every WARNING in closefile and closeall to FATAL. This broke awk's close and getline functions.
close no longer returns if there's an error, unless the stream doesn't exist.
getline read errors still return -1, but they are no longer ignorable. Eventually, one of the closing functions will inspect the stream with ferror and call FATAL.
In Jul 2020, fatal stdout write errors which had been detectable by closefile for a few months became invisible, a consequence of #89 switching standard streams from fclose (which reports flush errors) to freopen (which ignores them). The earlier changes which broke getline and close were themselves partially broken.
The solution is to finish printing before closing. That is to flush and ferror every stream opened for writing before calling fclose, pclose, or freopen. A failure to write print statement data is fatal. A failure to close a flushed stream is a warning. They must be handled separately.
Every redirected print statement is finished in printstat or awkprintf.
The same is not true of unredirected print statements. To finish these, stdout must be flushed at some point after the final such statement. Any problem with that flush is fatal.
Though only stdout needs it, let's defensively finish every stream opened for writing, so this bug won't recur if someone changes how redirected streams are flushed.
Write errors on stderr by the implementation are never fatal. When closing, we only warn of them. Write errors from an application attempting a redirected print to /dev/stderr are as immediately fatal as every other redirected print statement.
These changes compiled cleanly and I didn't see any errors in
make check
.Below, I've included some test results along with the test script.
Each test is an awk one-liner sabotaged with strace.
The first line of a test case report consists of the syscall that was sabotaged and its program. Then, for each executable tested, there's a line with its return value and path. If this line begins with an asterisk, the executable failed the test.
Tests are paired, with one closing explicitly (closefile) and the other implicitly (closeall).
That test output was produced with this command and script:
Lastly, here's how the behavior of the test supplied in #63 has changed over the past couple of years. I've paired it with an explicitly closing counterpart. The correct answer for both is 2. Command line, script, and results follow.
If you made it all the way down here, I really appreciate it.
Take care, Miguel