no exception when opening an existing file for output

[larceny-trac-import]

_Reported by: will on Thu Feb 7 10:34:42 2008 _ The R6RS says we should raise an exception when opening a file for output and the file already exists.

This is incompatible with Larceny's historical behavior, however, so it creates problems for legacy code. What to do?

Will thinks Larceny should raise an exception in R6RS mode but not in R5RS mode. ERR5RS mode should probably issue a warning instead of raising an exception.

[larceny-trac-import]

Author: will Larceny supports only the three file options mandated by the R6RS, which have effect only when a file is opened for output. The affected procedures are:

    open-file-output-port           ; calls file-io/open-file-output-port
    open-file-input/output-port     ; calls file-io/open-file-input/output-port
    call-with-output-file           ; calls open-output-file
    with-output-to-file             ; calls call-with-output-file
    open-output-file                ; calls open-file-output-port with default options

So it all comes down to

    file-io/open-file-output-port          ; in fileio.sch
    file-io/open-file-input/output-port    ; currently defined in portio.sch

With respect to open-file-input/output-port, I believe the R6RS requires us to raise an exception if the file doesn't already exist (because it is being opened as an input file). Unless the no-create or no-fail option is specified, the R6RS requires us to raise an exception if the file already exists (because it is being opened as an output file). If neither no-create nor no-fail option is specified, therefore, we have to raise an exception in either case, and the only reason for us to check whether the file exists is to determine what kind of exception to raise.

More later...

[larceny-trac-import]

Author: will The three standard file options provide 8 possible combinations of options. Three language standards (IEEE/ANSI/R5RS, ERR5RS, R6RS) have led to 3 different execution modes, with differing expectations and rules. Opening a file for output isn't quite the same as opening a file for combined input/output.

That works out to 48 different contexts in which we have to interpret the file options when opening a file for output or combined input/output. That does not even begin to consider the external variables, such as whether the file already exists.

The following tables are based upon my understanding of the R6RS and other relevant standards. They offer an outline of what Larceny should do when opening a file for output or combined input/output. Where the R6RS semantics do not conflict with Larceny's legacy semantics, the tables below follow the R6RS semantics even in ERR5RS and R5RS modes.

Opening a file for output only:

                    file already exists     file does not exist

no options          R6RS: &i/o exception    create file
(the default)       ERR5RS: warn&truncate
                    R5RS: truncate file

no-create           truncate file           &i/o exception

no-fail             truncate file           create file

no-truncate         &i/o exception          create file

no-create           truncate file           &i/o exception (?!)
no-fail

no-create           no truncation [1]       &i/o exception
no-truncate

no-fail             no truncation [1]       create file
no-truncate

no-create           no truncation [1]       &i/o exception (?!)
no-fail
no-truncate

Opening a file for combined input/output:

                    file already exists     file does not exist

no options          R6RS: &i/o exception    &i/o exception
(the default)       ERR5RS: warn&truncate
                    R5RS: truncate file

no-create           truncate file           &i/o exception

no-fail             truncate file           &i/o exception (?!)

no-truncate         &i/o exception          &i/o exception

no-create           truncate file           &i/o exception (?!)
no-fail

no-create           no truncation [1]       &i/o exception
no-truncate

no-fail             no truncation [1]       &i/o exception
no-truncate

no-create           no truncation [1]       &i/o exception (?!)
no-fail
no-truncate

Note that, in several situations, the R6RS apparently insists upon raising an exception even though the no-fail option has been specified.

Opening a file with the no-truncate option by itself appears to be equivalent to opening the file with the default options.

Opening a file with both the no-create and no-fail options appears to be equivalent to opening the file with the no-create option alone.

Opening a file with all three standard options (no-create, no-fail, and no-truncate) appears to be equivalent to opening the file with just the no-create and no-truncate options.

---

Note 1: Overwriting an existing file with no truncation makes sense for binary files, and it makes sense for textual files provided the encoding is fixed-width and the none end-of-line convention is used. With the R6RS io system, however, a file is opened first as binary and may then be transcoded as textual.

Will can't think of any reasonable way for Larceny to prevent programmers from opening a file for output without truncation and then writing to it as a textual file using a transcoder that may result in encoding errors at the end of the newly written data.

---

At present, Larceny's low-level io system is unable to open files for output without truncation, or for combined input/output at all.

One short-term solution is to proceed as follows when a file is opened for output with no truncation:

• Create a custom output port that does the following.
• Open the file again for reading, read its entire contents into a bytevector, and then close that port.
• Open the file for normal output, with truncation, and write the bytevector's entire contents into the file. Then keep that output port open until the custom output port is closed. (This provides whatever locking would be obtained by opening the file for output with no truncation, leaving only a couple of brief intervals in which something could go wrong. The first interval, after the file is read but before it is opened for normal output, is fairly harmless. My apologies to Douglas Adams.)
• Create a bytevector input/output port.
• When data are written to the custom port, just write the data into the bytevector input/output port.
• When the custom port is closed for real (but not by transcoded-port):
  • Close the output port that's been locking the file.
  • Hope nobody else opens the file during this brief interval.
  • Open the file as an ordinary output file.
  • Write the bytevector port's contents to the file.
  • Close the output file.

Larceny is already using a similar technique to implement combined input/output files.

[larceny-trac-import]

Author: will For Unix machines, this was fixed by changeset:5733.

Fixing this for Windows should be a matter of making the same changes to sys-win32.sch that were made to sys-unix.sch, but I'm holding off on that to avoid possible interactions with ticket #584.

It appears that sys-macos.sch is needed only for MacOS 9, which we no longer test and is not supported by setup.sch. Maybe it's time to deprecate sys-macos.sch.

[larceny-trac-import]

Author: will Will thinks this was mostly fixed for Windows by changeset:5770. The no-truncate option doesn't work on Windows, however, because there doesn't appear to be any reasonable way to implement that option using portable C io. We'll have to use Windows-specific io, which is a delicate change because it may interact with the conditional compilation set up by our build process.

[larceny-trac-import]

Author: will The problem has been fixed except for some quality-of-implementation issues.

Ticket #625 records the no-truncate problem for Windows.