google / renameio

Package renameio provides a way to atomically create or replace a file or symbolic link.
Apache License 2.0
609 stars 27 forks source link

Allow user to control tempfile names (plus: weird interaction with ioutil.TempFile) #9

Closed cespare closed 5 years ago

cespare commented 6 years ago

I'm replacing some code with renameio and it relies on knowing that tempfiles are named xxx.tmp. On startup it can delete any .tmp files knowing that they were left over from an interrupted rename. It would be nice to have a way to name my files a similar way with renameio.

With ioutil.TempFile, if I use a pattern of foo.txt the tempfile will be named like foo.txt12345789. (renameio also adds a . prefix, so with renameio.TempFile("", "foo.txt") I would get .foo.txt12345789.) However, ioutil.TempFile has a feature for controlling the name:

The filename is generated by taking pattern and adding a random string to the end. If pattern includes a "*", the random string replaces the last "*".

So in my code I could use a TempFile pattern of foo.txt-*.tmp and get temp files like foo.txt-123456789.tmp. However, renameio is unaware of this ioutil.TempFile feature, so if I do renameio.WriteFile("", "foo.txt-*.tmp"), then renameio will create a target file named foo.txt-*.tmp after the rename.

Separately, but relatedly, I would really prefer that my tempfiles not be named with a leading .. For most of my applications, having them generate hidden files is not what I want.

Here are some preliminary ideas that fix one or more of these issues.


Adding parallel APIs

func WriteFile2(dest, pattern string) string
func TempFileNamed(dir, path, pattern string) (*PendingFile, error)

This would let the user control the pattern that is passed to ioutil.TempFile. (So I could use renameio.TempFileNamed("", "foo.txt", "foo.txt-*.tmp").)

(This would obviously need better names.)


Handling * patterns explicitly

If the user passed * which is handled specially by ioutil.TempFile anyway, let's assume they do not want a file named with *. At minimum, we could just remove the * from the final filename (option 1), or we could do something more, such as removing the * and anything afterwards (option 2).

So foo*.tmp would become:

temp name final name
current .foo123456789.tmp foo*.tmp
option 1 .foo123456789.tmp foo.tmp
option 2 .foo123456789.tmp foo

Additionally, we could imagine dropping the leading . if the user provided a path containing *, on the presumption that they want more control over the tempfile name.

cespare commented 6 years ago

Or, to propose something a bit more out there, we could double down on the "special syntax" approach and add more stuff on top of the * that ioutil.TempFile provides: if path contains [...], that indicates a part of the name that is only present in the temp name. So path of foo[-*.tmp] yields a tempfile pattern of foo-*.tmp (which ioutil.TempFile uses to make a name like foo-123456789.tmp), and the final target filename is foo.

With this scheme, the default name may be expressed as "[.]" + path.

stapelberg commented 6 years ago

Thanks for filing this issue.

I think the best path forward is to add the parallel APIs. I’d suggest TempFileWithPattern and WriteFileWithPattern as names.

I don’t like adding more special syntax to the filename, as then callers have to take special care to pass a filename which reflects their intent (and doesn’t accidentally contain []).

Would you be interested in sending a PR to add the APIs?

cespare commented 5 years ago

For our purposes, it suffices to just change the naming scheme from .foo.txt12345789 to foo-12345789.tmp. It also turns out that we don't need some of the renameio API (Symlink, TempDir) so our internal library is only a few dozen lines of code. I'm content to just keep our package as-is instead of trying to make renameio more general.

Thanks for the great package and API ideas!