On file modification/creation times in 1.x

[dewiniaid]

Continuing a conversation on tonight's Twitch stream...

Conventional Filesystem Terminology In most cases of a real (not kOS) filesystem, the following are generally true:

• A file's creation time (ctime) represents when it was created at its current location.
• A file's modification time (mtime) represents the last time its contents were modified.

When copying a file from one filesystem to another, the destination file is created (and thus its creation time is updated to the time of the copy), but its contents aren't considered modified (thus it retains the source's modification time). This can and often does result in files that were last modified before they were created.

Generally, editing a file and saving it does not change its creation time, but overwriting it does.

Within kOS File times can be useful in some cases. The most useful for kOS would probably be seeing which of two different files is newer, or listing files in order of most- to least-recently modified. Thus, the exact semantics of a filetime can be irrelevant, other than "older = lower".

• For files on the archive, these can represent native (true) filesystem times.
• For files on any other volume, this can represent current (real) system time. This also means that copying files from a processor volume to the archive still maintains realistic file modification times.
• Copying files should follow the conventional semantics -- i.e. modification time copies as-is, creation time is set to the current timestamp.

What about Save/Load? From an in-game perspective, loading a save is travelling through time to the point where the save was made. This is no different than, say, reverting a virtual machine snapshot in a virtualization environment or using a filesystem that supports snapshots.

Thus, if file modification/creation time attributes are stored in the persistence file it should be able to restore them as-is, and there should be no ill effects for doing so.

[Dunbaratu]

This is the biggest problem: We can't make up our own ad-hoc changes to the filesystem time stamps that exist on the actual hard drive of the player's gaming computer.

When the file is on the archive, that means it is not stored as data fields of our own making inside persistence.sfs - that only happens for files we put on the local drive. When we get the timestamp of a file called "0:/foo.ks", that's an actual out-of-game real-world file, located here:

/wherever/you/installed/KerbalSpaceProgram/Ships/Script/foo.ks.

And that means if we ask for the timestamp of that file, we get the real-world time, not the in-game time. The real-world time doesn't pass at the same rate as in-game time (if you hit pause, the real clock doesn't pause). And the real-world time can't easily be rewound when loading a saved game (and shouldn't be - it would falsify the truth about the file on the native OS's filesystem - and at least on UNIX, can't be done by a normal permission user anyway (to populate falsified information about the ctime or mtime of a file would require a sudo.).)

Hypothetically we could somehow track our own version of the timestamps in the archive that exists as our own data in the saved game "i.e. here's a table mapping archive filenames to what we pretend their timestamps are according to the game's clock, since we can't use the filenames' real stat information for that."

Then there is the problem that the archive shares the Ships/Script files across all multiple saves. All your saved games share the same archive folder as each other. They'd each need a different notion of the timestamp of the archive files. Hypothetically you could keep a different idea of the timestamp of the archive files per saved game, that don't all agree with each other if you want to deal with that mess, but....

The biggest problem: The game is not in control of when files in the archive get created, moved, modified, or deleted because you can do all those things offline while the game isn't running, or, to confuse the game even more, while it is still running, using an outside program to do it.

In the end, this is what made us punt and say not to do timestamps on files - the problem had nothing to do with storing the data for the local volumes in the persistence.sfs file - it was about the timestamps of the archive, and all the problems mentioned above.

I tried covering all this during the stream, but Charter Internet conspired against me to cut off my sentences every time I tried explaining something that took more than 20 seconds to say - so in the end I had to just make a much more simplistic-sounding summary with hand-waving.

[dewiniaid]

My whole point was that we use real world timestamps for everything, archive or not, rather than in-universe time which has all the problems you mentioned. It solves all of those problems and still allows the general concept of comparing/sorting by timestamps (the only interesting operation most of the time).

On Fri, Jul 8, 2016, 5:37 PM Steven Mading notifications@github.com wrote:

This is the biggest problem: We can't make up our own ad-hoc changes to the filesystem time stamps that exist on the actual hard drive of the player's gaming computer.

When the file is on the archive, that means it is not stored as data fields of our own making inside persistence.sfs - that only happens for files we put on the local drive. When we get the timestamp of a file called "0:/foo.ks", that's an actual out-of-game real-world file, located here:

/wherever/you/installed/KerbalSpaceProgram/Ships/Script/foo.ks.

And that means if we ask for the timestamp of that file, we get the real-world time, not the in-game time. The real-world time doesn't pass at the same rate as in-game time (if you hit pause, the real clock doesn't pause). And the real-world time can't easily be rewound when loading a saved game (and shouldn't be - it would falsify the truth about the file on the native OS's filesystem - and at least on UNIX, can't be done by a normal permission user anyway (to populate falsified information about the ctime or mtime of a file would require a sudo.).

Hypothetically we could somehow track our own version of the timestamps in the archive that exists as our own data in the saved game "i.e. here's a table mapping archive filenames to what we pretend their timestamps are according to the game's clock, since we can't use the filenames' real stat information for that."

Then there is the problem that the archive shares the Ships/Script files across all multiple saves. All your saved games share the same archive folder as each other. They'd each need a different notion of the timestamp of the archive files. Hypothetically you could keep a different idea of the timestamp of the archive files per saved game, that don't all agree with each other if you want to deal with that mess, but....

The biggest problem: The game is not in control of when files in the archive get created, moved, modified, or deleted because you can do all those things offline while the game isn't running, or, to confuse the game even more, while it is still running, using an outside program to do it.

In the end, this is what made us punt and say not to do timestamps on files - the problem had nothing to do with storing the data for the local volumes in the persistence.sfs file - it was about the timestamps of the archive, and all the problems mentioned above.

I tried covering all this during the stream, but Charter Internet conspired against me to cut off my sentences every time I tried explaining something that took more than 20 seconds to say - so in the end I had to just make a much more simplistic-sounding summary with hand-waving.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/KSP-KOS/KOS/issues/1703#issuecomment-231503242, or mute the thread https://github.com/notifications/unsubscribe/ADPJpWlfDV2WkZii99K8muuSthOtCWRsks5qTu0zgaJpZM4JIfnx .

[Dunbaratu]

That sounds like precisely the opposite plan to what you talked about in the first post, where you talk about wanting to revert timestamps when you load saved games.

[dewiniaid]

That sounds like precisely the opposite plan to what you talked about in the first post, where you talk about wanting to revert timestamps when you load saved games.

Not quite. I was trying to establish that if we persist the timestamps of non-archive files, the side effect of that is that the timestamps on those files would revert upon loading a save game -- and this is (IMHO) both acceptable and correct behavior and thus not something that needs consideration in terms of development time. There was some talk about the implications of quickload/quicksave vs file times on the stream, which is the only reason I brought it up.

The "What about save/load?" section was meant solely to apply to non-archive files; archive files already get the desired behavior for free by virtue of being on a real (as far as we're concerned filesystem).

We can't make up our own ad-hoc changes to the filesystem time stamps that exist on the actual hard drive of the player's gaming computer.

Again assuming we're using real times: If a player copies a file from a non-archive volume to the archive volume, I see no harm in changing the modification (not creation!) time of the destination to match the origin. This semantic is also followed by other programs: Most tools for extracting compressed files will restore the original modification time as well.

This scenario is fairly uncommon, but could happen on e.g. a RT save with a probe logging data to a local file until it re-enters communication range.

I can't think of any other situation where we'd need to touch real FS modification times though.

[Dunbaratu]

and this is (IMHO) both acceptable and correct behavior

It allows the local timestamps to do a thing archive timestamps can't. (see below in my response to your claim about extracting compressed files). Whether this is wrong or not could be debatable.

I get the impression from the argument you're making that all you really care about for timestamps is for asking the question "is file X older than file Y", and you don't want to use them for anything else. (Thus the amount of time by which they differ doesn't matter to your needs. If it did then there'd be problems with using the out-of-game clock to timestamp in-game files - i.e. the game was paused, then resumed, and it looks like this file is now a few minutes old when in the game-clock no time has passed. If all you care about is which file is older when comparing two files, this isn't a problem because these "time jumps" only jump forward.)

If that's the only need you have, I'd be more comfortable with not exposing the false timestamp (false in the simulated world) to the game, and instead just providing a function "is_older" that takes two filenames and returns a Boolean. We can track the timestamp in the way you suggest, but not claim it counts as a real timestamp inside the Kerbal's universe.

Or, if we do expose the out-of-game timestamp, we do it as part of kuniverse, where 4th-wall breaking things are explicitly quarantined.

Most tools for extracting compressed files will restore the original modification time as well.

In Unix, tar can do this, but here's the clincher: Only if run under root privilege. It's baked into the OS security itself that normal permission users can't falsify timestamps on files by pretending they wrote to a file in the past when they're actually writing to it right now. (A file copy is writing to the new file (changing its mtime) right now.) The way tools like 'tar' "preserve" timestamps is in fact by writing the file first, which makes the OS give it a new mtime like always, but then at the end of the writing after they close the file they go back and falsify the timestamp on it and change it to a time in the past matching the original file's mtime. That last step, of retroactively altering the mtime of a file, is what can't be done under normal user permissions (it would make things harder for the sysadmin if tools to search for all new file changes returned bogus results because some user has lied about the mtime of their files).

KSP shouldn't be being run with root privilege. And users generally shouldn't be running under root privilege (even if they know the password and are the admin) until they need it. Most file manipulation operations will NOT be operating under this "it's okay to revert timestamps" mode. It's only when running something with root permission like a full system backup/restore, or a tar operation to move a directory to a new drive, that this sort of thing happens.

Oh, and since KSP is cross-platform, another thing that matters here is that UNIX filesystems have no such thing as a creation timestamp like you get in Windows, so the claim that this is a thing in the archive isn't reliably true. They have something called "ctime" but it means "change time" not "create time". There's 3 timestamps on a typical UNIX filesystem: atime - access time, the time of last reading. mtime - modification time - the time the content was most recently written to but it's not touched when merely the header metadata changed (i.e. changing ownership, or changing permissions doesn't change mtime), and ctime - change time - the time the file changed in any way, whether by content or by header metadata.