RP2: error in reported file creation time

[peterhinch]

If a file is copied to RP2 it is stored with the correct time: i.e. the creation time reads correctly in Python code running on the RP2.

However, if the PC's local time != UTC, the ls -l command reports an incorrect time. UK time is currently UTC + 1. If the time as reported on the PC and the RP2 is 08:00 when the file is copied, the creation time as reported by ls -l is 09:00 (UTC + 2). This fault prevents rsync from working correctly.

This can be fixed by changing this line to read:

        self.adjust_for_timezone = (epoch_tuple[0] != 1970) or (self.sysname == "rp2")

I am not entirely sure whether this is a valid fix or a hack, hence an issue rather than a PR.

[schauveau]

This is an annoying issue. I am in France at GMT+1 so I see that too.

My current workaround for syncing my pico on Linux is to execute rshell via a script that set the timezone to GMT by setting the environment variable TZ="GMT" . Be aware that this won't fix the timestamps that are already incorrect so you may have to manually remove or 'touch' the offending files.

TZ also seems to exist on Windows and MacOS so that trick could work there too.

[gmos]

It is not properly defined what the RTC on the device should run, either UTC or localtime.

When there is no RTC, the rshell sets the device time when connecting and uses (tried on Windows) localtime. So the time zone offset is already incorporated into the device time. MicroPython doesn't implement time zoning at all, so if rshell sets the clock to localtime all file times will be in localtime too.

However, when rshell stats the files on the device, it assumes they are in UTC and applies the TZ offset again. So only when you are on UTC, like UK in wintertime everything will work fine.

I think the best solution would be:

• fix the bug in rshell. It should at least be consistent and eiher use localtime or UTC everywhere.
• include a --tz parameter which would:
  • when not specified, use localtime as set on the host (for upwards compatibility)
  • when specified without an offset use UTC
  • when specified with an offset, use the offset.

Remains what to do with boards having a built-in RTC. Leave it alone and perhaps use it to guess the offset. But what to do when it seems completely wrong? Or just always set the RTC? ...

[bablokb]

I think this is a misunderstanding on how timestamps are stored in the filesystem: they are always stored as UTC and the programs that display the timestamps have to do the translation (or better: interpretation) of the timestamps. Just run the stat command on any local file and you will see what how it works: you get a timestamp that is in UTC but your computer will show you the correct time.

So rshell is correct. If you copy with rshell and then use the "ls -l" command from rshell, you will see correct times. This is because the rshell cp command sets the creation-time to the local time of the device (in fact it is not rshell but the MicroPython firmware running on the device). Later, the "ls -l" command corrects the timestamp knowing that the host OS will undo this correction again.

[schauveau]

@bablokb The timestamps on the filesystem are effectively stored in UTC but the real issue is that the rshell is systematically resetting the time of the RP2 using time.localtime() so the time using the host timezone.

https://github.com/dhylands/rshell/blob/b4e402ef44d51313c4a4056590b487b5ed740c4c/rshell/main.py#L1688 https://github.com/dhylands/rshell/blob/b4e402ef44d51313c4a4056590b487b5ed740c4c/rshell/main.py#L1075

As we can see in the second link, the rp2 is a special case where the provided date and time values are directly written at some "magic memory addresses" to configure a system clock. Simply speaking, the «UTC time» of the rp2 is set to the local time of the host and so are the timestamps written in the filesystem.

The modification proposed by @peterhinch works because it enables a timestamp offset correction in stat and lstat on the rp2 so that they become real UTC which is necessary for proper rsync behavior. This is already implemented in master so the issue could be marked as resolved. https://github.com/dhylands/rshell/blob/b4e402ef44d51313c4a4056590b487b5ed740c4c/rshell/main.py#L1075

However, one could argue that this is a strange way to fix the problem. It would makes more sense to set the rpi2 system clock to UTC and to also configure the timezone on the rpi2 so that time.localtime() remains correct for scripts that rely on the local time.