beer4all / rclone

"rsync for cloud storage" - Google Drive, Amazon Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Cloudfiles, Google Cloud Storage, Yandex Files
https://rclone.org
MIT License
0 stars 0 forks source link

The time is not correct #7

Open kiderrien opened 4 years ago

kiderrien commented 4 years ago

Most fs / operations tests fail because of the same error

=== RUN   TestCount
    TestCount: run.go:176: Remote "root://eosctaatlaspps:1094//eos/ctaatlaspps/tmp/a/rclone-test-cekowow0boqijux2dohetet8", Local "Local file system at /tmp/userxroot/rclone463573885", Modify Window "1s"
    TestCount: fstest.go:123:
                Error Trace:    fstest.go:123
                                                        fstest.go:142
                                                        fstest.go:186
                                                        fstest.go:299
                                                        fstest.go:329
                                                        fstest.go:341
                                                        operations_test.go:304
                Error:          Should be true
                Test:           TestCount
                Messages:       empty space: Modification time difference too big |73874h56m39.876543211s| > 1s (2020-05-29 17:56:39 +0200 CEST vs 2011-12-25 12:59:59.123456789 +0000 UTC) (precision 1s)

The file takes the current date instead of the date of the file copy. This problem is probably related to the SetModTime function.

beer4duke commented 4 years ago

@kiderrien how did you trigger this error?

It looks like this is a rclone test suite, can you give the command to run this?

kiderrien commented 4 years ago

@beer4duke from rclone directory: cd fs/operations/ go test -v -remote remote:path This command will launch a series of tests proposed by rclone. We have another series of tests in fs/sync with the same command for the launched.

The problem is the date is not the same as the original file. For example if we make a copy the date of the copy will be the time when the file was copied instead of the date of the original file.

Normally the setmodTime function should do this. For example in the sftp and local function it is the line with chtimes that does this.

beer4duke commented 4 years ago

mtime must be set at the initial file copy: using eos.mtime=<time_epoch_second>.<nanosecond>

For example:

xrdcp /etc/passwd root://eosctaatlaspps//eos/ctaatlaspps/tmp/testmtime14?eos.mtime=1577833200.652283000

but this is currently broken and reported in: https://its.cern.ch/jira/browse/EOS-4286

kiderrien commented 4 years ago

It is possible to add an attribute with xrootd extended attributes. The idea would be looked at first if possible and how to use it with backend xrootd go. And after implementing an attribute over time during creation and adding an option to the remote xrootd to use the attribute (True or False)