Modified date is changed when file is uploaded to AList

nZeus commented 1 year ago

Please make sure of the following things

[X] I have read the documentation. 我已经阅读了文档。
[X] I'm sure there are no duplicate issues or discussions. 我确定没有重复的issue或讨论。
[X] I'm sure it's due to AList and not something else(such as Dependencies or Operational). 我确定是AList的问题，而不是其他原因（例如依赖或操作）。
[X] I'm sure this issue is not fixed in the latest version. 我确定这个问题在最新版本中没有被修复。

AList Version / AList 版本

v3.24.0

Driver used / 使用的存储驱动

Local

Describe the bug / 问题描述

Every file that I upload to AList, gets a new Modified timestamp. It doesn't matter if I'm uploading via web interface or WebDav client, the timestamp is changed.

Original file:

It's important for me to preserve the original date 🙏

Reproduction / 复现链接

?

Config / 配置

TrueNAS Chart

Logs / 日志

No response

welcome[bot] commented 1 year ago

Thanks for opening your first issue here! Be sure to follow the issue template!

SeanHeuc commented 1 year ago

this will be addressed soon

nZeus commented 1 year ago

Hi @SeanHeuc ,

Thank you very much! Looking forward to it!

SeanHeuc commented 1 year ago

if u use webdav to upload, "X-OC-Mtime" and "X-OC-Ctime" is supported to sepcify updateTime and createTime. I will push MR to rclone to support these 2 headers natively later. (for now, u can use "fastmail" in rclone as a temporary solution) if u use web upload, there's no way to perserve the 2 dates. the recommended way for now is to mount local disk into alist, then use "copy" function to copy from local disk to remote, and if remote storage supports, the 2 dates will be applied during upload. (i tested baidu_netdisk and aliyun_open)

xhofe commented 1 year ago

Uploading on the web page also supports lastModified now.

nZeus commented 1 year ago

Could you please release the changes? 🙏

xhofe commented 1 year ago

There are still some issues to be fixed, and a new release will be shipped after the fix.

jiongxuan commented 10 months ago

There are still some issues to be fixed, and a new release will be shipped after the fix.

Hi. Has this problem been resolved? @xhofe @SeanHeuc

I have the same problem. The difference is that I used Synology's CloudSync WebDav to link AList + Alipan-Open. After synchronization, I found that the file modification date was modified to the synchronized date at that time, not the original date. Alist is the latest version (3.29.1).

Hint:

Using the same CloudSync + WebDav, but when synchronizing to Synology's local WebDav server, the file modification date is retained and is only modified when synchronized to Alist. Is it related to X-OC-Mtime?
Only uploads will be modified, while file downloads retain the original file modification date without problems

jiongxuan commented 10 months ago

the recommended way for now is to mount local disk into alist, then use "copy" function to copy from local disk to remote, and if remote storage supports, the 2 dates will be applied during upload. (i tested baidu_netdisk and aliyun_open)

@SeanHeuc Please tell me how to mount AList locally? Via rclone? Can you share the specific command?

I used Synology's File Station to mount the AList and put the files directly into it, but the problem still recurred.

SeanHeuc commented 10 months ago

the recommended way for now is to mount local disk into alist, then use "copy" function to copy from local disk to remote, and if remote storage supports, the 2 dates will be applied during upload. (i tested baidu_netdisk and aliyun_open)

@SeanHeuc Please tell me how to mount AList locally? Via rclone? Can you share the specific command?

I used Synology's File Station to mount the AList and put the files directly into it, but the problem still recurred.

I meant mount a local disk into alist, then use alist copy function to copy from local disk to cloud remote(aliyun). mount alist webdav into other system (synology,linux...) will not work.

jiongxuan commented 9 months ago

I meant mount a local disk into alist, then use alist copy function to copy from local disk to cloud remote(aliyun). mount alist webdav into other system (synology,linux...) will not work.

Thanks for the reply. @SeanHeuc

When I use local mount into alist and copy the file to alipan, the File Modified Date still changes to the date at the time of sync, not the original date. Attached are diagrams of the operation steps. Let's see if there's any problem.

@xhofe @SeanHeuc

jiongxuan commented 9 months ago

The following is the Log that was being copied and uploaded at that time, but no problem was found.

[GIN] 2023/12/29 - 16:59:30 | 200 |   91.652311ms |      172.30.0.1 | POST     "/api/fs/dirs"
[GIN] 2023/12/29 - 16:59:31 | 200 |    2.647145ms |      172.30.0.1 | POST     "/api/fs/copy"
2023/12/29 16:59:31 INFO worker execute task worker=0 task=DLhepzOxs3xxx
[GIN] 2023/12/29 - 16:59:31 | 200 |    3.614237ms |      172.30.0.1 | POST     "/api/fs/list"
[GIN] 2023/12/29 - 17:00:20 | 200 |     257.823µs |      172.30.0.1 | POST     "/api/fs/dirs"
[GIN] 2023/12/29 - 17:00:21 | 200 |     228.394µs |      172.30.0.1 | POST     "/api/fs/dirs"
[GIN] 2023/12/29 - 17:00:22 | 200 |   70.048754ms |      172.30.0.1 | POST     "/api/fs/dirs"
[GIN] 2023/12/29 - 17:01:16 | 200 |     264.472µs |      172.30.0.1 | POST     "/api/fs/list"
[GIN] 2023/12/29 - 17:01:17 | 200 |     230.593µs |      172.30.0.1 | GET      "/api/me"

SeanHeuc commented 9 months ago

I meant mount a local disk into alist, then use alist copy function to copy from local disk to cloud remote(aliyun). mount alist webdav into other system (synology,linux...) will not work.

Thanks for the reply. @SeanHeuc

When I use local mount into alist and copy the file to alipan, the File Modified Date still changes to the date at the time of sync, not the original date. Attached are diagrams of the operation steps. Let's see if there's any problem.

@xhofe @SeanHeuc

from the screenshot i guess you are running alist on an ARM decive? my previous test is done on Mac and windows only.

alist need to use go sdk to get the file modification & create time, different OS have diff implementation, the GO SDK support is not very good/clear on this
the remote side, aliyun/baiduCloud need to use the timestamp alist put in the request. not all cloud providers support them, and their support can change over time

you can:

clone the soure, add some log output for this time parameters
build your own binary. deploy and check the log. And you can diagnose by yourself. Im busy these few months, I propabably will check this when i am back. I actually use the upload function a lot, but im only checking the file size &name in backup for now. it will be nice to have timestamp to do incremental sync.

jiongxuan commented 9 months ago

from the screenshot i guess you are running alist on an ARM decive? my previous test is done on Mac and windows only.

alist need to use go sdk to get the file modification & create time, different OS have diff implementation, the GO SDK support is not very good/clear on this

the remote side, aliyun/baiduCloud need to use the timestamp alist put in the request. not all cloud providers support them, and their support can change over time

you can:

clone the soure, add some log output for this time parameters

build your own binary. deploy and check the log. And you can diagnose by yourself. Im busy these few months, I propabably will check this when i am back. I actually use the upload function a lot, but im only checking the file size &name in backup for now. it will be nice to have timestamp to do incremental sync.

OK, thanks for the guidance. @SeanHeuc

My NAS uses X86 architecture and runs on Synology Linux system, but I tried to configure it in Windows and the same situation happened.

I will take some time to write down the log later and communicate in a timely manner if I find any new findings. Thanks

jiongxuan commented 9 months ago

from the screenshot i guess you are running alist on an ARM decive? my previous test is done on Mac and windows only.

alist need to use go sdk to get the file modification & create time, different OS have diff implementation, the GO SDK support is not very good/clear on this

the remote side, aliyun/baiduCloud need to use the timestamp alist put in the request. not all cloud providers support them, and their support can change over time

you can:

clone the soure, add some log output for this time parameters

build your own binary. deploy and check the log. And you can diagnose by yourself. Im busy these few months, I propabably will check this when i am back. I actually use the upload function a lot, but im only checking the file size &name in backup for now. it will be nice to have timestamp to do incremental sync.

Hi, @SeanHeuc

I executed the rclone command on Windows to synchronize, but the file modification date was still changed to the "synchronization date".

After consulting Alipan's customer service and said: “Alipan does not support retaining the original file date” and there is no plan to do so. It can be understood that this is a limitation of Alipan.

If you have time recently, you can also check the latest file date. The file modification date should no longer be retained.

(It is recommended to add this restriction to the documentation of "Alipan Open")

jiongxuan commented 9 months ago

Note: rclone sync will determine the file modification date. If Alipan does not support retaining the date, it will be re-uploaded for each synchronization, or use --size-only to determine only the size. Of course, if the file size is the same but the content is different, more serious problems will occur.

Therefore, currently using AList + Alipan Open to synchronize files may cause serious problems with repeated uploads (the problem may not be in AList, but in Alipan), so it is more suitable for WebDAV mounting.

@SeanHeuc You can take a look, maybe the file has been re-uploaded

AlistGo / alist