sync should only write to `latest.json` and `extension.json` if contents have changed

[kenyon]

While running rsync --archive to copy the artifacts directory, I noticed that every latest.json and extension.json has to be transferred after every run of sync.py --syncall. This is because these json files are being rewritten every time, even if their contents haven't changed, so rsync has to transfer their timestamps. It would be a nice enhancement to sync.py if it would avoid writing these json files when not needed.

I think this is happening in these functions:

• https://github.com/LOLINTERNETZ/vscodeoffline/blob/e40fa50316b086c4f91217f5696836d2d8bc01a6/vscoffline/sync.py#L135
• https://github.com/LOLINTERNETZ/vscodeoffline/blob/e40fa50316b086c4f91217f5696836d2d8bc01a6/vscoffline/sync.py#L271

[tomer953]

I think extension.json changes every run since they contains some stats (number of downloads for example) and some timestamps inside. also, latest.json is actually the latest extension.json file. (it is just another copy of it)

In my end we are uploading the entire folder each run, but based on the file checksums. so only files that their checksums not exists in the storage server are being uploaded.

[kenyon]

@tomer953 that was my other thought, but didn't see anything that would have been changing every run, but maybe I just didn't notice.

I thought using rsync's --no-times might help, but from the documentation, sounds like that would be worse, but I might have to experiment with it.

[LOLINTERNETZ]

I think this really depends on your deployment scenario and how you transfer data around. As @tomer953 said, the statistics change within the files along with a few timestamps. This is super useful if you're in a complete airgap, because you can check timestamps and get some confidence the sync processes are running or if something has died.

You could run rsync with --checksum too which might provide functionality in line with this issue.