Feature Request: Download files only if they have been updated/modified

[jmealo]

It'd be helpful to have an option to download files only if they have been updated/modified.

[twolfson]

This seems like a reasonable idea. To achieve this, it looks like we would need to:

• Start saving Last-Modified headers to files as metadata
• Attempt to read file before making request
• Send up If-Modified-Since header with Last-Modifier metadata
• If it was a 304, take no action

http://www.cyberciti.biz/faq/linux-unix-curl-if-modified-since-command-linux-example/

I believe there is also E-Tag. Also, we would need to be aware of Cache-Control headers.

It is probably best to look for a library to handle all of this for us x_x

[hpurmann]

I think we should have a look at grunt-sync

[twolfson]

Yep, we should be able to use the mtime property they refer to. This can be altered via

http://nodejs.org/api/fs.html#fs_fs_utimes_path_atime_mtime_callback

For reference, this is the same property rsync uses in its last-modified implementation

http://unixhelp.ed.ac.uk/CGI/man-cgi?rsync+1

[twolfson]

Considered extended file system attributes to support etag and such

https://github.com/LinusU/fs-xattr

but I am realizing that is a bad idea because that mutates the original file. We should probably create a secondary file that stores all of this data

// .grunt-curl.cache.json
{
  filepath: {
    etag: 'abc'
  }
}

...but then we get into an issue of potentially having stale data if deletion manually occurs =/

[twolfson]

The second half of that idea was inspired by this library which creates double files for each of them which can be ignored by OSX:

https://github.com/rolftimmermans/node-xattr-file