agrc / palletjack

A library for updating AGOL data from various external sources
https://agrc.github.io/palletjack/palletjack/
MIT License
12 stars 0 forks source link

Get latest file from ftp #2

Open jacobdadams opened 2 years ago

jacobdadams commented 2 years ago

From @steveoh:

Two other use cases we've had for this are, download all the csv's in a directory and download this specific file. It would be nice to provide some metadata about the file from the sftp as well since one of those use cases stored the modified date and would compare it to the last run to ensure it wasn't loading the same data twice.

remote_mtime = sftp.stat(file_path).st_mtime

if remote_mtime > mtime:
    self.log.debug('downloading {}'.format(file_path))
    sftp.get(file_path, local_path, preserve_mtime=True)

    return True