Bash script for automatic upload

ArchiveTeam / grab-site

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

Other

1.35k stars 134 forks source link

Bash script for automatic upload #171

Closed raspher closed 3 years ago

raspher commented 3 years ago

Uploading files by hand is a mistake, this script checks for X secounds in Y folder for WARC file and will upload it to IA. All unneeded files (probably unneeded) are deleted and not uploaded. WARC file will be removed even if there was an upload error, this should be corrected.

I am not a bash programmer, but a working student. Please forgive me my mistakes.

ivan commented 3 years ago

just FYI, I have an uploader in https://gist.github.com/ivan/079530350ac94851d581b55b1d372440 that you might find useful.