Automate splitting and storage of 10Hz data

jsignell / campbellsci-tools

Tools for reading ascii files using the campbellsci TOA5 long header

3 stars 0 forks source link

Automate splitting and storage of 10Hz data #1

Open jsignell opened 8 years ago

jsignell commented 8 years ago

[x] rename file on lab computer when size exceeds ~2GB
[x] move file out of Cloudstation dir on server when there is more than one ts_data file in there
[x] split file into daily files with headers on each and bridging file boundaries
[x] push daily files to agnes.princeton.edu for permanent storage
[x] delete old file? (move to backup storage)

jsignell commented 8 years ago

Info on pushing files to agnes. From this SO answer.

Use ssh keypairs.

Karthik@A $: ssh-keygen    #keep the passphrase empty
Karthik@A $: ssh-copy-id B #enter your B password
#^ this will copy your public key to Karthik@B:.ssh/authorized_keys

From then on, you should be able to ssh from A to B (and by extension, scp from A to B) without a password.

jsignell commented 8 years ago

[ ] directory creation on agnes
[x] deleting files locally once they are pushed (side-stepped this by only pushing new files using rsync -u)