Open hermands opened 9 years ago
At Tuesday lab meeting, it was decided to start moving things to the SFS.
Currently moving data to /net/emc/projects/dan_herman_project/FCS_Files/ @ 35 MB/sec
I found out that SCP is the bottleneck. We can transfer multiple folders with multiple instances to SCP to fill the pipes. Another option is using bbcp
brew install bbcp
However this will also need to be installed in the remote machine (Yeti). Perhaps talk with Noah or Ed?
Sounds pretty slow. Hmm...If we want it installed on yeti, I would talk with Ed/Sheena/Tyler. Is there another built-in protocol that would be faster? rsync is the same issue? You are connected directly to the switch?
rsync has similar issues, I've open up multiple scp instances and I"m getting 80 MB/sec which is the close the max drive speed or 1GBPS speed. So we will see.
Sounds good
Data is now on /net/emc. Plan is to wait until shared file system is stable. Then delete USB drive data, reformat, attach to larf/yeti. Copy data over to new USB drive.
Need a direct rsync from SCCA and yeti to clean up.
Plan: (1) Wait until Jerry has made full 64 TB virtual (2) Wait until Ed/Tyler have built directory structure and permission scripts (3) Hook up USB/laptop to switch, transfer data to /net (4) Attach drive to larf/yeti
! make sure stable! Format drive (6) Copy data from /net to drive
DO STUFF!