Closed carrickr closed 5 years ago
davidschober
commented
6 years ago @Toputnal can you check the isilon sync. I'm hoping these weren't uploaded and deleted in the same day. @carrickr all he should need to do is run the name against the isilon sync correct?
Toputnal
commented
6 years ago @carrickr did the first entry get partially lost in a cut/paste or is lunchroom_manners_512kb.mp4 the first file?
carrickr
commented
6 years ago @Toputnal
First one is lunch room managers, the r781wg00k is the pid
also I just updated the opening list, found a few more
egspoony
commented
6 years ago Farchive21 create a folder here. Data might be in farchive21. File or directory names might have spaces and/or special characters. Some items might have been deleted within 24 hours and will be missing.
DONE looks like: File list manifest of both found and missing.
Toputnal
commented
5 years ago I wrote the following script which identified that all missing files exist on the Isilon under farchive21
ret_val=$? cd /repository/farchive21/dropbox cat ~/missing-files.txt | while read X; do find . -samefile $X > /dev/null 2>&1 if [ $ret_val -eq 0 ]; then echo "Found $X on the Isilon" else echo "$X NOT found" fi done
I have 'pinged' carrick to get the S3 path to which they should now go.
Toputnal
commented
5 years ago [root@awslync ~]# cat fixfiles.sh
ret_val=$? cd /repository/farchive21/dropbox cat ~/missing-files.txt | while read X; do find . -samefile $X > /dev/null 2>&1 if [ $ret_val -eq 0 ]; then echo "Found $X on the Isilon. Copying to S3" aws s3 cp $X s3://avalon-masterfiles-cfx3wj9/dropbox/$X else echo "$X NOT found" fi done
[root@awslync ~]# ./fixfiles.sh
Script is currently running. I'll post results here when finished....
These two I did by hand to test portions of the script above:
upload: ../repository/farchive21/dropbox/Course_Video/21546653770002441_noonek.mov to s3://avalon-masterfiles-cfx3wj9/dropbox/Course_Video/21546653770002441_noonek.mov
upload: ../repository/farchive21/dropbox/Course_Video/21421145580002441_inters_spaspa.mov to s3://avalon-masterfiles-cfx3wj9/dropbox/Course_Video/21421145580002441_inters_spaspa.mov
carrickr
commented
5 years ago Items have been found and uploading, I am beginning the process to load them into Avalon
Toputnal
commented
5 years ago I've found and uploaded every missing file back to its original location in S3. We ran into a snag with special characters like commas and spaces, but once that was worked out everything pushed up to S3 without issue.
carrickr
commented
5 years ago Up and present!
We are missing the following master files (473 missing out of 20750)