sort, store and modify all photos on `onedrive` so that all collaborators can see all photos

[NewGraphEnvironment]

https://github.com/NewGraphEnvironment/fpr/issues/32

For now we store report/map photos on github and push only photos tagged with _k_ and crossing_all to Github to reduce repo bloat. This makes collaboration difficult. Solution for this repo below.

We will now do no photo work (renaming, resizeing, rotating, etc) in the repo itself. All work will be done in onedrive with sync done after each modification

mac - uses rsync https://www.cherryservers.com/blog/how-to-use-rsync-on-linux-to-synchronize-local-and-remote-directories#:~:text=If%20the%20specified%20destination%20folder,the%20directory%20will%20be%20copied.

rsync –a SourceDirectory DestinationDirectory

rsync -av --delete ~/Library/CloudStorage/OneDrive-Personal/Projects/repo/fish_passage_peace_2023_reporting/data/photos/AI/test ~/Projects/repo/fish_passage_peace_2023_reporting/data/photos/

Flags -a, --archive archive mode implies recursive and preserves important file attributes such as file permissions, group and user ownership, modification times, symbolic links, etc.

-v, --verbose increase verbosity Just to show us what is happening

--delete delete extraneous files from destination dirs this could be considered risky because repo only photo work will be lost on syncs but allows us to operate so that onedrive is the main and the repo is the replica as we will now do no photo work in the repo.

windows

https://www.ubackup.com/synchronization/windows-sync-two-folders-command-line-5740.html

[NewGraphEnvironment]

here are some screenshots to show how this was tested on the mac.

before sync with repo files altered (not something we will do of course)

after sync with fresh change to onedrive file

[Mateo9569]

I got it to work. First I copied all the photos over to the repo using this command line call : robocopy C:\Users\matwi\OneDrive\Projects\repo\fish_passage_peace_2023_reporting\data\photos\photos_sorted\ C:\Users\matwi\Projects\repo\fish_passage_peace_2023_reporting\data\photos /e /copy:DAT

And then I tested. Photo below before sync:

Next I altered a photo in onedrive and added _test to end of file name. I then ran the following command line: robocopy C:\Users\matwi\OneDrive\Projects\repo\fish_passage_peace_2023_reporting\data\photos\photos_sorted\ C:\Users\matwi\Projects\repo\fish_passage_peace_2023_reporting\data\photos /mir

Photo after sync:

So it's the /mir call that is key to syncing.

[NewGraphEnvironment]

Not surprisingly onedrive seems to be glitching due to massive un-synced changes. The size of the diffs resulting from thousands of photo renameing and delete operations during repeated testing and delivery of fpr::fpr_photo_rename over the last week has exposed weaknesses in our ability to use this. Better sooner than later.

We will need to move to dropbox, aws or digital ocean. Dropbox is expenses but has intuitive user interfaces. We already use digital ocean for overall company file backups and remote servers but amazon is likely the cheapest and perhaps the most flexible solution. Being able to serve the photos directly to reports served on github would be a huge benefit. Storing any photos at all on Github is far from ideal. lfs for git could be an option but I have had major issues with that in the past so would like to avoid that option.

For now - renamed photos have been moved to new directories in onedrive/2023_data/{region}/photos. Lets try working from there for now to attempt to get the Phase 1 data submitted for both Peace and Skeena

local onedrive

cloud onedrive

[NewGraphEnvironment]

photos sorted using names from dff with fpr_photo_rename and duplicates removed including duplicate names of sets in different places.

Photos with fpr_photo_remove_dupes are in onedrive/projects/2023_data/peace/photos

photos moved from {onedrive}/projects/2023_data/peace/photos to {onedrive}/Projects/PSCIS/phase1_submissions/2023/peace with fpr_photo_paths_to_copy ready for PSCIS submission - Phase 1.

Note that new photos can be added to the directories for submission to PSCIS by usingfpr_photo_paths_to_copy to move from {onedrive}/projects/2023_data/peace/photos to {onedrive}/Projects/PSCIS/phase1_submissions/2023/peace once you have added the 'k' tag, a custom tag via the 'tag_add' param or by tagging with any of the identified tags in fpr::fpr_xref_photo_order. See ?fpr::fpr_xref_photo_order for info and run fpr::fpr_xref_photo_order for a visual

summary:

1. PSCIS ready (unless we add more) - {onedrive}/Projects/PSCIS/phase1_submissions/2023/peace
2. Sorted with duplicates removed - {onedrive}/projects/2023_data/peace/photos - -this will be copied to repo
3. All photos with no duplicates removed - {onedrive}/projects/repo/fish_passage_peace_2023_reporting/photos/sorted_all

[NewGraphEnvironment]

rsync -av --delete ~/Library/CloudStorage/OneDrive-Personal/Projects/2023_data/peace/photos ~/Projects/repo/fish_passage_peace_2023_reporting/data/

[NewGraphEnvironment]

need to build crossing_all.JPG photos on onedrive so that we don't erase them in the local repo when we run a sync

[lucy-schick]

Some things to note when using rsync on mac:

• you must have the OneDrive folder on your local device with the source/destination folder in it. To add a shared folder to "my files" look here
• When calling rsync, you must be outside of the source folder. An example from blackwater_2022_hep#1 on my machine looks this this, I am in my home directory: ~ rsync -av OneDrive/blackwater_2022_hep/data Projects/repo/blackwater_2022_hep/data

[NewGraphEnvironment]

When calling rsync, you must be outside of the source folder

or do we need to just call full (absolute or complete vs relative) path names (right-click + option + copy {directory location} as pathname) (ex. /Users/airvine/Library/CloudStorage/OneDrive-Personal/Projects/2023_data/skeena/fish - ie. home directory location is included in the path name that begins with a /)

[NewGraphEnvironment]

closing since this is happening