HPC | Encryption Script

NDCLab / lab-devOps

NDCLab mgmt and operations

GNU Affero General Public License v3.0

0 stars 0 forks source link

HPC | Encryption Script #217

Closed jessb0t closed 1 year ago

jessb0t commented 1 year ago

Revert to send e-mail for every unencrypted file.
Script should check recursively for any unencrypted file in sourcedata/raw and sourcedata/checked.
Update project lead: casper-dataset (ostib001) memory-for-error-dataset (khoss005) mfe-online-dataset (khoss005) missing-link-dataset (jalexand) oops-faces-dataset (emart459) putt-putt-dataset (smaly002) rwe-dataset (jalexand) rwe-eeg-dataset (jalexand) readAloud-valence-dataset (jalexand) social-context-dataset (emach039) social-flanker-eeg-dataset (apoly003) thrive-dataset (gbuzzell)

jessb0t commented 1 year ago

@davhunt As George mentioned on our call, it would also be really helpful to have some kind of back-up/escalation system to notify the lab manager, in addition to the project lead. Please let me know what you think would be feasible/elegant to implement! :pray:

jessb0t commented 1 year ago

Current action: research option to search zipped files (without unzipping) to find digitization data. May be two levels of zip. (Example dataset: memory-for-error-dataset.)

Working note (not yet ready to deploy): should only throw errors for:

audio files (.mp4, .wav)
video files (.mp3)
digitization/image files (model.jpg, model.obj, model.mtl -- even if within a zip)

davhunt commented 1 year ago

Also seeing some .m4a files

davhunt commented 1 year ago

So verify-encryption.sh should now:

Search in raw/zoom, raw/audio, raw/video, checked, case insensitive for unencrypted identifiable files (.mp4, .wav, .mp3, model files, .m4a)
Notify lab manager about week-old unencrypted files
Search two levels of zipped folders for unencrypted files
Run on a daily crontab

jessb0t commented 1 year ago

3/27/2023: limit to "gpg: encrypted" (email for all other message), check if possible to simplify to lumping all unencrypted into one category without explicitly mentioning error message types

jessb0t commented 1 year ago

Definitely unencrypted:

Email researcher, then escalated to lab manager after one try.

This file is unencrypted. It must be encrypted immediately. Please contact the lab manager once corrected to confirm.

Maybe unencrypted (else statement):

Email lab tech and lab manager together with researcher.

This file is unencrypted. It must be encrypted immediately. Please contact the lab manager once corrected to confirm. If the file appears to be properly encrypted, contact the lab manager.

jessb0t commented 1 year ago

If something unencrypted but no project lead specified in config-leads.json:

immediate notification to lab tech and lab manager that an unencrypted file exists in a repo and there is no project lead to notify