'git-crypt status' with the '-f' option to stage an encrypted version fails for me?

TheBigBear commented 8 years ago

git-crypt status tells me about some "staged/comited" files that are NOT encrypted, but should be.

And then tells me to run "Run 'git-crypt status' with the '-f' option to stage an encrypted version."

not encrypted: .git-crypt/.gitattributes
not encrypted: .gitattributes
not encrypted: .gitignore

<cut some lines>

not encrypted: etc/config/uhttpd
    encrypted: etc/config/wireless *** WARNING: staged/committed version is NOT ENCRYPTED! ***
not encrypted: etc/dnsmasq.conf
    encrypted: etc/dropbear/dropbear_dss_host_key *** WARNING: staged/committed version is NOT ENCRYPTED! ***
    encrypted: etc/dropbear/dropbear_rsa_host_key *** WARNING: staged/committed version is NOT ENCRYPTED! ***
not encrypted: etc/firewall.user

<cut some lines>

    encrypted: etc/shadow *** WARNING: staged/committed version is NOT ENCRYPTED! ***
not encrypted: etc/shells
not encrypted: etc/sysctl.conf
not encrypted: etc/sysupgrade.conf
    encrypted: etc/uhttpd.crt *** WARNING: staged/committed version is NOT ENCRYPTED! ***
    encrypted: etc/uhttpd.key *** WARNING: staged/committed version is NOT ENCRYPTED! ***

Warning: one or more files is marked for encryption via .gitattributes but
was staged and/or committed before the .gitattributes file was in effect.
Run 'git-crypt status' with the '-f' option to stage an encrypted version.

git-crypt status -f

it returns with:

Error: etc/config/wireless: still unencrypted even after staging
Error: etc/dropbear/dropbear_dss_host_key: still unencrypted even after staging
Error: etc/dropbear/dropbear_rsa_host_key: still unencrypted even after staging
Error: etc/shadow: still unencrypted even after staging
Error: etc/uhttpd.crt: still unencrypted even after staging
Error: etc/uhttpd.key: still unencrypted even after staging
Unable to stage 6 files.

What now? How do I fix this?

AGWA commented 8 years ago

This should not happen. What version of Git are you running?

TheBigBear commented 8 years ago

OK, I got past that now. But now can't remember what I did in the past to avoid this.

OK, so first how I got 'past it'? If I do a git-crypt init in this repo I can then do a `git-crypt status -fand that now does encrypt the staged files that it should encrypt according to the .gitattributes file. So far so good, but this now created a new symmetric key in this repo in.git/git-crypt/keys/default``, which is not wrong per se, but not what I have for my other encrypted git repos.

for all the other git-crypt repos the 'default' key is same as on the upstream master.

I can't remember what I used to do to create a new clone, but I do remember what I did this time.

I used simple git clone myupstream-master-repo-with-git-crypt my-new-slave-repo.

So for some reason all my other 'slave' repos do have the same 'default' key but the two I created today do not? This is likely NOT a issue with the prodcut but an issue with MY MEMORY. I jsut can't remember what I used to do differently, to clone rpeos and have them use the same git-crypt key as the upstream master? I used to be bback on 0.4 and have upgraded to 0.5 before doing the two repos today, but surely that would not have this effect, right?

my upstream master git-crypt repo

dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551  OpenWrt-AP-Configs/.git/git-crypt/keys/default

shasum -a 256 OpenWrt-AP-Config*/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551  OpenWrt-AP-Config_2Lv-dn/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551  OpenWrt-AP-Config_2Lv-up/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551  OpenWrt-AP-Config_3Lv/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551  OpenWrt-AP-Config_4Lv/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551  OpenWrt-AP-Config_Bungalow/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551  OpenWrt-AP-Config_Flat1/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551  OpenWrt-AP-Config_Flat5/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551  OpenWrt-AP-Config_Flat6/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551  OpenWrt-AP-Config_Flat7/.git/git-crypt/keys/default
dbb61775dc7255d2a5bc1e87fda958da59efbe29280a3bbdc3798187dea31551  OpenWrt-AP-Config_Test_AP/.git/git-crypt/keys/default

are all using the same git-crypt default key but todays repos are not, because I jsut typed git-crypt init in them.

ba7079c92061c44ee1e919cf210d217abfdb2b37bd3ca09f042afdb0bffb5e7e  OpenWrt-AP-Config_Flat2b/.git/git-crypt/keys/default
8cff463861522bd2023adc604c661c03864bca75db380b4d591ea7153f3f90cf  OpenWrt-AP-Config_Flat4/.git/git-crypt/keys/default

TheBigBear commented 8 years ago

@AGWA git version "2.3.8 (Apple Git-58)" and git-crypt version 0.50

TheBigBear commented 8 years ago

@AGWA sorry the hashing of .git/git-crypt/keys/default above makes absolutely no sense. ignore it, I hadn't noticed that the default file is a directory.

But I now have a problem and don't know how to reverse things back to a stable git repo in my upstream master git repo.

If I try any git reset -hard HEAD or a git reset -hard some-commit-sha

then it gives me a series of:

git-crypt: error: encrypted file has been tampered with!
error: external filter "git-crypt" smudge failed 1
error: external filter "git-crypt" smudge failed
fatal: etc/config/wireless: smudge filter git-crypt failed

Help, how do I get back and resume working using git-crypt ?

figtrap commented 7 years ago

Same problem. Users key was added with git-crypt add-user, user cloned repo, added file which matched the filter, the file was pushed up to remote unencrypted.

git-crypt-status says:

$ git-crypt status
not encrypted: .git-crypt/.gitattributes
not encrypted: .git-crypt/keys/default/0/6171E0D3EEFAAB1253838C9A883985B9345C8EE2.gpg
not encrypted: .git-crypt/keys/default/0/78211BF6AA2FAA2AC717FADA846CB77A2F46D007.gpg
not encrypted: .git-crypt/keys/default/0/ACBA87D39C7E935A0858F7DF6538A56FC91FAD09.gpg
not encrypted: .git-crypt/keys/default/0/B035C493C480B1B523D301C67B3549A7371F5324.gpg
not encrypted: .gitattributes
not encrypted: TEST.md
    encrypted: a_secret.sls
not encrypted: contributors.txt
    encrypted: foobar-secret.sls *** WARNING: staged/committed version is NOT ENCRYPTED! ***
    encrypted: foo/secret_things.sls
    encrypted: phony.key
Warning: one or more files is marked for encryption via .gitattributes but
was staged and/or committed before the .gitattributes file was in effect.
Run 'git-crypt status' with the '-f' option to stage an encrypted version.

git version 2.7.4 git-crypt .5

andrewhowdencom commented 7 years ago

Additional +1 to this problem. Starting investigation, will edit this comment as I figure out what's what. To start,

$ git version
git version 2.11.0

$ git-crypt version
git-crypt 0.5.0

$ lsb_release  -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 9.0 (stretch)
Release:    9.0
Codename:   stretch

Update 12:06pm: It's affecting multiple files. Unsure why. Update 12.08pm: It's the same new file in a new checkout. Update: 12.11pm: Looks like in the new checkout, it fails when git-crypt hasn't' been unlocked yet. Update: 12.14pm: Yeah, looks like that was the problem in the old repo also. Running the command

$ git-crypt unlock

restored the previous transparent behaviour of encrypting resources.

Suggest as a resolution you refuse to commit encrypted files if git-crypt is locked? Also, suggest that you warn somehow during the commit process if you cannot encrypt files? Both @figtrap and I managed to push an unencrypted resource accidentally (doesn't matter for me, as I just burn the credentials and it gets caught in review, but would be an easy one for a new person not to check)

transitive-bullshit commented 7 years ago

I've also run into this exact issue and it resulted in accidentally pushing unencrypted secrets.

aboritskiy commented 6 years ago

hmm for me steps that are suggested by @andrewhowdencom don not work, can not get one file staged and encrypted.

What I've done:

git crypt init
git crypt add-gpg-user
vim .gitattributes (add 3 files there)
git add (those 3 files)
git commit
git crypt status (shows 2 files encrypted and 1 not encrypted)
tried reverting the commit with git reset HEAD~1 and git crypt status -f broken-file didn't help
tried applying that in a separate commit - didn't help
tried locking and unlocking the repo and then committing again - didn't help

git version 2.7.4
git-crypt 0.5.0

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.4 LTS
Release:    16.04
Codename:   xenial

gpg --list-keys
/home/aboritskiy/.gnupg/pubring.gpg
-----------------------------------
pub   4096R/26179899 2017-04-12 [expires: 2033-04-XX]
uid                  Anton Bortitskiy <anton.boritskiy@gmail.com>
uid                  Anton Bortitskiy <tonybo@mail.ru>
sub   4096R/E5D8E6FD 2017-04-12 [expires: 2033-04-XX]

aboritskiy commented 6 years ago

ok figured it out, the file which git crypt was unable to encrypt was referenced in two .gitattributes files on different levels of the folder structure. as soon as I fixed that - git crypt happily encrypted the file.

ipatch commented 6 years ago

@AGWA, Not sure why this issue is still open, and I noticed there hasn't been any code activity in this repo since ~ November of 2017. That said, I am firm believer of "if ain't broke don't fix it™️".

@TheBigBear, I've only recently started using git-crypt in some of my public repos on GitHub so I am no expert by any stretch, but I'd like to think I have a working "workflow" for how to maintain sanity 👩🏾‍⚕️ when using git-crypt with a public repo.

@aboritskiy, for you guys n gals the git-crypt two step 🕺 goes as follows,

Obviously the working repo needs to be initialized using git-crypt with the below command.
```
cd /path/to/secret/sauce/git/repo
git-crypt init
```
Create a .gitattributes file within the repo root.
```
touch .gitattributes
```
🚨 Now this is where things become highly opinionated.
Open the .gitattributes file with preferred text editor.
```
nvim .gitattributes
```
Add the below lines to the .gitattribues file
```
*.shu filter=git-crypt diff=git-crypt
```
From my experience the above line should recursively encrypt all files throughout the repo that have an extension with .shu.
Save and close the .gitattribues file.
Add, move, copy files with .shu extension to repo
Verify files are encrypted

git-crypt status | grep -i "shu"

7b. Files can be force encrypted with the below command

git-crypt status -f

The above command will print error messages if there is any issue with staging or committing the encrypted files.

Stage, commit, push files as required

git add --all
git commit -m "🤫"
git push

⚠️ See gotcha section if and when running into issues.

Gotchas

Staging a whitelisted file in a locked repo state

If a file is created, while git-crypt has a repo in a locked state, and even if the file is whitelisted in the .gitattribues to be encrypted, the file will not be encrypted when staging an "untracked" file in a git repo. From my personal experience I run into the below issue.

To get around such mentioned issue, make certain the repo isn't in a "locked state" when adding a new file that conforms to the .gitattributes file for files being encrypted with git-crypt. So before adding a new .shu file to the repo make certain the repo is in an unlocked state.

git-crypt unlock

Then stage the file to be committed.

git add --all
git commit -m "add some secret sauce 🍔 to the repo

Optional, verify the newly added and committed file has been encrypted git-crypt status | grep -i "shu"

⚠ The repo still should probably be in a locked state before pushing files to a public repo, so one should lock the repo. The present fish prompt I'm using will display dirty working tree even if all files have been added and committed, that said to alleviate the dirty working tree prompt, lock the repo using git-crypt.
git-crypt lock

Now one can push the encrypted files to a public git repository.

git push

Optional one can verify the files have been encrypted by trying to view the file on the public repo.

Updating white listed encrypted files

The repo will need to be placed back in an unencrypted state in order to modify / update file that has been encrypted using git-crypt because the file locally will be in an encrypted state, which can be verified by opening a encrypted file in a text editor.

To put the repo back in a state where files can be updated

git-crypt unlock

Then, update white listed encrypted files accordingly.

🚨 To the best of my knowledge, the repo can remain in an unlocked state if the .gitattribues file has not be modified, and the updates to the white listed encrypted files can be added, committed, and pushed to a public repo, and should remain encrypted on the repo, while the repo is in unlocked state locally. Thus, my reasoning for adding a .shu extension to files that should be white listed for encryption, ie. only one rule needs to be added to the .gitattribues file.

Homework

[ ] write a git hook to unencrypt .shu files, copy the secrect.sause.shu file to secret.sause, ie. without the .shu extension within the same directory and add and verify the secret.sauce file is in the global .gitignore file for the repo.

mkmuto commented 6 years ago

I ran into a similar issue. How it happened was a certain type of file was pushed first, then .gitattributes was updated for that file type (should have been in reverse order). However, after I did "git-crypt lock" the repo and then unlocked it, the file was no longer listed as " WARNING: staged/committed version is NOT ENCRYPTED! " file by "git crypt status".

kingbuzzman commented 5 years ago

@ipatch following your instructions to the letter. Still not working for me. I either get:

$ git-crypt status -f
Error: a9docs-pybackend/.env.integr: still unencrypted even after staging
Unable to stage 1 file.

When its too darn late and it's game over. or....

$ git-crypt status -f; echo $?
0

and does nothing.

@mkmuto i see this same behavior, do you have any updates?

ipatch commented 5 years ago

@kingbuzzman

Yeah the workflow is clunky to say the least and could definitely be improved with using some form a git hook to auto encrypt / decrypt secret sauce 🍔 files. That said, I'd create a completely new repo all follow the steps I outlined in a test / experiment repo and see if you're still experiencing the same issues.

woopstar commented 5 years ago

+1

moreiraandre commented 1 year ago

Solved for me:

git filter-branch
git-crypt unlock <dir for your key>
git-crypt lock -k <dir for your key>
git-crypt status -e

jtara1 commented 3 months ago

fixed it by fumbling around

rm -rf .secrets/
git commit -am "chore: setup"
git-crypt unlock
git-crypt lock
git reset --hard HEAD~1 # or to where you last updated .gitattributes
mkdir .secrets
echo 'A=1' > .secrets/.env
git add .secrets/.env
git commit -am "chore: testing .env"
git-crypt status -f
# no errors or warnings
git push origin main
# verify your secrets are encrypted by viewing elsewhere

I'm using a GPG (asymmetric) key

AGWA / git-crypt