elasticdog / transcrypt

transparently encrypt files within a git repository
MIT License
1.43k stars 102 forks source link

Transcrypt can fail to run in Jenkins when Git hooks are disabled: `mkdir: can't create directory '/dev/null': File exists` #153

Closed paomeow closed 1 year ago

paomeow commented 1 year ago

Newer version of transcrypt does not work on containers for our DevOps automated deployment. transcrypt -c aes-256-cbc -p 'xxxxxxxxxxxxx' --yes The result is always the error below: mkdir: can't create directory '/dev/null': File exists

jmurty commented 1 year ago

Hi @paomeow I haven't been able to reproduce this problem. Can you provide more information about your containers, such as the operating system version and version of bash?

After checking code changes since version 2.1.0 I have made a guess about something that might cause a problem, if a version of bash interprets things differently than on my system. Could you try the Transcrypt version in commit bbce281 to see if the change helps?

velvetant commented 1 year ago

I have the same error occuring when cloning a repository from jenkins then trying to decrypt, not in a container though, but on Manjaro. That fix in the PR did not work for me. Is there any way to get more verbose information from transcrypt to know where exactly it fails?

transcrypt 2.2.3

jmurty commented 1 year ago

Hi @velvetant unfortunately transcrypt doesn't have a verbose mode. You could try Git trace options like these but I'm not sure how much they will tell you.

The most likely problem is the Manjaro system missing a requirement as outlined in the README

The best way to debug an issue is probably to run the underlying commands that transcrypt does, but manually and one step at a time to see if you get any useful error feedback. This will be difficult to do in a Jenkins test environment, but maybe you can spin up an equivalent system that you can interact with?

If you can, start by installing/configuring transcrypt in the repo as usual.

Here are the commands to run just the smudge (decrypt) operation on a file called _sensitivefile:

Show raw encrypted file contents
# git show :sensitive_file

Decrypt file contents using smudge operation (git_smudge function in Transcrypt)
# git show :sensitive_file | ./transcrypt smudge

If that doesn't give you any clues you could try running the underlying openssl command run by the git_smudge function directly in the terminal.

This would look something like:

# git show :sensitive_file | ENC_PASS="correct horse battery staple" openssl enc -d -aes-256-cbc -md MD5 -pass env:ENC_PASS -a
velvetant commented 1 year ago

Hi @jmurty - thanks for your quick support! I tried all of your steps above and while git show :sensitive_file | ./transcrypt smudge had no output, git show :sensitive_file | ENC_PASS="correct horse battery staple" openssl enc -d -aes-256-cbc -md MD5 -pass env:ENC_PASS -a worked like it should. No error messages at all though :/

Oh, funnily enough I just tried with another user on that system and it worked. I need to find out what is different. Edit: scratch that, it works when I clone with another user, but not with the "jenkins"-cloned repo (which I then chowned to another user). So I presume it has something to do with the way jenkins clones and detaches the git repo?

jmurty commented 1 year ago

You can see in the git_smudge function that there isn't much else happening within transcrypt: there's some temporary file management, a few git config lookups, then the openssl command.

https://github.com/elasticdog/transcrypt/blob/154218afb6ec5ad4e706a81b8aff5425714778ff/transcrypt#L178-L185

Try running each of the commands on the problem system and user environment, and hopefully one of the steps will give an error with some actionable feedback.

I suppose it's also possible there's an error somewhere in the transcrypt script before it calls that function. To debug that you'd need to edit transcrypt to have print statements at key points to try and find where it's failing, especially around the section at the bottom of the file near the comment # parse command line options where the git_smudge function is invoked.

velvetant commented 1 year ago

Dear @jmurty, thank you so much for the support and guiding me in debugging this, much appreciated. Turns out, in the end, it is not a transcrypt bug at all. The jenkins git plugin introduced a config change sometimes last year and disabled all hooks by setting hooksPath = /dev/null in the .git/config file for newly checked out repositories*. Tbh I have not enough knowledge of git's inner workings to understand why this is happening, but manually commenting out above line in the config fixes decryption. Apparently there is also a setting in jenkins to disable this. I need to learn more about the potential attacks because of this though.

Anyway, nothing to do from trancrypt's side I reckon?

Thank you again and more broadly thanks to all working on transcrypt, impressive script!

jmurty commented 1 year ago

Hi @velvetant what a weird issue, I bet that config change broke things in surprising ways for many people. Thanks for following up, and I agree there's nothing much for us to do on the transcrypt side.

jmurty commented 1 year ago

I've renamed this issue to make it easier for people to find, and now that you've found the root cause others will be able to fix it relatively quickly. Cheers!