update to 3.6.4 breaks git sync

[vsubram95]

git-sync v3.6.4 breaks the cloning process in our k8s pods with the issue mentioned. However, it works fine until version v3.6.3 .

Note:

1. We use submodules in our repos.
2. This is git-sync deployed as sidecar container in airflow via helm

12 main.go:535] "msg"="too many failures, aborting" "error"="Run(git rev-parse HEAD): exit status 128: { stdout: "", stderr: "fatal: detected dubious ownership in repository at '/dags'
To add an exception for this directory, call:

\tgit config --global --add safe.directory /dags" }" "failCount"=1

[thockin]

This looks like a fairly recent change in git. I will have to figure out what tickles this and add a test case.

On Mon, Mar 13, 2023, 4:41 PM vsubram95 @.***> wrote:

git-sync v3.6.4 breaks the cloning process in our k8s pods with the issue mentioned. However, it works fine until version v3.6.3 .

Note:

1. We use submodules in our repos.
2. This is git-sync deployed as sidecar container in airflow via helm

`E0313 04:24:55.774672 12 main.go:535] "msg"="too many failures, aborting" "error"="Run(git rev-parse HEAD): exit status 128: { stdout: "", stderr: "fatal: detected dubious ownership in repository at '/dags' To add an exception for this directory, call:

\tgit config --global --add safe.directory /dags" }" "failCount"=1`

— Reply to this email directly, view it on GitHub https://github.com/kubernetes/git-sync/issues/696, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKWAVC4GACZXHV7U3JZQ43W36WDNANCNFSM6AAAAAAVZXKEZ4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[thockin]

Ok, well the fix is pretty easy. I will try to prep a PR this week.

[thockin]

https://github.com/kubernetes/git-sync/releases/tag/v3.6.5

[vineetnegi009]

@thockin , i am using v3.6.5 tag of git-sync container and i am getting the error "msg"="error syncing repo, will retry" "error"="Run(git sparse-checkout init): exit status 128: { stdout: "", stderr: "fatal: detected dubious ownership in repository at '/tmp'\nTo add an exception for this directory, call:\n\n\tgit config --global --add safe.directory /tmp" }" "failCount"=25 "waitTime"=60000000000

is this problem related to the issue mentioned above ?

[thockin]

Can you show me:

1) Full logs with -v 6 2) Your full commandline and env vars (you can XXX the repo and stuff)

On Mon, Mar 20, 2023 at 6:52 AM vineet @.***> wrote:

@thockin https://github.com/thockin , i am using v3.6.5 tag of git-sync container and i am getting the error "msg"="error syncing repo, will retry" "error"="Run(git sparse-checkout init): exit status 128: { stdout: "", stderr: "fatal: detected dubious ownership in repository at '/tmp'\nTo add an exception for this directory, call:\n\n\tgit config --global --add safe.directory /tmp" }" "failCount"=25 "waitTime"=60000000000

is this problem related to the issue mentioned above ?

— Reply to this email directly, view it on GitHub https://github.com/kubernetes/git-sync/issues/696#issuecomment-1476274761, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKWAVBSCHHZG7VOK4MWCMTW5BOKFANCNFSM6AAAAAAVZXKEZ4 . You are receiving this because you were mentioned.Message ID: @.***>

[vineetnegi009]

here are the logs for the git-sync sidecar container:

I0321 06:40:40.730611 13 main.go:539] "level"=1 "msg"="syncing repo" I0321 06:40:40.730673 13 main.go:950] "level"=0 "msg"="cloning repo" "origin"="" "path"="/tmp" I0321 06:40:40.730687 13 main.go:952] "level"=5 "msg"="running command" "cwd"="" "cmd"="git clone -v --no-checkout -b master --depth 1 /tmp" I0321 06:40:40.732234 13 main.go:956] "level"=0 "msg"="git root exists and is not empty (previous crash?), cleaning up" "path"="/tmp" I0321 06:40:40.732307 13 main.go:623] "level"=2 "msg"="removing path recursively" "path"="/tmp/.git" "isDir"=true I0321 06:40:40.733731 13 main.go:964] "level"=5 "msg"="running command" "cwd"="" "cmd"="git clone -v --no-checkout -b master --depth 1 /tmp" I0321 06:40:42.125068 13 main.go:964] "level"=6 "msg"="command result" "stdout"="" "stderr"="Cloning into '/tmp'... Failed to add the RSA host key for IP address 'x.x.x.x' to the list of known hosts (/etc/git-secret/known_hosts)." I0321 06:40:42.125088 13 main.go:974] "level"=0 "msg"="configuring sparse checkout" I0321 06:40:42.125200 13 main.go:1006] "level"=5 "msg"="running command" "cwd"="/tmp" "cmd"="git sparse-checkout init" E0321 06:40:42.126918 13 main.go:551] "msg"="error syncing repo, will retry" "error"="Run(git sparse-checkout init): exit status 128: { stdout: "", stderr: "fatal: detected dubious ownership in repository at '/tmp'\nTo add an exception for this directory, call:\n\n\tgit config --global --add safe.directory /tmp" }" "failCount"=3 "waitTime"=60000000000

and the args for the sync container are:

• args:
  • -repo=
  • -branch=master
  • -depth=1
  • -max-sync-failures=-1
  • -git-config=core.filemode:false
  • -wait=60
  • -ssh=true
  • -v=6
  • -root=/tmp
  • -dest=dashboards
  • -sparse-checkout-file=/sparse-file securityContext: runAsUser: 65533

[thockin]

I think there are some logs missing from that listing? I ran this myself using 3.6.5 and I see:

$ /git-sync --repo https://github.com/kubernetes/git-sync --depth 1 --root /tmp -v 6

I0321 18:42:14.081921      19 main.go:401] "level"=0 "msg"="starting up" "pid"=19 "args"=["/git-sync","--repo","https://github.com/kubernetes/git-sync","--depth","1","--root","/tmp","-v","6"]
I0321 18:42:14.081973      19 main.go:1306] "level"=5 "msg"="running command" "cwd"="" "cmd"="git config --global gc.autoDetach false"
I0321 18:42:14.094982      19 main.go:1306] "level"=6 "msg"="command result" "stdout"="" "stderr"=""
I0321 18:42:14.095003      19 main.go:1306] "level"=5 "msg"="running command" "cwd"="" "cmd"="git config --global gc.pruneExpire now"
I0321 18:42:14.096869      19 main.go:1306] "level"=6 "msg"="command result" "stdout"="" "stderr"=""
I0321 18:42:14.096887      19 main.go:1306] "level"=5 "msg"="running command" "cwd"="" "cmd"="git config --global credential.helper "cache --timeout 3600""
I0321 18:42:14.104320      19 main.go:1306] "level"=6 "msg"="command result" "stdout"="" "stderr"=""
I0321 18:42:14.104331      19 main.go:1306] "level"=5 "msg"="running command" "cwd"="" "cmd"="git config --global safe.directory *"
I0321 18:42:14.105838      19 main.go:1306] "level"=6 "msg"="command result" "stdout"="" "stderr"=""
I0321 18:42:14.105861      19 main.go:539] "level"=1 "msg"="syncing repo"
I0321 18:42:14.105900      19 main.go:950] "level"=0 "msg"="cloning repo" "origin"="https://github.com/kubernetes/git-sync" "path"="/tmp"
I0321 18:42:14.105928      19 main.go:952] "level"=5 "msg"="running command" "cwd"="" "cmd"="git clone -v --no-checkout -b master --depth 1 https://github.com/kubernetes/git-sync /tmp"
I0321 18:42:14.107415      19 main.go:956] "level"=0 "msg"="git root exists and is not empty (previous crash?), cleaning up" "path"="/tmp"
I0321 18:42:14.107527      19 main.go:623] "level"=2 "msg"="removing path recursively" "path"="/tmp/.gitconfig" "isDir"=false
I0321 18:42:14.107591      19 main.go:623] "level"=2 "msg"="removing path recursively" "path"="/tmp/git" "isDir"=true
I0321 18:42:14.107658      19 main.go:964] "level"=5 "msg"="running command" "cwd"="" "cmd"="git clone -v --no-checkout -b master --depth 1 https://github.com/kubernetes/git-sync /tmp"
I0321 18:42:15.222096      19 main.go:964] "level"=6 "msg"="command result" "stdout"="" "stderr"="Cloning into '/tmp'...
POST git-upload-pack (353 bytes)
POST git-upload-pack (194 bytes)"
I0321 18:42:15.222141      19 main.go:1017] "level"=5 "msg"="running command" "cwd"="/tmp" "cmd"="git rev-parse HEAD"
E0321 18:42:15.223598      19 main.go:547] "msg"="too many failures, aborting" "error"="Run(git rev-parse HEAD): exit status 128: { stdout: "", stderr: "fatal: detected dubious ownership in repository at '/tmp'\nTo add an exception for this directory, call:\n\n\tgit config --global --add safe.directory /tmp" }" "failCount"=1

The relevant parts are:

I0321 18:42:14.104331      19 main.go:1306] "level"=5 "msg"="running command" "cwd"="" "cmd"="git config --global safe.directory *"
...
I0321 18:42:14.107415      19 main.go:956] "level"=0 "msg"="git root exists and is not empty (previous crash?), cleaning up" "path"="/tmp"
I0321 18:42:14.107527      19 main.go:623] "level"=2 "msg"="removing path recursively" "path"="/tmp/.gitconfig" "isDir"=false

We added a safe.directory for you, but because you pointed the root at /tmp, which happens to be the default $HOME, the .gitconfig file gets blown away by the sync logic (git can't clone into a non-empty directory).

This might have worked before because the config options in that file were not CRITICAL to your operation. But now they are.

I could add logic to store the gitconfig somewhere else, but I would need to make a different $HOME, because I need a place to write the config that isn't /tmp.

I think the better option is for you to use a different --root - blowing away all of /tmp is sort of unexpected and I can't say for sure that NOTHING ELSE will hit that. Lots of programs use tmp files, and git-sync execs git, which exec other tools.

If you set --root=/tmp/git I think it will all just work.

[vineetnegi009]

yes, changing the --root to /tmp/git resolved the problem with git-sync.