Open epg-mori-k opened 2 years ago
Having several dozens of applications sourced from gitlab.com and using webhooks for refreshing the applications, I can only say that it works for me.
If the refresh for your application(s) is not requested, it may have several other causes, for example:
targetRevision
)Can you provide more details about your application and repository?
The push was made to another branch (i.e. doesn't match the application's targetRevision)
https://github.com/argoproj/argocd-example-apps We have cloned the above repository for testing purposes. I have confirmed that it works by specifying guestbook as the path and setting the load balancer to add or remove. In addition, I've added --insecure option to argocd-server and set up frontend in ingress.
apiVersion: v1
kind: Service
metadata:
name: guestbook-ui-lb
spec:
type: LoadBalancer
selector:
app: guestbook-ui
ports:
- protocol: TCP
port: 80
targetPort: 80
The push contained only files outside of what your application renders
The target revision was checked by specifying HEAD branchs or master branchs. It can be refreshed manually from the WebUI or at polling intervals to reflect the update.
$ argocd --grpc-web app list
NAME CLUSTER NAMESPACE PROJECT STATUS HEALTH SYNCPOLICY CONDITIONS REPO PATH TARGET
github https://kubernetes.default.svc github github Synced Healthy Auto-Prune <none> git@github.com:username/examples.git guestbook HEAD
gitlab https://kubernetes.default.svc gitlab gitlab Synced Healthy Auto-Prune <none> git@mydomain:username/example.git guestbook HEAD
$ argocd --grpc-web app get gitlab
Name: gitlab
Project: gitlab
Server: https://kubernetes.default.svc
Namespace: gitlab
URL: https://argocd.mydomain/applications/gitlab
Repo: git@mydomain:username/example.git
Target: HEAD
Path: guestbook
SyncWindow: Sync Allowed
Sync Policy: Automated (Prune)
Sync Status: Synced to HEAD (7acc4ad)
Health Status: Healthy
GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE
Service gitlab guestbook-ui-lb Succeeded Pruned pruned
Service gitlab guestbook-ui Synced Healthy service/guestbook-ui unchanged
apps Deployment gitlab guestbook-ui Synced Healthy deployment.apps/guestbook-ui unchanged
However, in my environment the immediate refresh via webhook trigger from Gitlab is not working.
# argocd-server
$ kubectl logs -f `kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-server -o name | cut -d'/' -f 2` -n argocd
time="2021-11-15T09:49:32Z" level=info msg="Received push event repo: https://mydomain/gitlab/username/example, revision: master, touchedHead: true"
# argocd-repo-server
$ kubectl logs -f `kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-repo-server -o name | cut -d'/' -f 2` -n argocd
time="2021-11-15T09:51:03Z" level=info msg="manifest cache hit: &ApplicationSource{RepoURL:git@github.com:username/examples.git,Path:guestbook,TargetRevision:HEAD,Helm:nil,Kustomize:nil,Ksonnet:nil,Directory:nil,Plugin:nil,Chart:,}/34a9ca72ca94bfc0b46eb732b48f6ca962652764"
time="2021-11-15T09:51:03Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=GenerateManifest grpc.request.deadline="2021-11-15T09:52:03Z" grpc.service=repository.RepoServerService grpc.start_time="2021-11-15T09:51:03Z" grpc.time_ms=2.916 span.kind=server system=grpc
time="2021-11-15T09:51:03Z" level=info msg="manifest cache hit: &ApplicationSource{RepoURL:git@mydomain:username/example.git,Path:guestbook,TargetRevision:HEAD,Helm:nil,Kustomize:nil,Ksonnet:nil,Directory:nil,Plugin:nil,Chart:,}/74ac83aff44ee8d85857352a2b8d5dd924a4fa98"
time="2021-11-15T09:51:03Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=GenerateManifest grpc.request.deadline="2021-11-15T09:52:03Z" grpc.service=repository.RepoServerService grpc.start_time="2021-11-15T09:51:03Z" grpc.time_ms=3.628 span
time="2021-11-15T09:54:03Z" level=info msg="manifest cache miss: &ApplicationSource{RepoURL:git@mydomain:username/example.git,Path:guestbook,TargetRevision:HEAD,Helm:nil,Kustomize:nil,Ksonnet:nil,Directory:nil,Plugin:nil,Chart:,}/e9e2af4ba6b67e38f322a7097aec578c3d9b64e0"
time="2021-11-15T09:54:03Z" level=info msg="git fetch origin --tags --force" dir=/tmp/git@mydomain_username_example execID=DDEGY
time="2021-11-15T09:54:04Z" level=info msg=Trace args="[git fetch origin --tags --force]" dir=/tmp/git@mydomain_username_example operation_name="exec git" time_ms=491.977447
time="2021-11-15T09:54:04Z" level=info msg="git checkout --force e9e2af4ba6b67e38f322a7097aec578c3d9b64e0" dir=/tmp/git@mydomain_username_example execID=TsVRn
time="2021-11-15T09:54:04Z" level=info msg=Trace args="[git checkout --force e9e2af4ba6b67e38f322a7097aec578c3d9b64e0]" dir=/tmp/git@mydomain_username_example operation_name="exec git" time_ms=7.602257000000001
time="2021-11-15T09:54:04Z" level=info msg="git clean -fdx" dir=/tmp/git@mydomain_username_example execID=sha6G
time="2021-11-15T09:54:04Z" level=info msg=Trace args="[git clean -fdx]" dir=/tmp/git@mydomain_username_example operation_name="exec git" time_ms=3.242479
time="2021-11-15T09:54:04Z" level=info msg="git rev-parse HEAD" dir=/tmp/git@mydomain_username_example execID=z6rfZ
time="2021-11-15T09:54:04Z" level=info msg=Trace args="[git rev-parse HEAD]" dir=/tmp/git@mydomain_username_example operation_name="exec git" time_ms=2.855512
time="2021-11-15T09:54:04Z" level=info msg="manifest cache miss: &ApplicationSource{RepoURL:git@mydomain:username/example.git,Path:guestbook,TargetRevision:HEAD,Helm:nil,Kustomize:nil,Ksonnet:nil,Directory:nil,Plugin:nil,Chart:,}/e9e2af4ba6b67e38f322a7097aec578c3d9b64e0"
time="2021-11-15T09:54:04Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=GenerateManifest grpc.request.deadline="2021-11-15T09:55:03Z" grpc.service=repository.RepoServerService grpc.start_time="2021-11-15T09:54:03Z" grpc.time_ms=1160.56 span.kind=server system=grpc
time="2021-11-15T09:54:04Z" level=info msg="manifest cache hit: &ApplicationSource{RepoURL:git@mydomain:username/example.git,Path:guestbook,TargetRevision:HEAD,Helm:nil,Kustomize:nil,Ksonnet:nil,Directory:nil,Plugin:nil,Chart:,}/e9e2af4ba6b67e38f322a7097aec578c3d9b64e0"
time="2021-11-15T09:54:04Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=GenerateManifest grpc.request.deadline="2021-11-15T09:55:04Z" grpc.service=repository.RepoServerService grpc.start_time="2021-11-15T09:54:04Z" grpc.time_ms=0.745 span.kind=server system=grpc
time="2021-11-15T09:54:04Z" level=info msg="manifest cache hit: &ApplicationSource{RepoURL:git@github.com:username/examples.git,Path:guestbook,TargetRevision:HEAD,Helm:nil,Kustomize:nil,Ksonnet:nil,Directory:nil,Plugin:nil,Chart:,}/34a9ca72ca94bfc0b46eb732b48f6ca962652764"
time="2021-11-15T09:54:04Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=GenerateManifest grpc.request.deadline="2021-11-15T09:55:03Z" grpc.service=repository.RepoServerService grpc.start_time="2021-11-15T09:54:03Z" grpc.time_ms=1428.516 span.kind=server system=grpc
time="2021-11-15T09:54:05Z" level=info msg="manifest cache hit: &ApplicationSource{RepoURL:git@mydomain:username/example.git,Path:guestbook,TargetRevision:HEAD,Helm:nil,Kustomize:nil,Ksonnet:nil,Directory:nil,Plugin:nil,Chart:,}/e9e2af4ba6b67e38f322a7097aec578c3d9b64e0"
time="2021-11-15T09:54:05Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=GenerateManifest grpc.request.deadline="2021-11-15T09:55:05Z" grpc.service=repository.RepoServerService grpc.start_time="2021-11-15T09:54:05Z" grpc.time_ms=1.174 span.kind=server system=grpc
time="2021-11-15T09:54:05Z" level=info msg="manifest cache hit: &ApplicationSource{RepoURL:git@mydomain:username/example.git,Path:guestbook,TargetRevision:HEAD,Helm:nil,Kustomize:nil,Ksonnet:nil,Directory:nil,Plugin:nil,Chart:,}/e9e2af4ba6b67e38f322a7097aec578c3d9b64e0"
If you have an SSH connection with a Repository URL starting with git@, the refresh on receiving the webhook doesn't seem to be executed. For HTTPS connections starting with https, Gitlab now also refreshes on receipt of the webhook
GitHub webhooks work fine when SSHing into URLs starting with git@.
@epg-mori-k do you have an example of a failing gitlab webhook payload?
I think the use case you're describing is covered by tests in this PR: https://github.com/argoproj/argo-cd/pull/7981
But those tests pass, so I'm still not sure what's up.
@crenshaw-dev We seem to be affected by the same issue, in our case the git-http-url and git-ssh-url are different for Kubernetes reasons. argo-cd uses the Repo URL ssh://git@git-ssh.corp.dev/corp/ops/argoapps.git
{
"object_kind": "push",
"event_name": "push",
"before": "$BEFORESHA",
"after": "$COMMITSHA",
"ref": "refs/heads/master",
"checkout_sha": "$COMMITSHA",
"message": null,
"user_id": 23,
"user_name": "My User",
"user_username": "myname",
"user_email": "",
"user_avatar": "https://git.corp.dev/uploads/-/system/user/avatar/92/avatar.png",
"project_id": 87,
"project": {
"id": 87,
"name": "argoapps",
"description": "",
"web_url": "https://git.corp.dev/corp/ops/argoapps",
"avatar_url": "https://git.corp.dev/uploads/-/system/project/avatar/87/argo-icon-color.png",
"git_ssh_url": "git@git-ssh.corp.dev:corp/ops/argoapps.git",
"git_http_url": "https://git.corp.dev/corp/ops/argoapps.git",
"namespace": "Ops",
"visibility_level": 10,
"path_with_namespace": "corp/ops/argoapps",
"default_branch": "master",
"ci_config_path": null,
"homepage": "https://git.corp.dev/corp/ops/argoapps",
"url": "git@git-ssh.corp.dev:corp/ops/argoapps.git",
"ssh_url": "git@git-ssh.corp.dev:corp/ops/argoapps.git",
"http_url": "https://git.corp.dev/corp/ops/argoapps.git"
},
"commits": [
{
"id": "$COMMITSHA",
"message": "mymessage\n",
"title": "mymessage",
"timestamp": "2022-09-23T08:30:11+02:00",
"url": "https://git.corp.dev/corp/ops/argoapps/-/commit/$COMMITSHA",
"author": {
"name": "My Name",
"email": "myname@corp.com"
},
"added": [
],
"modified": [
],
"removed": [
"argocd/templates/ingress.yaml"
]
}
],
"total_commits_count": 1,
"push_options": {
},
"repository": {
"name": "argoapps",
"url": "git@git-ssh.corp.dev:corp/ops/argoapps.git",
"description": "",
"homepage": "https://git.corp.dev/corp/ops/argoapps",
"git_http_url": "https://git.corp.dev/corp/ops/argoapps.git",
"git_ssh_url": "git@git-ssh.corp.dev:corp/ops/argoapps.git",
"visibility_level": 10
}
}
In the code base for the webhook the gitlab push event is using the project.web_url of the Gitlab event to match applications, if you're using ssh-based authentication this won't match and the webhook will not trigger any refresh.
here's the code that triggers this issue : https://github.com/argoproj/argo-cd/blob/master/util/webhook/webhook.go#L130
Facing similar issue with applicationset-controler based on image quay.io/argoproj/argocd:v2.5.2 Any plan for a fix or a workaround ? Thx
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: cw-datapulse-dev
annotations:
# resolves to the 'deployment/develop-cw/*' directory
# argocd.argoproj.io/manifest-generate-paths: .
spec:
generators:
- git:
repoURL: ssh://git@gitlab.tech.orange/webcom/devops/argocd/services/datapulse/deployments.git
revision: release
directories:
- path: deployment/develop-cw/*
"commits": [
{
"id": "893b5afb2cdf73de15389290683afc47232f5b05",
"message": "develop-cw: change serviceFqdn value for webcom-server-orange-mongodb to test propagation into Internal Conf\n",
"title": "develop-cw: change serviceFqdn value for webcom-server-orange-mongodb to test...",
"timestamp": "2023-01-23T17:08:27+00:00",
"url": "https://gitlab.tech.orange/webcom/devops/argocd/services/datapulse/deployments/-/commit/893b5afb2cdf73de15389290683afc47232f5b05",
"author": {
"name": "",
"email": "[REDACTED]"
},
"added": [
],
"modified": [
"deployment/develop-cw/oauth2/deployment.yaml"
],
"removed": [
]
}
]
I have the same problem - I guess the problem is that the project.web_url dont have the .git extension
Project from webhook
"project": {
"id": 33,
"name": "unifi-controller",
"description": null,
"web_url": "https://git.abc.lan/pond/apps/unifi-controller",
"avatar_url": null,
"git_ssh_url": "git@git.abc.lan:pond/apps/unifi-controller.git",
"git_http_url": "https://git.abc.lan/pond/apps/unifi-controller.git",
"namespace": "apps",
"visibility_level": 10,
"path_with_namespace": "pond/apps/unifi-controller",
"default_branch": "main",
"ci_config_path": null,
"homepage": "https://git.abc.lan/pond/apps/unifi-controller",
"url": "git@git.abc.lan:pond/apps/unifi-controller.git",
"ssh_url": "git@git.abc.lan:pond/apps/unifi-controller.git",
"http_url": "https://git.abc.lan/pond/apps/unifi-controller.git"
},
ApplicationSet yaml
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: all-gitlab-apps
namespace: argocd
spec:
generators:
- scmProvider:
gitlab:
# The base GitLab group to scan. You can either use the group id or the full namespaced path.
group: "39"
# For self-hosted GitLab:
api: https://git.abc.lan/
# If true, scan every branch of every repository. If false, scan only the default branch. Defaults to false.
allBranches: false
# If true, recurses through subgroups. If false, it searches only in the base group. Defaults to false.
includeSubgroups: true
# Reference to a Secret containing an access token. (optional)
tokenRef:
secretName: gitlab-token
key: token
filters:
- repositoryMatch: .*
pathsDoNotExist:
- argocd/disablerepo.txt
- disablerepo.txt
pathsExist:
- argocd
template:
metadata:
name: '{{ repository }}'
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
destination:
server: https://kubernetes.default.svc
namespace: '{{ repository }}'
project: default
source:
repoURL: 'https://git.abc.lan/pond/apps/{{ repository }}.git'
targetRevision: '{{ branch }}'
path: argocd/
syncPolicy:
syncOptions:
- CreateNamespace=true
automated:
selfHeal: true # If we manually run kubectl apply, it will sync the git repository
prune: true
We also experience this bug, but only for namespaced argocd apps, GitLab hooks work fine for argocd apps in the argocd
namespace, so webhooks work for this application:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: test
namespace: argocd
spec:
project: x
source:
repoURL: https://gitlab.com/xxx/yyy/zzz.git
targetRevision: foo
path: bar
destination:
namespace: baz
name: in-cluster
but not for this one
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: test
namespace: custom
spec:
project: x
source:
repoURL: https://gitlab.com/xxx/yyy/zzz.git
targetRevision: foo
path: bar
destination:
namespace: baz
name: in-cluster
the only difference being metadata.namespace
field of the application resource.
Should I create a new issue about this?
我也遇到同样的问题,不知道有解决办法么?
日志中只有这个:time="2023-05-09T05:37:26Z" level=info msg="Received push event repo: http://gitlab-kust.xxx.com/root/godprotect, revision: master, touchedHead: true"
Checklist:
argocd version
.Describe the bug
refresh when receiving a webhook from GitLab do not work. It refreshes when it receives a webhook from GitHub.
To Reproduce
https://github.com/argoproj/argo-cd/tree/v2.1.6 https://argo-cd.readthedocs.io/en/stable/operator-manual/webhook/ I set up WebHook with reference to the above.
Configuring a WebHook for ArgoCD from GitHub and GitLab We have successfully sent WebHooks from both GitHub and GitLab. However, when receiving a WebHook from GitLab, the behavior is different from when receiving a WebHook from GitHub
Immediate updates when receiving a WebHook from GitLab do not work. It does not take effect until the next polling run, which is 3 minutes by default.
Expected behavior
As with setting up a webhook from GitHub
We expect the webhook from GitLab to be refreshd and reflected immediately.
Version
Logs When you receive a webhook from GitLab, you can use level=info msg="Requested app 'appname' refresh" logs are not output.