search

apache / apisix-ingress-controller

APISIX Ingress Controller for Kubernetes
https://apisix.apache.org/
Apache License 2.0
1.02k stars 343 forks source link

bug: ingress-controller always shows this log not found service, but the service is actually there #886

Closed GhangZh closed 2 years ago

GhangZh commented 2 years ago

Issue description

ingress-controller always shows this log, but the service is actually there image image image

Environment

Minimal test code / Steps to reproduce

kubectl -n ingress-apisix logs -f apisix-ingress-controller-xxxxxx |grep retry kubectl -n ingress-apisix describe ar apisixroute-dashboard

Actual result

no error log

Error log

2022-02-24T15:39:39+08:00 warn ingress/apisix_route.go:307 sync ApisixRoute failed, will retry {"object": {"Type":1,"Object":{"Key":"ingress-apisix/apisixroute-dashboard","OldObject":null,"GroupVersion":"apisix.apache.org/v2beta1"},"Tombstone":null}, "error": "service \"apisix-dashboard\" not found"}

Expected result

No response

tao12345666333 commented 2 years ago

can you run kubectl api-resources --api-group=apisix.apache.org ?

GhangZh commented 2 years ago

kubectl api-resources --api-group=apisix.apache.org

image

tao12345666333 commented 2 years ago

Are you using the latest CRD? https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix-ingress-controller/crds/customresourcedefinitions.yaml

GhangZh commented 2 years ago

Are you using the latest CRD? https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix-ingress-controller/crds/customresourcedefinitions.yaml

Okay, I'll try it out.

tao12345666333 commented 2 years ago

We have made some adjustments to CRD in the new version

elricli commented 2 years ago

Are you using the latest CRD? https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix-ingress-controller/crds/customresourcedefinitions.yaml

Same. It's using ltest CRD. image

tao12345666333 commented 2 years ago

@drrrMikado can you show me your full logs? And definitions for resources such as ApisixRoute and Service

elricli commented 2 years ago

log:

 2022-03-01T09:49:03+08:00  error   translation/apisix_route.go:338 failed to get service port in backend   {"backend": {"serviceName":"flow-control-server-go-dev","servicePort":80,"weight":null}, "apisix_route": {"kind":"ApisixRoute","apiVersion":"apisix.apache.org/v2beta2","metadata":{"name":"flow-control-server-go-dev","namespace":"flow-control-server-go","selfLink":"/apis/apisix.apache.org/v2beta2/namespaces/flow-control-server-go/apisixroutes/flow-control-server-go-dev","uid":"178a1217-3674-465e-b07b-be7d60acc57c","resourceVersion":"9564966417","generation":1,"creationTimestamp":"2022-02-28T03:51:33Z","annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apisix.apache.org/v2beta3\",\"kind\":\"ApisixRoute\",\"metadata\":{\"annotations\":{},\"name\":\"flow-control-server-go-dev\",\"namespace\":\"flow-control-server-go\"},\"spec\":{\"http\":[{\"backends\":[{\"serviceName\":\"flow-control-server-go-dev\",\"servicePort\":80}],\"match\":{\"hosts\":[\"flow-control-server-go-dev.xxx.com\"],\"paths\":[\"/ws\"]},\"name\":\"websocket\",\"websocket\":true},{\"backends\":[{\"serviceName\":\"flow-control-server-go-dev\",\"servicePort\":80}],\"match\":{\"hosts\":[\"flow-control-server-go-dev.xxx.com\"],\"paths\":[\"/*\"]},\"name\":\"default\"}]}}\n"},"managedFields":[{"manager":"apisix-ingress-controller","operation":"Update","apiVersion":"apisix.apache.org/v2beta2","time":"2022-02-28T03:51:33Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:http":{}},"f:status":{".":{},"f:conditions":{}}}},{"manager":"kubectl-client-side-apply","operation":"Update","apiVersion":"apisix.apache.org/v2beta3","time":"2022-02-28T03:51:33Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}}},"f:spec":{}}}]},"spec":{"http":[{"name":"websocket","match":{"paths":["/ws"],"hosts":["flow-control-server-go-dev.xxx.com"]},"backends":[{"serviceName":"flow-control-server-go-dev","servicePort":80,"weight":null}],"websocket":true,"authentication":{"enable":false,"type":"","keyauth":{}}},{"name":"default","match":{"paths":["/*"],"hosts":["flow-control-server-go-dev.xxx.com"]},"backends":[{"serviceName":"flow-control-server-go-dev","servicePort":80,"weight":null}],"websocket":false,"authentication":{"enable":false,"type":"","keyauth":{}}}]},"status":{"conditions":[{"type":"ResourcesAvailable","status":"True","observedGeneration":1,"lastTransitionTime":null,"reason":"ResourcesSynced","message":"Sync Successfully"}]}}, "error": "service \"flow-control-server-go-dev\" not found"}

 2022-03-01T09:49:03+08:00  error   ingress/apisix_route.go:159 failed to translate ApisixRoute v2beta2 {"error": "service \"flow-control-server-go-dev\" not found", "object": {}}

 2022-03-01T09:49:03+08:00  warn    ingress/apisix_route.go:307 sync ApisixRoute failed, will retry {"object": {"Type":1,"Object":{"Key":"flow-control-server-go/flow-control-server-go-dev","OldObject":null,"GroupVersion":"apisix.apache.org/v2beta2"},"Tombstone":null}, "error": "service \"flow-control-server-go-dev\" not found"}

// 'platform/account-system' also not found, but log level is info, not error.
 2022-03-01T09:49:03+08:00  info    ingress/controller.go:609   service platform/account-system not found

ApisixRoute:

apiVersion: apisix.apache.org/v2beta3
kind: ApisixRoute
metadata:
  name: flow-control-server-go-dev
  namespace: flow-control-server-go
spec:
  http:
  - name: websocket
    match:
      hosts:
      - flow-control-server-go-dev.xxx.com
      paths:
      - /ws
    backends:
      - serviceName: flow-control-server-go-dev
        servicePort: 80
    websocket: true
  - name: default
    match:
      hosts:
      - flow-control-server-go-dev.xxx.com
      paths:
      - /*
    backends:
      - serviceName: flow-control-server-go-dev
        servicePort: 80

Service:

kind: Service
apiVersion: v1
metadata:
  name: flow-control-server-go-dev
  namespace: flow-control-server-go
  labels:
    app: flow-control-server-go-dev
spec:
  ports:
    - name: web
      protocol: TCP
      port: 80
      targetPort: 8000
  selector:
    app: flow-control-server-go-dev
    version: v1
  type: NodePort
  sessionAffinity: None
tao12345666333 commented 2 years ago

hi @drrrMikado I see that your ApisixRoute resource is in flow-control-server-go namespace, but Service flow-control-server-go-dev is in flow-control-server-go-dev namespace, so it will not be found.

elricli commented 2 years ago

hi @drrrMikado I see that your ApisixRoute resource is in flow-control-server-go namespace, but Service flow-control-server-go-dev is in flow-control-server-go-dev namespace, so it will not be found.

I'm sorry. It's my fault. I copied it wrong. I updated the above configuration and added screenshots. image

elricli commented 2 years ago

Also, I get a different error.

 2022-03-01T17:55:49+08:00  error   ingress/status.go:197   failed to get APISIX gateway external IPs   {"error": "resource name may not be empty"}

 2022-03-01T17:55:49+08:00  error   ingress/status.go:197   failed to get APISIX gateway external IPs   {"error": "resource name may not be empty"}
tao12345666333 commented 2 years ago

Could you please post the complete installation and reproduction steps?

Your newly posted log indicates that you have enabled the Ingress LB status function, which requires you to check your Kubernetes environment.

elricli commented 2 years ago

Could you please post the complete installation and reproduction steps?

It took a while for the problem to occur, so I can't give you the reproduction steps.

Your newly posted log indicates that you have enabled the Ingress LB status function, which requires you to check your Kubernetes environment.

Thanks, I will check the Kubernetes environment.

github-actions[bot] commented 2 years ago

This issue has been marked as stale due to 90 days of inactivity. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@apisix.apache.org list. Thank you for your contributions.

github-actions[bot] commented 2 years ago

This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.