search

ironcore-dev / ipam

k8s operator for IPAM
Apache License 2.0
1 stars 3 forks source link

IPAM controller panic on bulk subnets deletion #18

Closed aobort closed 3 years ago

aobort commented 3 years ago

What happened: On bulk subnets deletion ipam controller throws panic during subnet resource finalization

E1027 11:38:09.121592       1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
goroutine 288 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic(0x1665080, 0x2399fd0)
        /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/runtime/runtime.go:74 +0x95
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
        /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/runtime/runtime.go:48 +0x89
panic(0x1665080, 0x2399fd0)
        /usr/local/go/src/runtime/panic.go:969 +0x1b9
github.com/onmetal/ipam/api/v1alpha1.(*CIDR).CanReserve(0x0, 0xc0001071e8, 0xc0001b2fc0)
        /workspace/api/v1alpha1/cidr_types.go:275 +0x29
github.com/onmetal/ipam/api/v1alpha1.(*Subnet).Release(0xc0001b2d80, 0xc0001071e8, 0xc000249380, 0xc000315c80)
        /workspace/api/v1alpha1/subnet_types.go:298 +0x57
github.com/onmetal/ipam/controllers.(*SubnetReconciler).finalizeSubnet(0xc0006dee40, 0x1a12720, 0xc000249380, 0x1a1b3e0, 0xc000237800, 0xc000315c80, 0x7, 0xc000279300, 0x17, 0xc0001b2900, ...)
        /workspace/controllers/subnet_controller.go:342 +0x1405
github.com/onmetal/ipam/controllers.(*SubnetReconciler).Reconcile(0xc0006dee40, 0x1a12720, 0xc000249380, 0xc000315c80, 0x7, 0xc000279300, 0x17, 0xc000249380, 0x40a1ff, 0xc000030000, ...)
        /workspace/controllers/subnet_controller.go:91 +0x2e5
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc00043cb40, 0x1a12660, 0xc0006df780, 0x16bbc40, 0xc00024a920)
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.7.2/pkg/internal/controller/controller.go:263 +0x317
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc00043cb40, 0x1a12660, 0xc0006df780, 0x0)
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.7.2/pkg/internal/controller/controller.go:235 +0x205
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.1(0x1a12660, 0xc0006df780)
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.7.2/pkg/internal/controller/controller.go:198 +0x4a
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1()
        /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185 +0x37
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00016ff50)
        /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00040bf50, 0x19db000, 0xc000249260, 0xc0006df701, 0xc0004302a0)
        /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:156 +0xad
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00016ff50, 0x3b9aca00, 0x0, 0x1, 0xc0004302a0)
        /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext(0x1a12660, 0xc0006df780, 0xc000237760, 0x3b9aca00, 0x0, 0x1)
        /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:185 +0xa6
k8s.io/apimachinery/pkg/util/wait.UntilWithContext(0x1a12660, 0xc0006df780, 0xc000237760, 0x3b9aca00)
        /go/pkg/mod/k8s.io/apimachinery@v0.20.2/pkg/util/wait/wait.go:99 +0x57
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.7.2/pkg/internal/controller/controller.go:195 +0x4e7
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1512109]

What you expected to happen: Subnet resources deleted

How to reproduce it (as minimally and precisely as possible):

  1. deploy ipam operator built from main branch
  2. create several nested subnets, i.e. 
    apiVersion: ipam.onmetal.de/v1alpha1
kind: Network
metadata:
name: underlay
namespace: default
spec:
description: sample
---
apiVersion: ipam.onmetal.de/v1alpha1
kind: Subnet
metadata:
name: switch-ranges-v4
namespace: default
spec:
cidr: "100.64.0.0/16"
networkName: underlay
regions:
- name: eu-west
  availabilityZones:
  - A
---
apiVersion: ipam.onmetal.de/v1alpha1
kind: Subnet
metadata:
name: subnet-parent-001-v4
namespace: default
spec:
capacity: 512
networkName: underlay
parentSubnetName: switch-ranges-v4
regions:
- name: eu-west
  availabilityZones:
  - A
---
apiVersion: ipam.onmetal.de/v1alpha1
kind: Subnet
metadata:
name: subnet-parent-002-v4
namespace: default
spec:
capacity: 512
networkName: underlay
parentSubnetName: switch-ranges-v4
regions:
- name: eu-west
  availabilityZones:
  - A
---
apiVersion: ipam.onmetal.de/v1alpha1
kind: Subnet
metadata:
name: subnet-parent-003-v4
namespace: default
spec:
capacity: 512
networkName: underlay
parentSubnetName: switch-ranges-v4
regions:
- name: eu-west
  availabilityZones:
  - A
---
apiVersion: ipam.onmetal.de/v1alpha1
kind: Subnet
metadata:
name: subnet-child-001-001-v4
namespace: default
spec:
capacity: 4
networkName: underlay
parentSubnetName: subnet-parent-001-v4
regions:
- name: eu-west
  availabilityZones:
  - A
---
apiVersion: ipam.onmetal.de/v1alpha1
kind: Subnet
metadata:
name: subnet-child-001-002-v4
namespace: default
spec:
capacity: 4
networkName: underlay
parentSubnetName: subnet-parent-001-v4
regions:
- name: eu-west
  availabilityZones:
  - A
---
apiVersion: ipam.onmetal.de/v1alpha1
kind: Subnet
metadata:
name: subnet-child-001-003-v4
namespace: default
spec:
capacity: 4
networkName: underlay
parentSubnetName: subnet-parent-001-v4
regions:
- name: eu-west
  availabilityZones:
  - A
---
apiVersion: ipam.onmetal.de/v1alpha1
kind: Subnet
metadata:
name: subnet-child-002-001-v4
namespace: default
spec:
capacity: 4
networkName: underlay
parentSubnetName: subnet-parent-002-v4
regions:
- name: eu-west
  availabilityZones:
  - A
---
apiVersion: ipam.onmetal.de/v1alpha1
kind: Subnet
metadata:
name: subnet-child-003-001-v4
namespace: default
spec:
capacity: 4
networkName: underlay
parentSubnetName: subnet-parent-003-v4
regions:
- name: eu-west
  availabilityZones:
  - A
---
  3. delete subnets k delete subnets.ipam.onmetal.de --all

Environment: 

kube-user@minikube:~/ipam$ uname -a
Linux minikube 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 GNU/Linux

kube-user@minikube:~/ipam$ minikube version
minikube version: v1.20.0
commit: c61663e942ec43b20e8e70839dcca52e44cd85ae

kube-user@minikube:~/ipam$ git log --format="%H" -n 1
8bd1d806158ebbd8681db50386f24187ab802966
aiivashchenko commented 3 years ago

Fixed in #20