[Arktos-Mizar-Integration] case 1.5 - create new vpc in system tenant and attach pod got pod IP from VPC0

[Sindica]

What happened: Create new VPC, subnet, network, and attach pod to new VPC, pod was assigned IP in VPC0 range.

What you expected to happen: The new pod within new network should have IP belongs to new VPC range.

How to reproduce it (as minimally and precisely as possible):

1. Setting up arktos env with PR 1223 https://github.com/CentaurusInfra/arktos/pull/1223, follow step 1,2,3 on docs/setup-guide/arktos-with-mizar-cni.md
2. Wait till default bouncer to be provisioned
3. Create 2 test pods on default vpc0, each got IP starting as 20.0.0., which is correct. Verified pods can access each other.
4. Create new VPC/Subnet/Network/Pod one by one using following spec:

apiVersion: mizar.com/v1
kind: Vpc
metadata:
  name: vpc-ying
spec:
  ip: "21.0.0.0"
  prefix: "16"
  dividers: 1
  status: "Init"

For the following yaml, fill in vni value

apiVersion: mizar.com/v1
kind: Subnet
metadata:
  name: net-ying
spec:
  vni: "<replace with vni created for vpc-ying>"
  ip: "21.0.0.0"
  prefix: "24"
  bouncers: 1
  vpc: "vpc-ying"
  status: "Init"

apiVersion: arktos.futurewei.com/v1
kind: Network
metadata:
  name: network-ying
spec:
  type: mizar
  vpcID: vpc-ying

apiVersion: v1
kind: Pod
metadata:
  name: ying-nginx
  labels:
    arktos.futurewei.com/network: network-ying
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
      - containerPort: 443

5. pod ying-nginx was created and running, but got IP starting with 20.0.0:

   ~/go/src/k8s.io/arktos$ ./cluster/kubectl.sh get pods -AT -o wide
   TENANT   NAMESPACE     NAME                                    HASHKEY               READY   STATUS    RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
   system   default       mizar-daemon-7lr5r                      6137066370859238222   1/1     Running   0          26m     172.30.0.41   ip-172-30-0-41   <none>           <none>
   system   default       mizar-operator-b445854c4-ld9gs          119023066103715926    1/1     Running   0          26m     172.30.0.41   ip-172-30-0-41   <none>           <none>
   system   default       netpod1                                 1026460665228584755   1/1     Running   0          21m     20.0.0.46     ip-172-30-0-41   <none>           <none>
   system   default       netpod2                                 4287236189788627686   1/1     Running   0          21m     20.0.0.45     ip-172-30-0-41   <none>           <none>
   system   default       ying-nginx                              5083737802278720504   1/1     Running   0          9s      20.0.0.58     ip-172-30-0-41   <none>           <none>
   system   kube-system   coredns-default-75d7fb94bd-r2qfq        4074958189996175074   1/1     Running   0          26m     20.0.0.10     ip-172-30-0-41   <none>           <none>
   system   kube-system   coredns-network-ying-5f7d5c8c78-2tgf5   7424768106562695321   1/1     Running   0          3m30s   20.0.0.30     ip-172-30-0-41   <none>           <none>
   system   kube-system   kube-dns-554c5866fc-ncxl7               5892732495534980291   3/3     Running   0          26m     20.0.0.6      ip-172-30-0-41   <none>           <none>
   system   kube-system   virtlet-c6jw4                           1950166177480033596   3/3     Running   0          26m     172.30.0.41   ip-172-30-0-41   <none>           <none>

[Sindica]

Followed new instruction on Mizar https://github.com/CentaurusInfra/mizar/blob/dev-next/docs/design/mp_pod_to_vpc_subnet.md, pod ip for second VPC was corrected assigned. This is a change of design. Will need to discuss further

[Sindica]

Closing this issue as system pod works with Mizar VPC. Tracking tenant pod creating issue in https://github.com/CentaurusInfra/mizar/issues/568