search

ovn-org / ovn-kubernetes

A robust Kubernetes networking platform
https://ovn-kubernetes.io/
Apache License 2.0
772 stars 334 forks source link

Invalid schema issue #2898

Open shiresky opened 2 years ago

shiresky commented 2 years ago

I tried to perform your procedure: https://github.com/ovn-org/ovn-kubernetes/blob/master/docs/INSTALL.SSL.md as follows:

# master
OVN_CONTROL_PLANE=$(kubectl get pods -n kube-system --no-headers -o custom-columns=":metadata.name" -l app=ovn-control-plane)
OVN_CONTROLLERS=$(kubectl get pods -n kube-system --no-headers -o custom-columns=":metadata.name" -l app=ovn-controller)

kubectl exec -n kube-system $OVN_CONTROL_PLANE -- bash -c '
cd /etc/openvswitch/;
ovs-pki init --force;

cp /var/lib/openvswitch/pki/switchca/cacert.pem /etc/openvswitch;

ovs-pki req+sign --force ovnnb;
ovn-nbctl set-ssl /etc/openvswitch/ovnnb-privkey.pem \
    /etc/openvswitch/ovnnb-cert.pem  /etc/openvswitch/cacert.pem;
ovn-nbctl set-connection pssl:6641;

ovs-pki req+sign --force ovnsb;
ovn-sbctl set-ssl /etc/openvswitch/ovnsb-privkey.pem \
    /etc/openvswitch/ovnsb-cert.pem  /etc/openvswitch/cacert.pem;
ovn-sbctl set-connection pssl:6642;

ovs-pki req ovncontroller;
ovs-pki -b sign ovncontroller switch'

kubectl cp -n kube-system ${OVN_CONTROL_PLANE}:etc/openvswitch/ovncontroller-cert.pem ovncontroller-cert.pem
for controller in $OVN_CONTROLLERS
do
    kubectl cp -n kube-system ovncontroller-cert.pem ${controller}:etc/openvswitch/ovncontroller-cert.pem
done

kubectl exec -n kube-system $OVN_CONTROL_PLANE -- /usr/share/openvswitch/scripts/ovn-ctl restart_northd

for controller in $OVN_CONTROLLERS
do
    kubectl exec -n kube-system $controller -- bash -c '
    echo ''OVN_CTL_OPTS="--ovn-controller-ssl-key=/etc/openvswitch/ovncontroller-privkey.pem  --ovn-controller-ssl-cert=/etc/openvswitch/ovncontroller-cert.pem --ovn-controller-ssl-ca-cert=/etc/openvswitch/cacert.pem"'' >> /etc/default/ovn-host;
    /usr/share/openvswitch/scripts/ovn-ctl \
        --ovn-controller-ssl-key="/etc/openvswitch/ovncontroller-privkey.pem"  \
        --ovn-controller-ssl-cert="/etc/openvswitch/ovncontroller-cert.pem"    \
        --ovn-controller-ssl-ca-cert="/etc/openvswitch/cacert.pem" \
        restart_controller'
done

# master and minions
mkdir certs
for file in ovnnb-privkey.pem ovnnb-cert.pem ovnsb-privkey.pem ovnsb-cert.pem cacert.pem
do
    kubectl cp -n kube-system ${OVN_CONTROL_PLANE}:etc/openvswitch/$file certs/$file
    for controller in $OVN_CONTROLLERS
    do
        kubectl cp -n kube-system certs/$file ${controller}:etc/openvswitch/$file
    done
done

snap install go --classic

git clone https://github.com/ovn-org/libovsdb.git
cd libovsdb/cmd/modelgen/
go build -o /usr/local/bin/modelgen
cd

git clone https://github.com/kubernetes/code-generator.git
cd code-generator/cmd/deepcopy-gen
go build -o /usr/local/bin/deepcopy-gen 
cd

apt install -y arping ovn-common openvswitch-switch make

git clone https://github.com/openvswitch/ovn-kubernetes
cd ovn-kubernetes/go-controller
make codegen
make
make install
cd

ovnkube -k8s-kubeconfig .kube/config -loglevel=9 \
 -k8s-apiserver="https://10.96.0.1:443" \
 -logfile="/var/log/ovn-kubernetes/ovnkube.log" \
 -init-master=master -cluster-subnets=10.233.64.0/18 \
 -k8s-service-cidr=10.96.0.0/12 \
 -nodeport \
 -nb-address="ssl:$(kubectl get svc -n kube-system ovn-nb-tcp --no-headers -o custom-columns=':spec.clusterIP'):6641" \
 -sb-address="ssl:$(kubectl get svc -n kube-system ovn-sb-tcp --no-headers -o custom-columns=':spec.clusterIP'):6642" \
 -nb-client-privkey ./certs/ovnnb-privkey.pem \
 -nb-client-cert ./certs/ovnnb-cert.pem \
 -nb-client-cacert ./certs/cacert.pem \
 -nb-cert-common-name "ovnnb id:c706274f-1396-44a5-8c62-b33617d5a286" \
 -sb-client-privkey ./certs/ovnsb-privkey.pem \
 -sb-client-cert ./certs/ovnsb-cert.pem \
 -sb-client-cacert ./certs/cacert.pem \
 -sb-cert-common-name "ovnsb id:9eeb88db-f842-48d1-9ac6-11325d6b7c58"

Durng the last step execution I receive an error related with invalid schema:

root@master:~# ovnkube -k8s-kubeconfig .kube/config -loglevel=9 \
>  -k8s-apiserver="https://10.96.0.1:443" \
>  -logfile="/var/log/ovn-kubernetes/ovnkube.log" \
>  -init-master=master -cluster-subnets=10.233.64.0/18 \
>  -k8s-service-cidr=10.96.0.0/12 \
>  -nodeport \
>  -nb-address="ssl:$(kubectl get svc -n kube-system ovn-nb-tcp --no-headers -o custom-columns=':spec.clusterIP'):6641" \
>  -sb-address="ssl:$(kubectl get svc -n kube-system ovn-sb-tcp --no-headers -o custom-columns=':spec.clusterIP'):6642" \
>  -nb-client-privkey ./certs/ovnnb-privkey.pem \
>  -nb-client-cert ./certs/ovnnb-cert.pem \
>  -nb-client-cacert ./certs/cacert.pem \
>  -nb-cert-common-name "ovnnb id:c706274f-1396-44a5-8c62-b33617d5a286" \
>  -sb-client-privkey ./certs/ovnsb-privkey.pem \
>  -sb-client-cert ./certs/ovnsb-cert.pem \
>  -sb-client-cacert ./certs/cacert.pem \
>  -sb-cert-common-name "ovnsb id:9eeb88db-f842-48d1-9ac6-11325d6b7c58"
I0407 14:07:03.548197   21681 ovs.go:93] Maximum command line arguments set to: 191102
I0407 14:07:03.551106   21681 config.go:1797] Default config: {MTU:1400 RoutableMTU:0 ConntrackZone:64000 EncapType:geneve EncapIP: EncapPort:6081 InactivityProbe:100000 OpenFlowProbe:180 MonitorAll:true LFlowCacheEnable:true LFlowCacheLimit:0 LFlowCacheLimitKb:0 RawClusterSubnets:10.233.64.0/18 ClusterSubnets:[{CIDR:10.233.64.0/18 HostSubnetLength:24}]}
I0407 14:07:03.551236   21681 config.go:1798] Logging config: {File:/var/log/ovn-kubernetes/ovnkube.log CNIFile:/var/log/ovn-kubernetes/ovn-k8s-cni-overlay.log Level:9 LogFileMaxSize:100 LogFileMaxBackups:5 LogFileMaxAge:5 ACLLoggingRateLimit:20}
I0407 14:07:03.551264   21681 config.go:1799] Monitoring config: {RawNetFlowTargets: RawSFlowTargets: RawIPFIXTargets: NetFlowTargets:[] SFlowTargets:[] IPFIXTargets:[]}
I0407 14:07:03.551278   21681 config.go:1800] IPFIX config: {Sampling:400 CacheActiveTimeout:60 CacheMaxFlows:0}
I0407 14:07:03.551298   21681 config.go:1801] CNI config: {ConfDir:/etc/cni/net.d Plugin:ovn-k8s-cni-overlay}
I0407 14:07:03.551322   21681 config.go:1802] Kubernetes config: {Kubeconfig:.kube/config CACert: CAData:[] APIServer:https://10.96.0.1:443 Token: TokenFile: CompatServiceCIDR:10.96.0.0/12 RawServiceCIDRs:10.96.0.0/12 ServiceCIDRs:[10.96.0.0/12] OVNConfigNamespace:ovn-kubernetes MetricsBindAddress: OVNMetricsBindAddress: MetricsEnablePprof:false OVNEmptyLbEvents:false PodIP: RawNoHostSubnetNodes: NoHostSubnetNodes:nil HostNetworkNamespace: PlatformType:}
I0407 14:07:03.551359   21681 config.go:1803] Gateway config: {Mode: Interface: EgressGWInterface: NextHop: VLANID:0 NodeportEnable:true DisableSNATMultipleGWs:false V4JoinSubnet:100.64.0.0/16 V6JoinSubnet:fd98::/64 DisablePacketMTUCheck:false RouterSubnet:}
I0407 14:07:03.551377   21681 config.go:1804] OVN North config: {Address:ssl:10.96.0.113:6641 PrivKey:./certs/ovnnb-privkey.pem Cert:./certs/ovnnb-cert.pem CACert:./certs/cacert.pem CertCommonName:ovnnb id:c706274f-1396-44a5-8c62-b33617d5a286 Scheme:ssl ElectionTimer:0 northbound:true exec:0x2c741a8}
I0407 14:07:03.551395   21681 config.go:1805] OVN South config: {Address:ssl:10.102.64.73:6642 PrivKey:./certs/ovnsb-privkey.pem Cert:./certs/ovnsb-cert.pem CACert:./certs/cacert.pem CertCommonName:ovnsb id:9eeb88db-f842-48d1-9ac6-11325d6b7c58 Scheme:ssl ElectionTimer:0 northbound:false exec:0x2c741a8}
I0407 14:07:03.551419   21681 config.go:1806] Hybrid Overlay config: {Enabled:false RawClusterSubnets: ClusterSubnets:[] VXLANPort:4789}
I0407 14:07:03.551439   21681 config.go:1807] Ovnkube Node config: {Mode:full MgmtPortNetdev: DisableOVNIfaceIdVer:false}
I0407 14:07:03.553042   21681 loader.go:372] Config loaded from file:  .kube/config
I0407 14:07:03.554666   21681 loader.go:372] Config loaded from file:  .kube/config
I0407 14:07:03.555960   21681 client.go:325]  "msg"="trying to connect" "database"="OVN_Northbound" "endpoint"="ssl:10.96.0.113:6641"
F0407 14:07:03.562772   21681 ovnkube.go:133] error when trying to initialize libovsdb NB client: failed to connect to ssl:10.96.0.113:6641: database OVN_Northbound validation error (15): Mapper Error. Object type nbdb.LogicalRouterPolicy contains field ExternalIDs (map[string]string) ovs tag external_ids: Column does not exist in schema. Mapper Error. Object type nbdb.LogicalRouter contains field Copp (*string) ovs tag copp: Column does not exist in schema. Mapper Error. Object type nbdb.LogicalRouterPort contains field Ipv6Prefix ([]string) ovs tag ipv6_prefix: Column does not exist in schema. database model contains a model for table Forwarding_Group that does not exist in schema. Mapper Error. Object type nbdb.LoadBalancer contains field HealthCheck ([]string) ovs tag health_check: Column does not exist in schema. Mapper Error. Object type nbdb.NBGlobal contains field HvCfgTimestamp (int) ovs tag hv_cfg_timestamp: Column does not exist in schema. database model contains a model for table Copp that does not exist in schema. Mapper Error. Object type nbdb.ACL contains field Label (int) ovs tag label: Column does not exist in schema. database model contains a model for table BFD that does not exist in schema. database model contains a model for table Load_Balancer_Health_Check that does not exist in schema. database model contains a model for table Load_Balancer_Group that does not exist in schema. Mapper Error. Object type nbdb.LogicalRouterStaticRoute contains field BFD (*string) ovs tag bfd: Column does not exist in schema. Mapper Error. Object type nbdb.Meter contains field Fair (*bool) ovs tag fair: Column does not exist in schema. Mapper Error. Object type nbdb.NAT contains field AllowedExtIPs (*string) ovs tag allowed_ext_ips: Column does not exist in schema. Mapper Error. Object type nbdb.LogicalSwitch contains field Copp (*string) ovs tag copp: Column does not exist in schema

So I switched to codegen directive provided in your Makefile but it also results with an error about some missing files. Any idea what files are expected?

root@master:~/ovn-kubernetes/go-controller# make codegen
hack/update-codegen.sh
controller-gen not found, installing sigs.k8s.io/controller-tools
Generating deepcopy funcs for egressfirewall
F0407 14:08:15.800856   24127 deepcopy.go:131] Failed loading boilerplate: open : no such file or directory
goroutine 1 [running]:
k8s.io/klog/v2.stacks(0x1)
        /home/vagrant/go/pkg/mod/k8s.io/klog/v2@v2.60.1/klog.go:860 +0x8a
k8s.io/klog/v2.(*loggingT).output(0x9ca400, 0x3, 0x0, 0xc00a92a230, 0x1, {0x859697?, 0x1?}, 0xc000f00000?, 0x0)
        /home/vagrant/go/pkg/mod/k8s.io/klog/v2@v2.60.1/klog.go:825 +0x686
k8s.io/klog/v2.(*loggingT).printfDepth(0x9ca400, 0x500?, 0x0, {0x0, 0x0}, 0xc00021e1c0?, {0x77d241, 0x1e}, {0xc00a905790, 0x1, ...})
        /home/vagrant/go/pkg/mod/k8s.io/klog/v2@v2.60.1/klog.go:630 +0x1f2
k8s.io/klog/v2.(*loggingT).printf(...)
        /home/vagrant/go/pkg/mod/k8s.io/klog/v2@v2.60.1/klog.go:612
k8s.io/klog/v2.Fatalf(...)
        /home/vagrant/go/pkg/mod/k8s.io/klog/v2@v2.60.1/klog.go:1516
k8s.io/gengo/examples/deepcopy-gen/generators.Packages(0xc00a619500, 0xc0000f0420)
        /home/vagrant/go/pkg/mod/k8s.io/gengo@v0.0.0-20211129171323-c02415ce4185/examples/deepcopy-gen/generators/deepcopy.go:131 +0x10a
k8s.io/gengo/args.(*GeneratorArgs).Execute(0xc0000f0420, 0xc0000e1e30?, {0x758a59, 0x6}, 0x78acc8)
        /home/vagrant/go/pkg/mod/k8s.io/gengo@v0.0.0-20211129171323-c02415ce4185/args/args.go:212 +0x1a2
main.main()
        /home/vagrant/code-generator/cmd/deepcopy-gen/main.go:75 +0x385
Makefile:50: recipe for target 'codegen' failed
make: *** [codegen] Error 255
github-actions[bot] commented 5 days ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 5 days.