Open neiba opened 1 year ago
Sorry, I fail to reproduce the problem in my environment. Could you please provide more information including cloudcore.yaml
, edgecore.yaml
and the yaml of your cm, pod?
cloudhub would check objectsync ObjectResourceVersion with cm-1‘s ResourceVersion to decide send msg to edge or not.
cloudcore.yaml
apiVersion: cloudcore.config.kubeedge.io/v1alpha1
commonConfig:
monitorServer:
bindAddress: 127.0.0.1:9091
tunnelPort: 10350
kind: CloudCore
kubeAPIConfig:
burst: 5000
contentType: application/vnd.kubernetes.protobuf
kubeConfig: /root/.kube/config
master: ""
qps: 2500
modules:
cloudHub:
advertiseAddress:
- 192.168.16.128
dnsNames:
- ""
edgeCertSigningDuration: 365
enable: true
https:
address: 0.0.0.0
enable: true
port: 10002
keepaliveInterval: 30
nodeLimit: 500
quic:
address: 0.0.0.0
enable: false
maxIncomingStreams: 10000
port: 10001
tlsCAFile: /tmp/etc/kubeedge/ca/cloudhub/rootCA.crt
tlsCAKeyFile: /tmp/etc/kubeedge/ca/cloudhub/rootCA.key
tlsCertFile: /tmp/etc/kubeedge/certs/cloudhub/server.crt
tlsPrivateKeyFile: /tmp/etc/kubeedge/certs/cloudhub/server.key
tokenRefreshDuration: 12
unixsocket:
address: unix:///tmp/var/lib/kubeedge/kubeedge.sock
enable: true
websocket:
address: 0.0.0.0
enable: true
port: 10000
writeTimeout: 30
cloudStream:
enable: true
streamPort: 10003
tlsStreamCAFile: /tmp/etc/kubeedge/ca/streamCA.crt
tlsStreamCertFile: /tmp/etc/kubeedge/certs/stream.crt
tlsStreamPrivateKeyFile: /tmp/etc/kubeedge/certs/stream.key
tlsTunnelCAFile: /tmp/etc/kubeedge/ca/rootCA.crt
tlsTunnelCertFile: /tmp/etc/kubeedge/certs/server.crt
tlsTunnelPrivateKeyFile: /tmp/etc/kubeedge/certs/server.key
tunnelPort: 10004
deviceController:
buffer:
deviceEvent: 1
deviceModelEvent: 1
updateDeviceStatus: 1024
enable: true
load:
updateDeviceStatusWorkers: 1
dynamicController:
enable: true
edgeController:
buffer:
configMapEvent: 1
createLease: 1524
createNode: 1024
deletePod: 1024
patchNode: 1274
patchPod: 1024
podEvent: 1
queryConfigMap: 1024
queryLease: 1024
queryNode: 1524
queryPersistentVolume: 1024
queryPersistentVolumeClaim: 1024
querySecret: 1024
queryVolumeAttachment: 1024
ruleEndpointsEvent: 1
rulesEvent: 1
secretEvent: 1
serviceAccountToken: 1024
updateNode: 1024
updateNodeStatus: 1024
updatePodStatus: 1024
enable: true
load:
ServiceAccountTokenWorkers: 100
UpdateRuleStatusWorkers: 4
createLeaseWorkers: 500
createNodeWorkers: 100
deletePodWorkers: 100
patchNodeWorkers: 110
patchPodWorkers: 100
queryConfigMapWorkers: 100
queryLeaseWorkers: 100
queryNodeWorkers: 500
queryPersistentVolumeClaimWorkers: 4
queryPersistentVolumeWorkers: 4
querySecretWorkers: 100
queryVolumeAttachmentWorkers: 4
updateNodeStatusWorkers: 1
updateNodeWorkers: 4
updatePodStatusWorkers: 1
nodeUpdateFrequency: 10
iptablesManager:
enable: true
mode: internal
nodeUpgradeJobController:
buffer:
nodeUpgradeJobEvent: 1
updateNodeUpgradeJobStatus: 1024
enable: false
load:
nodeUpgradeJobWorkers: 1
router:
address: 0.0.0.0
enable: true
port: 9443
restTimeout: 60
syncController:
enable: true
edgecore.yaml
apiVersion: edgecore.config.kubeedge.io/v1alpha2
database:
aliasName: default
dataSource: /tmp/var/lib/kubeedge/edgecore.db
driverName: sqlite3
kind: EdgeCore
modules:
dbTest:
enable: false
deviceTwin:
enable: true
edgeHub:
enable: true
heartbeat: 15
httpServer: https://192.168.16.128:10002
messageBurst: 60
messageQPS: 30
projectID: e632aba927ea4ac2b575ec1603d56f10
quic:
enable: false
handshakeTimeout: 30
readDeadline: 15
server: 192.168.16.128:10001
writeDeadline: 15
rotateCertificates: true
tlsCaFile: /tmp/etc/kubeedge/ca/rootCA.crt
tlsCertFile: /tmp/etc/kubeedge/certs/server.crt
tlsPrivateKeyFile: /tmp/etc/kubeedge/certs/server.key
token: ""
websocket:
enable: true
handshakeTimeout: 30
readDeadline: 15
server: 192.168.16.128:10000
writeDeadline: 15
edgeStream:
enable: true
handshakeTimeout: 30
readDeadline: 15
server: 127.0.0.1:10004
tlsTunnelCAFile: /tmp/etc/kubeedge/ca/rootCA.crt
tlsTunnelCertFile: /tmp/etc/kubeedge/certs/server.crt
tlsTunnelPrivateKeyFile: /tmp/etc/kubeedge/certs/server.key
writeDeadline: 15
edged:
cniBinDir: /opt/cni/bin
cniCacheDir: /var/lib/cni/cache
cniConfDir: /tmp/etc/cni/net.d
containerRuntime: docker
enable: true
hostnameOverride: edge-node
imagePullProgressDeadline: 1m0s
masterServiceNamespace: default
maxContainerCount: -1
maxPerPodContainerCount: 1
minimumGCAge: 0s
networkPluginMTU: 1500
nodeIP: 192.168.16.128
nonMasqueradeCidr: 10.0.0.0/8
podSandboxImage: kubeedge/pause:3.6
registerNode: true
registerNodeNamespace: default
registerSchedulable: true
remoteImageEndpoint: unix:///var/run/dockershim.sock
remoteRuntimeEndpoint: unix:///var/run/dockershim.sock
rootDirectory: /var/lib/edged
seccompProfileRoot: /var/lib/edged/seccomp
tailoredKubeletConfig:
address: 127.0.0.1
cgroupDriver: cgroupfs
cgroupsPerQOS: true
clusterDomain: cluster.local
configMapAndSecretChangeDetectionStrategy: Get
containerLogMaxFiles: 5
containerLogMaxSize: 10Mi
contentType: application/json
cpuCFSQuota: true
cpuCFSQuotaPeriod: 100ms
cpuManagerPolicy: none
cpuManagerReconcilePeriod: 10s
enableControllerAttachDetach: true
enableDebugFlagsHandler: true
enableDebuggingHandlers: true
enableProfilingHandler: true
enableSystemLogHandler: true
enforceNodeAllocatable:
- pods
evictionHard:
imagefs.available: 15%
memory.available: 100Mi
nodefs.available: 10%
nodefs.inodesFree: 5%
evictionPressureTransitionPeriod: 5m0s
failSwapOn: false
hairpinMode: promiscuous-bridge
imageGCHighThresholdPercent: 80
imageGCLowThresholdPercent: 40
imageMinimumGCAge: 720h0m0s
iptablesDropBit: 15
iptablesMasqueradeBit: 14
logging:
flushFrequency: 5000000000
format: text
options:
json:
infoBufferSize: "0"
verbosity: 0
makeIPTablesUtilChains: true
maxOpenFiles: 1000000
maxPods: 110
memoryManagerPolicy: None
memorySwap: {}
memoryThrottlingFactor: 0.8
nodeLeaseDurationSeconds: 40
nodeStatusMaxImages: 50
nodeStatusReportFrequency: 5m0s
nodeStatusUpdateFrequency: 10s
oomScoreAdj: -999
podPidsLimit: -1
readOnlyPort: 10350
registerNode: true
registryBurst: 10
registryPullQPS: 5
resolvConf: /etc/resolv.conf
runtimeRequestTimeout: 2m0s
seccompDefault: false
serializeImagePulls: true
shutdownGracePeriod: 0s
shutdownGracePeriodCriticalPods: 0s
streamingConnectionIdleTimeout: 4h0m0s
syncFrequency: 1m0s
topologyManagerPolicy: none
topologyManagerScope: container
volumePluginDir: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/
volumeStatsAggPeriod: 1m0s
eventBus:
enable: true
eventBusTLS:
enable: false
tlsMqttCAFile: /tmp/etc/kubeedge/ca/rootCA.crt
tlsMqttCertFile: /tmp/etc/kubeedge/certs/server.crt
tlsMqttPrivateKeyFile: /tmp/etc/kubeedge/certs/server.key
mqttMode: 0
mqttPassword: ""
mqttPubClientID: ""
mqttQOS: 0
mqttRetain: false
mqttServerExternal: tcp://127.0.0.1:1883
mqttServerInternal: tcp://127.0.0.1:1884
mqttSessionQueueSize: 100
mqttSubClientID: ""
mqttUsername: ""
metaManager:
contextSendGroup: hub
contextSendModule: websocket
enable: true
metaServer:
enable: true
server: 127.0.0.1:10550
tlsCaFile: /etc/kubeedge/ca/rootCA.crt
tlsCertFile: /tmp/etc/kubeedge/certs/server.crt
tlsPrivateKeyFile: /tmp/etc/kubeedge/certs/server.key
remoteQueryTimeout: 60
serviceBus:
enable: true
port: 9060
server: 127.0.0.1
timeout: 60
pod and configmap yaml:
---
apiVersion: v1
kind: ConfigMap
metadata:
name: cm-1
data:
test.txt: |
test
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: node-role.kubernetes.io/edge
operator: Exists
containers:
- name: nginx
image: nginx:1.14.2
volumeMounts:
- name: vol-1
mountPath: /vol-1
ports:
- containerPort: 80
hostPort: 8080
volumes:
- name: vol-1
configMap:
name: cm-1
Why path of metaManager.metaServer.tlsCaFile
in egdecore.yaml is not save with others? I'm not sure if it caused the problem.
Why path of
metaManager.metaServer.tlsCaFile
in egdecore.yaml is not save with others? I'm not sure if it caused the problem.
metaserver is working, curl without watch is ok. I think this is cloudcore problem, cloudhub would check objectsync ObjectResourceVersion with cm-1‘s ResourceVersion to decide send msg to edge or not.
cc @wackxu
What happened: curl metaserver with watch but no response
What you expected to happen: curl metaserver with watch, and get response
How to reproduce it (as minimally and precisely as possible):
expected get cm-1, but get none.
Anything else we need to know?:
Environment:
kubectl version
): 1.24cloudcore --version
andedgecore --version
): 1.13Cloud nodes Environment:
- Hardware configuration (e.g. `lscpu`): - OS (e.g. `cat /etc/os-release`): - Kernel (e.g. `uname -a`): - Go version (e.g. `go version`): - Others:Edge nodes Environment:
- edgecore version (e.g. `edgecore --version`): - Hardware configuration (e.g. `lscpu`): - OS (e.g. `cat /etc/os-release`): - Kernel (e.g. `uname -a`): - Go version (e.g. `go version`): - Others: