apache / rocketmq-operator

Apache RocketMQ Operator
https://rocketmq.apache.org/
Apache License 2.0
308 stars 126 forks source link

[ISSUE #202 #159]when the broker uses hostNetwork, the IP registered by the broker to the nameserver is not the podIP #203

Closed usernameisnull closed 5 months ago

usernameisnull commented 6 months ago

when the broker uses hostNetwork, the IP registered by the broker to the nameserver is not the podIP related #202 #159

What is the purpose of the change

when the broker uses hostNetwork, the broker's IP returned by NameServer is the IP of the k8s node where the broker is located.

This is because the broker uses the cluster ip(or other ip) instead of its own pod ip when registering.

The root cause is that when using hostnetwork, the host's network is used, including /etc/hosts. When generating the brokerIP1configuration project, use the hostname -i command in the images/broker/alpine/brokerGenConfig.sh to obtain the ip.

/images/broker/alpine/brokerGenConfig.sh#L30

echo "brokerIP1=`hostname -i`" >> $BROKER_CONFIG_FILE

but in many cases this command does not correctly obtain the correct ip. for exampels:

#/etc/hosts
127.0.0.1 localhost
127.0.1.1 master01

bash-4.4# hostname -i 127.0.1.1

when use 127.0.1.1 to register, the nameserver will select the IP based on its own algorithm. Most of the time it is not the status.podIP of the pod we want: https://github.com/apache/rocketmq/blob/2c898c9b31bf195174cf1e3a626a7c61f7576381/common/src/main/java/org/apache/rocketmq/common/utils/NetworkUtil.java#L91

#/etc/hosts
127.0.0.1 localhost
127.0.1.1 master01
10.6.178.178 master01

bash-4.4# hostname -i 127.0.1.1 10.6.178.178

check the log, it seems to be successful image

but when we used the mqadmin command to check the registered broker address, we found that this was not the case. image

Brief changelog

When setting up with hostNetwork option, we explicitly add an environment variable HOST_NETWORK_POD_IP on the pod. When obtaining brokerIP1, we obtain it from this environment variable and register it with the nameserver.

Verifying this change

the log from the pod: image

the podIP: image

image

use mqadmin see the broker address: image

Please go through this checklist to help us incorporate your contribution quickly and easily.

Notice: It would be helpful if you could finish the following checklist (the last one is not necessary) before request the community to review your PR.

usernameisnull commented 6 months ago

PTAL @caigy

shangjin92 commented 6 months ago
BROKER_NETWORK_DEV=`/sbin/route |awk '/^default/{print $NF}'`
BROKER_IP=`ip a s ${BROKER_NETWORK_DEV} |awk -F' +|/' '/ inet /{print $3}'`
echo "brokerIP1=${BROKER_IP}" >> $BROKER_CONFIG_FILE

You can try this instead, that doesn't require manual IP specification, it works for me.

usernameisnull commented 6 months ago
BROKER_NETWORK_DEV=`/sbin/route |awk '/^default/{print $NF}'`
BROKER_IP=`ip a s ${BROKER_NETWORK_DEV} |awk -F' +|/' '/ inet /{print $3}'`
echo "brokerIP1=${BROKER_IP}" >> $BROKER_CONFIG_FILE

You can try this instead, that doesn't require manual IP specification, it works for me.

thank you, I will try this

usernameisnull commented 5 months ago

PTAL @caigy