search

canonical / microceph

Ceph for a one-rack cluster and appliances
https://snapcraft.io/microceph
GNU Affero General Public License v3.0
193 stars 27 forks source link

Unable to set microcephd ip address #303

Closed krombel closed 4 months ago

krombel commented 4 months ago

Issue report

I cannot set microcephd listening ip when using microceph cluster bootstrap

What version of MicroCeph are you using ?

microceph (reef/stable) 18.2.0+snap3f8909a69a

What are the steps to reproduce this issue ?

microceph cluster bootstrap --mon-ip 10.165.2.8 --public-network 10.165.2.0/24 --cluster-network 10.165.2.0/24

What happens (observed behaviour) ?

root@vm-1:~# ss -ltunp                                                             
Netid State  Recv-Q Send-Q     Local Address:Port  Peer Address:PortProcess                                                                                    
tcp   LISTEN 0      512           10.165.2.8:6800       0.0.0.0:*    users:(("ceph-mds",pid=1392829,fd=18))       
tcp   LISTEN 0      512           10.165.2.8:6801       0.0.0.0:*    users:(("ceph-mds",pid=1392829,fd=19))       
tcp   LISTEN 0      512           10.165.2.8:6802       0.0.0.0:*    users:(("ceph-mgr",pid=1392722,fd=27))       
tcp   LISTEN 0      512           10.165.2.8:6803       0.0.0.0:*    users:(("ceph-mgr",pid=1392722,fd=28))       
tcp   LISTEN 0      4096      <my-public-ip>:7443       0.0.0.0:*    users:(("microcephd",pid=1392402,fd=9))
tcp   LISTEN 0      512           10.165.2.8:3300       0.0.0.0:*    users:(("ceph-mon",pid=1392604,fd=27))       
tcp   LISTEN 0      512           10.165.2.8:6789       0.0.0.0:*    users:(("ceph-mon",pid=1392604,fd=28))

What were you expecting to happen ?

root@vm-1:~# ss -ltunp                                                             
Netid State  Recv-Q Send-Q     Local Address:Port  Peer Address:PortProcess                                                                                    
tcp   LISTEN 0      512           10.165.2.8:6800       0.0.0.0:*    users:(("ceph-mds",pid=1392829,fd=18))       
tcp   LISTEN 0      512           10.165.2.8:6801       0.0.0.0:*    users:(("ceph-mds",pid=1392829,fd=19))       
tcp   LISTEN 0      512           10.165.2.8:6802       0.0.0.0:*    users:(("ceph-mgr",pid=1392722,fd=27))       
tcp   LISTEN 0      512           10.165.2.8:6803       0.0.0.0:*    users:(("ceph-mgr",pid=1392722,fd=28))       
tcp   LISTEN 0      4096          10.165.2.8:7443       0.0.0.0:*    users:(("microcephd",pid=1392402,fd=9))
tcp   LISTEN 0      512           10.165.2.8:3300       0.0.0.0:*    users:(("ceph-mon",pid=1392604,fd=27))       
tcp   LISTEN 0      512           10.165.2.8:6789       0.0.0.0:*    users:(("ceph-mon",pid=1392604,fd=28))

Additional comments.

While looking at the bootstrap code it does not seem to be possible to overwrite the microcephd-IP. It only seems to be possible with microceph init in the interactive setup (as this code shows) but there it is not possible to set the ceph ips...

sabaini commented 4 months ago

Thanks @krombel.

As I see it there's two use cases for splitting public/cluster networks

a) reducing attack surface by separating networks b) being able to better manage bandwidth requirements with an extra cluster net

For maximum flexibility I'd suggest to add an extra cli param for the bootstrap command to bind microcephd to a specific net

jsantosa-minsait commented 4 months ago

Thanks for checking this issue.

By using the command microceph init you can set the IP address and the server name if you want. For the server name I have some troubles because the length of the string, since it must be 7-8 characters at most. However the IP seems to works fine and it uses that interface for the public network and mon services.

Regards.