search

eclipse / amlen

Message Broker for IoT/Mobile/Web. Mainly uses MQTT v3.x and v5. Aims to be easy to use, scalable and reliable
Eclipse Public License 2.0
46 stars 11 forks source link

Amlen fail to maintenance after switch and restart #143

Open jochen123456 opened 10 months ago

jochen123456 commented 10 months ago

Hi,

I have amlen installed within kubernetes in active/passive. It works fine except in one cluster. I think ot has something to do with the nfs volumes.

  1. The psb-messagesight-0 goes to Maintenance after Restart of Pods. psb-messagesight-1 waits for sync.
  2. I clean the store psb-messagesight-0. Now psb-messagesight-0 active.
  3. I delete pod psb-messagesight-0. psb-messagesight-0 and psb-messagesight-1 goes to maintenance.
  4. I clean the store psb-messagesight-0 and psb-messagesight-1. Now psb-messagesight-0 is active.

Can anyone help and point out what is the problem with the storage,

x.x.109.18:/psb_FS/psb-qa-psb-imaserver-data-psb-messagesight-0-pvc-7f22f362-4c70-4952-9186-04274e51188a on /var/lib/amlen-server type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=x.x.184.20,local_lock=none,addr= x.x.109.18) x.x.109.18:/psb_FS/psb-qa-psb-imaserver-log-psb-messagesight-0-pvc-ee319c4d-2a82-41ec-abf0-43d29834b853 on /var/lib/amlen-server/diag type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=x.x.184.20,local_lock=none,a ddr=x.x.109.18) x.x.109.18:/psb_FS/psb-qa-psb-imaserver-data-psb-messagesight-1-pvc-34925f78-0a0c-45d2-b66f-531b036712a8 on /var/lib/amlen-server type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=x.x.184.21,local_lock=none,addr= x.x.109.18) x.x.109.18:/psb_FS/psb-qa-psb-imaserver-log-psb-messagesight-1-pvc-79e99cfb-443d-45b1-81a4-4e7f1f1dd87d on /var/lib/amlen-server/diag type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=x.x.184.21,local_lock=none,a ddr=x.x.109.18)

jochen123456 commented 10 months ago

[Uploading psb-messagesight-0-imatrace.log…]()

jochen123456 commented 10 months ago

psb-messagesight-1-imatrace.log

jonquark commented 10 months ago

I was going to take a quick look at this, this morning - but weirdly, the "link" to the messagesight-0 log is a link to the issue noto to the file itself and I'd want logs from both machines (especially including a time that it goes into maintenance)