search

vatesfr / xen-orchestra

The global orchestration solution to manage and backup XCP-ng and XenServer.
https://xen-orchestra.com
Other
793 stars 265 forks source link

Error on sdnController.createPrivateNetwork #8027

Open gkirk90 opened 1 month ago

gkirk90 commented 1 month ago

Are you using XOA or XO from the sources?

XO from the sources

Which release channel?

stable

Provide your commit number

cc3d5

Describe the bug

Error when trying to create a private network. Appears no PIF is found on 1 of my 7 hosts. All 6 others are fine.

Error message

{
  "generatedMessage": false,
  "code": "ERR_ASSERTION",
  "actual": false,
  "expected": true,
  "operator": "==",
  "message": "No PIF found",
  "name": "AssertionError",
  "stack": "AssertionError [ERR_ASSERTION]: No PIF found\n    at PrivateNetwork.addHost (/opt/xen-orchestra/packages/xo-server-sdn-controller/src/private-network/private-network.js:70:11)\n    at map (/opt/xen-orchestra/packages/xo-server-sdn-controller/src/private-network/private-network.js:149:57)\n    at Array.map (<anonymous>)\n    at PrivateNetwork.electNewCenter (/opt/xen-orchestra/packages/xo-server-sdn-controller/src/private-network/private-network.js:149:40)\n    at runNextTicks (node:internal/process/task_queues:60:5)\n    at processImmediate (node:internal/timers:454:9)\n    at process.callbackTrampoline (node:internal/async_hooks:130:17)\n    at SDNController._createPrivateNetwork (/opt/xen-orchestra/packages/xo-server-sdn-controller/src/index.js:770:7)\n    at Task.runInside (/opt/xen-orchestra/@vates/task/index.js:169:22)\n    at Task.run (/opt/xen-orchestra/@vates/task/index.js:153:20)\n    at Api.#callApiMethod (file:///opt/xen-orchestra/packages/xo-server/src/xo-mixins/api.mjs:389:20)"
}

To reproduce

  1. Go to '...'
  2. Click on '...'
  3. Scroll down to '...'
  4. See error

Expected behavior

New Private network connected without error.

Screenshots

No response

Node

16.12.1

Hypervisor

XCP-ng 8.2.1

Additional context

No response

olivierlambert commented 1 month ago

Hi,

  1. Please follow instructions here first: https://xen-orchestra.com/docs/community.html (ie get on latest commit first).
  2. Check that you have the same number of PIFs/same NIC order between your hosts in your pool (assuming they are similar machines?)

Adding @benjamreis in the loop here.

gkirk90 commented 1 month ago

Updated to the latest commit a609e, issue persists.

All 7 hosts are exactly the same. They are blades running on a UCS running the same profile.

Are there commands I can run to get the PIF list that the host is seeing?

From: Olivier Lambert @.> Sent: Monday, September 30, 2024 11:57 AM To: vatesfr/xen-orchestra @.> Cc: Gary Kirk @.>; Author @.> Subject: Re: [vatesfr/xen-orchestra] Error on sdnController.createPrivateNetwork (Issue #8027)

Hi,

  1. Please follow instructions here first: https://xen-orchestra.com/docs/community.html (ie get on latest commit first).
  2. Check that you have the same number of PIFs/same NIC order between your hosts in your pool (assuming they are similar machines?)

Adding @benjamreishttps://github.com/benjamreis in the loop here.

— Reply to this email directly, view it on GitHubhttps://github.com/vatesfr/xen-orchestra/issues/8027#issuecomment-2383588256, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AE2LK66K5LKX2W22IRLHBCTZZFYD3AVCNFSM6AAAAABPDTMDCGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBTGU4DQMRVGY. You are receiving this because you authored the thread.Message ID: @.**@.>>

benjamreis commented 1 month ago

Hello!

1st things 1st: can you check that the PIFs are attached to the host? To list the PIFs you need to run xe pif-list network-uuid=<uuid of network of selected PIF>

gkirk90 commented 1 month ago

So running the pif-list on the network the private networks are attached to I get the following: (this appears correct to me as the 6 currently created private networks +the queried network. All stating connected)

[10:56 dlpaxcp07 ~]# xe pif-list network-uuid=3e31e4d5-ffb8-5ca9-fae6-cbefede5259a
uuid ( RO)                  : 41325c78-3ed2-6561-2e1d-89361765ee82
                device ( RO): bond1
    currently-attached ( RO): true
                  VLAN ( RO): -1
          network-uuid ( RO): 3e31e4d5-ffb8-5ca9-fae6-cbefede5259a

uuid ( RO)                  : f16667ae-3865-74a6-097b-7545af68cb80
                device ( RO): bond1
    currently-attached ( RO): true
                  VLAN ( RO): -1
          network-uuid ( RO): 3e31e4d5-ffb8-5ca9-fae6-cbefede5259a

uuid ( RO)                  : 693fb26b-bc4e-41d5-48c7-ed6ffd0d856f
                device ( RO): bond0
    currently-attached ( RO): true
                  VLAN ( RO): -1
          network-uuid ( RO): 3e31e4d5-ffb8-5ca9-fae6-cbefede5259a

uuid ( RO)                  : bc4d2cf4-8f3b-c8ce-09f1-c1c27624a8a1
                device ( RO): bond1
    currently-attached ( RO): true
                  VLAN ( RO): -1
          network-uuid ( RO): 3e31e4d5-ffb8-5ca9-fae6-cbefede5259a

uuid ( RO)                  : 82607768-41a0-53e0-7cc6-77b5812efcf6
                device ( RO): bond1
    currently-attached ( RO): true
                  VLAN ( RO): -1
          network-uuid ( RO): 3e31e4d5-ffb8-5ca9-fae6-cbefede5259a

uuid ( RO)                  : bab19ca8-54ed-4e57-43da-8887b7df5010
                device ( RO): bond1
    currently-attached ( RO): true
                  VLAN ( RO): -1
          network-uuid ( RO): 3e31e4d5-ffb8-5ca9-fae6-cbefede5259a

uuid ( RO)                  : 1a54cc15-12e5-9046-05f5-e59771acba20
                device ( RO): bond1
    currently-attached ( RO): true
                  VLAN ( RO): -1
          network-uuid ( RO): 3e31e4d5-ffb8-5ca9-fae6-cbefede5259a

From: BenjiReis @.> Sent: Tuesday, October 1, 2024 2:37 AM To: vatesfr/xen-orchestra @.> Cc: Gary Kirk @.>; Author @.> Subject: Re: [vatesfr/xen-orchestra] Error on sdnController.createPrivateNetwork (Issue #8027)

Hello!

1st things 1st: can you check that the PIFs are attached to the host? To list the PIFs you need to run xe pif-list network-uuid=

— Reply to this email directly, view it on GitHubhttps://github.com/vatesfr/xen-orchestra/issues/8027#issuecomment-2384917353, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AE2LK64O5T2JBO3RY56X4WLZZI7JTAVCNFSM6AAAAABPDTMDCGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBUHEYTOMZVGM. You are receiving this because you authored the thread.Message ID: @.**@.>>

benjamreis commented 1 month ago

Are the PIFs' IP configured or does the pif on the 7th host have no IP configuration? There should be more info in error in XO.

gkirk90 commented 1 month ago

Yes, the PIF on the 7th does have an IP configured as well.

Here is the screenshot showing IP assignments on this NIC (notice that the one on XCP07 says device is Bond0 when the others say Bond1, they are all set up the same)

@.***

When I create a private network this is the error I get from this host:

sdnController.createPrivateNetwork { "poolIds": [ "f88d3c37-bda9-8db8-f484-6567ea570d54" ], "pifIds": [ "4c06358d-8481-ef8d-2926-36cf35461e68" ], "name": "Corp-FIrstLightEnergy", "description": "Corp FIrstLightEnergy", "encapsulation": "gre", "encrypted": false, "preferredCenterId": "bef286cd-4eb4-4c34-b3be-aa140752cf8e" } { "generatedMessage": false, "code": "ERR_ASSERTION", "actual": false, "expected": true, "operator": "==", "message": "No PIF found", "name": "AssertionError", "stack": "AssertionError [ERR_ASSERTION]: No PIF found at PrivateNetwork.addHost (/opt/xen-orchestra/packages/xo-server-sdn-controller/src/private-network/private-network.js:70:11) at map (/opt/xen-orchestra/packages/xo-server-sdn-controller/src/private-network/private-network.js:149:57) at Array.map () at PrivateNetwork.electNewCenter (/opt/xen-orchestra/packages/xo-server-sdn-controller/src/private-network/private-network.js:149:40) at runNextTicks (node:internal/process/task_queues:60:5) at processImmediate (node:internal/timers:454:9) at process.callbackTrampoline (node:internal/async_hooks:130:17) at SDNController._createPrivateNetwork (/opt/xen-orchestra/packages/xo-server-sdn-controller/src/index.js:770:7) at Task.runInside @./task/index.js:169:22) at Task.run @./task/index.js:153:20) at Api.#callApiMethod (file:///opt/xen-orchestra/packages/xo-server/src/xo-mixins/api.mjs:389:20)" }

From: BenjiReis @.> Sent: Wednesday, October 2, 2024 3:37 AM To: vatesfr/xen-orchestra @.> Cc: Gary Kirk @.>; Author @.> Subject: Re: [vatesfr/xen-orchestra] Error on sdnController.createPrivateNetwork (Issue #8027)

Are the PIFs' IP configured or does the pif on the 7th host have no IP configuration? There should be more info in error in XO.

— Reply to this email directly, view it on GitHubhttps://github.com/vatesfr/xen-orchestra/issues/8027#issuecomment-2387816232, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AE2LK63KGPSQN2KE6MKWMZDZZOPCVAVCNFSM6AAAAABPDTMDCGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBXHAYTMMRTGI. You are receiving this because you authored the thread.Message ID: @.**@.>>

gkirk90 commented 1 month ago

In addition, here is the screenshot from XCP-NG-Center (I know this is unsupported now but it does give a good view of this configuration).

@.***

From: BenjiReis @.> Sent: Wednesday, October 2, 2024 3:37 AM To: vatesfr/xen-orchestra @.> Cc: Gary Kirk @.>; Author @.> Subject: Re: [vatesfr/xen-orchestra] Error on sdnController.createPrivateNetwork (Issue #8027)

Are the PIFs' IP configured or does the pif on the 7th host have no IP configuration? There should be more info in error in XO.

— Reply to this email directly, view it on GitHubhttps://github.com/vatesfr/xen-orchestra/issues/8027#issuecomment-2387816232, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AE2LK63KGPSQN2KE6MKWMZDZZOPCVAVCNFSM6AAAAABPDTMDCGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBXHAYTMMRTGI. You are receiving this because you authored the thread.Message ID: @.**@.>>

benjamreis commented 1 month ago

the issue is the bond0 thing indeed. because the device needs to be the same on all hosts.

gkirk90 commented 1 month ago

That’s what I figured. How can I correct it?

From: BenjiReis @.> Sent: Wednesday, October 2, 2024 8:03 AM To: vatesfr/xen-orchestra @.> Cc: Gary Kirk @.>; Author @.> Subject: Re: [vatesfr/xen-orchestra] Error on sdnController.createPrivateNetwork (Issue #8027)

the issue is the bond0 thing indeed. because the device needs to be the same on all hosts.

— Reply to this email directly, view it on GitHubhttps://github.com/vatesfr/xen-orchestra/issues/8027#issuecomment-2388470120, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AE2LK6Y6EOLDABTMH72BMBDZZPOGPAVCNFSM6AAAAABPDTMDCGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBYGQ3TAMJSGA. You are receiving this because you authored the thread.Message ID: @.**@.>>

benjamreis commented 1 month ago

Hmm since the device is read-only in a PIf IDK except from recreating the network. But I don't unbderstand how one host has bond0 and the other have bond1.

gkirk90 commented 1 month ago

I don’t understand either 😊 And I have recreated it a number of times.

Especially weird as these servers are UCS blades using exactly the same profile so the NICs have to be the same type, name and order.

From: BenjiReis @.> Sent: Wednesday, October 2, 2024 8:14 AM To: vatesfr/xen-orchestra @.> Cc: Gary Kirk @.>; Author @.> Subject: Re: [vatesfr/xen-orchestra] Error on sdnController.createPrivateNetwork (Issue #8027)

Hmm since the device is read-only in a PIf IDK except from recreating the network. But I don't unbderstand how one host has bond0 and the other have bond1.

— Reply to this email directly, view it on GitHubhttps://github.com/vatesfr/xen-orchestra/issues/8027#issuecomment-2388493689, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AE2LK64VVRY5RNWE73RUSWDZZPPPNAVCNFSM6AAAAABPDTMDCGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBYGQ4TGNRYHE. You are receiving this because you authored the thread.Message ID: @.**@.>>

benjamreis commented 1 month ago

the naming bondX comes from XAPI maybe the 7th host was joined later in the pool? and there already was a bond0 on the other? Do the other hosts have a bond0 now on another network?

gkirk90 commented 1 month ago

So the 7th host was definitely joined later on. Maybe a month after the initial creation of the original pool of 6 so there was definitely a bond0 on those 6.

As you can see from this screenshot the bond0 is reserved for the management network (assume I’m answering the right question). Here is a sample from XCP02 (good host) following by XCP07 (the one that isn’t working). You can clearly see the device names are incorrect for the associated networks.

Could it be to do with the order the bonds were created on XCP07? created in a different order to the first 6?

@.***

@.***

From: BenjiReis @.> Sent: Wednesday, October 2, 2024 8:21 AM To: vatesfr/xen-orchestra @.> Cc: Gary Kirk @.>; Author @.> Subject: Re: [vatesfr/xen-orchestra] Error on sdnController.createPrivateNetwork (Issue #8027)

the naming bondX comes from XAPI maybe the 7th host was joined later in the pool? and there already was a bond0 on the other? Do the other hosts have a bond0 now on another network?

— Reply to this email directly, view it on GitHubhttps://github.com/vatesfr/xen-orchestra/issues/8027#issuecomment-2388506854, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AE2LK642HRFEUIIY65ASYB3ZZPQKVAVCNFSM6AAAAABPDTMDCGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBYGUYDMOBVGQ. You are receiving this because you authored the thread.Message ID: @.**@.>>