"Wrong" warning message in Openstack clouds: nomad.rpc: no path found to region: region=<region-name>

[neuroserve]

Nomad version

root@nomad-prod1-0:~/nomad-jobs# nomad version
Nomad v1.8.3
BuildDate 2024-08-13T07:37:30Z
Revision 63b636e5cbaca312cf6ea63e040f445f05f00478

Operating system and Environment details

root@nomad-prod1-0:~/nomad-jobs# cat /etc/debian_version 
11.10
root@nomad-prod1-0:~/nomad-jobs# uname -ra
Linux nomad-prod1-0 5.10.0-28-cloud-amd64 #1 SMP Debian 5.10.209-2 (2024-01-31) x86_64 GNU/Linux

Nomad is used inside an Openstack environment. The servers use the retry_join statement to join the cluster:

server {
  enabled          = true
  encrypt          = "<some-encrypt-string>" 
  bootstrap_expect = 3
  server_join {
    retry_join     = [ "provider=os tag_key=nomad-role tag_value=server auth_url=<some-auth-url:5000> user_name=<someusername> domain_name=<somedomainname> password="somepassword" region=an-openstack-region" ] 
    retry_interval = "15s"
  }
}

Issue

Using the openstack-region in the retry_join statement creates a warning in the Nomad log:

Aug 21 11:09:18 nomad-prod1-0 nomad[12667]:     2024-08-21T11:09:18.064Z [WARN]  nomad.rpc: no path found to region: region=someregion
Aug 21 11:09:18 nomad-prod1-0 nomad[12667]:  nomad.rpc: no path found to region: region=someregion

Reproduction steps

Use the retry_join statement in a Nomad configuration in an Openstack environment and use the Openstack region in it. That will create the above warnings.

Expected Result

The use of the Openstack region should not create warning messages.

Actual Result

The use of the Openstack region creates a warning message in the Nomad log.

Nomad Server logs (if appropriate)

S. above.

[jrasell]

Hi @neuroserve and thanks for raising this issue.

The OpenStack go-discovery configuration options do not have a region key and quickly looking at the code I believe the help options to be accurate.

The log messages you detailed are coming from Nomad's internal RPC handler which go-discover does not use. I wonder if the discovery process is finding a server that has a region configuration different to what is expected, and therefore when it joins it attempts to find a region that is not available? If you have any additional information, please let me know.

I don't have the ability to spin up an OpenStack cluster at the moment, to properly test this out myself.

[neuroserve]

Oh - I didn't look into the code. I thought, all authentication parameters would be available. Like

      auth_url: 
      username: 
      project_id: 
      project_name:
      user_domain_name: 
    region_name: 
    interface: 
    identity_api_version: 

I'll try to create a cluster without the region key and see, whether it changes anything. I hope, that there is no server with a different region-setting, as they are deployed via terraform.