How to registry service to consul

[zh4n7wm]

Hi cneira,

The driver can not support register service to consul

job "neverwinter" {
    datacenters = ["dc1"]
    type        = "service"

    group "nwn-group" {
        network {
            mode = "cni/microvms"
        }

        service {
            name = "nwn-service"
            port = 22
            address_mode = "alloc"
            check {
                type = "tcp"
                interval = "10s"
                timeout = "2s"
                address_mode = "alloc"
            }
        }

        task "nwn-server" {
            driver = "firecracker-task-driver"
            config {
                Vcpus = 1
                KernelImage = "/home/cneira/Development/vmlinuxs/vmlinux"
                BootDisk= "/home/cneira/Development/rootfs/ubuntu/18.04/nwnrootfs.ext4"
                Disks = [ "/home/cneira/Development/disks/disk0.ext4:rw" ]
                Mem = 1000
                Network = "microvms"
            }
        }
    }
}

I modify some code, but it's not work correctly. support-cni-service.txt

note: move support-cni-service.txt support-cni-service.patch

I think I should get the IP Address assigned by group -> network section, then setup taskConfigSpec.Nic.

Could you give me so me advice?

Thx!

[cneira]

I'll take a look next week, if you solve this before let me know.

[zh4n7wm]

I tried but I failed, I can not make it work.

Because I can't get the CNI assigned IP Address.

[cneira]

@ox0spy I have not checked that yet, but I'm assuming is a CNI more related configuration. I want to make a couple of updates to the driver this couple of weeks so I could check this problem at the same time.

[cneira]

@ox0spy is any of the examples working for you?, the network interface should pickup the ip address provided by your cni network configuration.

[cneira]

@ox0spy This is working for me I just installed consul and tried the following

job "consul-example" {
  datacenters = ["dc1"]
  type        = "service"
  group "test" {
    restart {
      attempts = 0
      mode     = "fail"
    }
 service {
            name = "nwn-service"
            port = 22
            check {
                type = "tcp"
                interval = "10s"
                timeout = "5s"
            }
        }
    task "nwn" {
     driver = "firecracker-task-driver"
      config {
       Vcpus = 1 
       Mem = 128
       KernelImage= "/home/cneira/kernel-images/vmlinux.bin"
       BootDisk = "/dev/zvol/vms/centos7"
       Network = "default"
      }
    }
  }
}

and I also deploy consul by nomad

job "consul" {
  datacenters = ["dc1"]  group "consul" {
    count = 1
    task "consul" {
      driver = "raw_exec"

      config {
        command = "consul"
        args    = ["agent", "-dev"]
      }     
    }
  }
}

I followed this guide https://medium.com/hashicorp-engineering/hashicorp-nomad-from-zero-to-wow-1615345aa539 and the service registers on consul.

[cneira]

@ox0spy feel free to re-open if needed.

[zh4n7wm]

@cneira Thanks for your update.

Now also can not support address_mode = "alloc"

cni conf: /etc/cni/conf.d/firecracker.conflist

{
  "name": "firecracker",
  "cniVersion": "0.4.0",
  "plugins": [
    {
      "type": "ptp",
      "ipMasq": true,
      "ipam": {
        "type": "host-local",
        "subnet": "192.168.60.0/24",
        "resolvConf": "/etc/resolv.conf"
      }
    },
    {
      "type": "tc-redirect-tap"
    }
  ]
}

job config

job "hello" {
    datacenters = ["dc1"]
    type = "service"

    group "sshd" {
        network {
            # mode = "cni/mynet"
            port "ssh" {
                to = 22
            }
        }
        service {
            name = "sshd"
            port = "ssh"
            address_mode = "alloc"
            check {
                type = "tcp"
                interval = "10s"
                timeout = "2s"
                address_mode = "alloc"
            }
        }

        task "sshd" {
            driver = "firecracker-task-driver"

            config {
                KernelImage = "/home/ox0spy/projects/nomad/study/firecracker/vmlinux.bin"
                BootDisk = "/home/ox0spy/projects/nomad/study/firecracker/rootfs.ext4"
                Firecracker = "/usr/local/bin/firecracker"
                Vcpus       = 1
                Mem         = 128
                Network     = "firecracker"
            }
        }
    }
}

docs for address_mode in service block: https://www.nomadproject.io/docs/job-specification/service#address_mode

run job

nomad status <alloc-id> got the below error message:

Setup Failure  failed to setup alloc: pre-run hook "group_services" failed: unable to get address for service "sshd": cannot use address_mode="alloc": no allocation network status reported