systemd: Error loading unit file: `system.Error.ENOEXEC`

[dx0x58]

Hey! I have a playbook that looks like this:

---
- hosts: redash
  become: yes
  remote_user: ubuntu
  vars:
    cf_cert_content: "{{ vault_redash_cf_tunnel_cert }}"
    cf_tunnels:
      redash:
        account_tag: "{{ vault_redash_cf_tunnel_account_tag }}"
        tunnel_secret: "{{ vault_redash_cf_tunnel_secret }}"
        tunnel_id: "{{ vault_redash_cf_tunnel_id }}"
        routes:
          dns:
          - "redash.internal"
        ingress:
        - hostname: "redash.internal.mydomain.com"
          service: http://localhost:80 # redash local webserver
  roles:
    - redash
  roles:
    - papanito.cloudflared

and after running I get the error:

Is this line the problem ? https://github.com/papanito/ansible-role-cloudflared/blob/0f55bb79322176f6b218c864b5a62c8227b5815e/tasks/configure_systemd.yml#L8

Perhaps we need to use the name of the service here (like dest: "{{ cf_systemd_target_dir }}/{{ systemd_filename }}@ {{ **cf_tunnel.key** }}.service")?

There is only cloudflared@.service file on my server, but not cloudflared@redash as expected

[papanito]

There is only cloudflared@.service file on my server

That is fine cause it's a template - see #3.

I will have a look what the issues might be

[dx0x58]

Thank you! It also seems that the template for systemd should have the absolute path to the executable file

[papanito]

Well the error indicates there is something wrong with the Exec in the service file l can you post the content of the created file (not screenshot).

Not sure if an absolute path is needed, as default install directory is /usr/bin. I never had problems but might be worth checking it.

[papanito]

I think I will have time tomorrow to look into it

[papanito]

What distribution are you using? How is cloudflare installed? Can you do a which cloudflared and a tell me your PATH

[dx0x58]

I use cloudflared-stable-linux-amd64 from my repo cf_download_baseurl: https://github.com/dx0x58/cloudflared-bin/raw/master/

My distro:

ubuntu@ip-10-0-28-18:~$ cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.3 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.3 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

It is state after run playbook:

[dx0x58]

if I change the line

dest: "{{cf_systemd_target_dir}} / {{systemd_filename}} @ {{cf_tunnel.key}}.service"

then the unit will start, but there will be an error:

[papanito]

As mentioned above I suspect a problem with the path. Can you check where the daemon is installed. And paste me the content of the service file please.

[dx0x58]

which cloudflared does not return anything. That's all that was found by name cloudflared:

root@ip-10-0-28-18:/# find . -name "*cloudflared*"
./root/.cloudflared
./sys/kernel/slab/:A-0000192/cgroup/cred_jar(2124:system-cloudflared.slice)
./sys/kernel/slab/:A-0001024/cgroup/PING(2124:system-cloudflared.slice)
./sys/kernel/slab/sock_inode_cache/cgroup/sock_inode_cache(2124:system-cloudflared.slice)
./sys/kernel/slab/kmalloc-64/cgroup/kmalloc-64(2124:system-cloudflared.slice)
./sys/kernel/slab/dentry/cgroup/dentry(2124:system-cloudflared.slice)
./sys/kernel/slab/kmalloc-1k/cgroup/kmalloc-1k(2124:system-cloudflared.slice)
./sys/kernel/slab/inode_cache/cgroup/inode_cache(2124:system-cloudflared.slice)
./sys/kernel/slab/skbuff_head_cache/cgroup/skbuff_head_cache(2124:system-cloudflared.slice)
./sys/kernel/slab/:A-0000256/cgroup/filp(2124:system-cloudflared.slice)
./sys/kernel/slab/:A-0000128/cgroup/pid(2124:system-cloudflared.slice)
./sys/fs/cgroup/devices/system.slice/system-cloudflared.slice
./sys/fs/cgroup/memory/system.slice/system-cloudflared.slice
./sys/fs/cgroup/blkio/system.slice/system-cloudflared.slice
./sys/fs/cgroup/pids/system.slice/system-cloudflared.slice
./sys/fs/cgroup/cpu,cpuacct/system.slice/system-cloudflared.slice
./sys/fs/cgroup/systemd/system.slice/system-cloudflared.slice
./sys/fs/cgroup/unified/system.slice/system-cloudflared.slice
./usr/bin/cloudflared-stable-linux-amd64
./usr/bin/._cloudflared-stable-linux-amd64
./etc/systemd/system/cloudflared@redash.service
./etc/systemd/system/multi-user.target.wants/cloudflared@redash.service
./etc/cloudflared
./var/lib/lxcfs/cgroup/devices/system.slice/system-cloudflared.slice
./var/lib/lxcfs/cgroup/memory/system.slice/system-cloudflared.slice
./var/lib/lxcfs/cgroup/blkio/system.slice/system-cloudflared.slice
./var/lib/lxcfs/cgroup/pids/system.slice/system-cloudflared.slice
./var/lib/lxcfs/cgroup/cpu,cpuacct/system.slice/system-cloudflared.slice
./var/lib/lxcfs/cgroup/name=systemd/system.slice/system-cloudflared.slice

Daemon content:

root@ip-10-0-28-18:/# cat /etc/systemd/system/cloudflared@redash.service
[Unit]
Description=cloudflared tunnel for %I
After=network.target

[Service]
Type=simple
ExecStart=cloudflared tunnel --config /etc/cloudflared/%i.yml run %i
User=root
Group=root

Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.targetroot

[papanito]

The problem is that your binary name is /usr/bin/cloudflared-stable-linux-amd64 while ExecStart is expecting cloudflared

[dx0x58]

@papanito Hi! I pulled the new changes from the repository and now I have a cloudflared-stable-linux-amd64.tgz archive which contains the cloudflared binary, but i still get the error

ubuntu@ip-10-0-28-18:~$ ls -la /etc/systemd/system/ | egrep "*cloud*"
drwxr-xr-x  2 root root   4096 Oct 21  2019 cloud-final.service.wants
drwxr-xr-x  2 root root   4096 Oct 21  2019 cloud-init.target.wants
-rw-r--r--  1 root root    250 Dec  2 10:29 cloudflared@.service

ubuntu@ip-10-0-28-18:~$ cat /etc/systemd/system/cloudflared@.service
[Unit]
Description=cloudflared tunnel for %I
After=network.target

[Service]
Type=simple
ExecStart=cloudflared tunnel --config /etc/cloudflared/%i.yml run %i
User=root
Group=root

Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

[papanito]

There is a new parameter to override the binary. Did you use that?

[papanito]

Ahh I see, your binary is named cloudflared. And the binary is where on the file system? And can you execute cloudflared manually?

[dx0x58]

Binary is here. But with strange permissions

root@ip-10-0-28-18:~# ll /usr/bin/cloudflared
-rw-r--r-- 1 501 staff 31384993 Dec  1 11:17 /usr/bin/cloudflared

[papanito]

If you manually change the permissions can you start the service?

[papanito]

Can you try to consume the role from branch papanito/issue-40-file-permissions directly?

[dx0x58]

Yes, I am getting the same error :(

[dx0x58]

I noticed an interesting thing, if i change unit config like:

i am getting the following error

the owner/group on a binary file look like this

[dx0x58]

Finally I figured it out.

1. I deleted the old binary file on the server, now the binary is downloaded with the correct permissions, group and user

   

2. Changed the unit configuration file:

   

3. Launched the role

4. The daemon started up through the absolute path to the file

Could you change the unit config file to an absolute path?

[papanito]

I guess the problem were the permissions and the groups. Can you see if the daemon starts without having the full path --> please test

I am not sure about the full path cause I am not sure if the path is always /usr/bin when cloudflare is installed via package manager.

My list fix - the file permissions and user groups - is only applies when installing and as you already had the binary installed, it did not run.

I will change that behavior.

[dx0x58]

Hi! If i use role without changes i get an error:

But i can call cloudflared directy:

In my case unit work well if path is absolute, not like in default config

[papanito]

Thanks for testing. I will update and make a new release soon

[papanito]

Can you try again from master-branch. If all is good, I will make a new release

[dx0x58]

Everything works fine with the edits of this branch https://github.com/papanito/ansible-role-cloudflared/pull/43. Thanks for the help! There is still an error in the master branch.

[papanito]

Yeah forgot to merge the PR. I will create a new version now.