The role 'preflight' was not found

[JohnCido]

Describe the bug

ERROR! the role 'preflight' was not found in amazon.aws:community.aws:community.general:ansible.legacy:/mnt/c/Users/*/selfhost/playbooks/providers/roles:/home/*/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/mnt/c/Users/johnc/selfhost/playbooks/providers

The error appears to be in '/mnt/c/Users/*/selfhost/playbooks/providers/aws.yml': line 24, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  roles:
    - preflight
      ^ here

To Reproduce Steps to reproduce the behavior:

1. Do the AWS setup
2. Run AWS playbook

Expected behavior Proper AWS setup

Screenshots If applicable, add screenshots to help explain your problem.

OS (please complete the following information):

• WSL2 on Windows 11

Cloud Provider (please complete the following information):

• AWS

Additional context Add any other context about the problem here.

[andygeorge]

Hey @JohnCido. The preflight role is part of this repository and should be included by Ansible automatically (see these docs that describe how Ansible finds roles).

A few questions:

• Which directory are you running the AWS provision playbook from? It should be the "root" of selfhost, possibly /mnt/c/Users/johnc/selfhost/ in your environment.
• What version of Ansible are you running (ansible --version), and how did you install it?
• what Linux distribution are you using with WSL2?

You might be our first Selfhoster using Windows/WSL2, so thank you for trying this out!

[JohnCido]

Hey @JohnCido. The preflight role is part of this repository and should be included by Ansible automatically (see these docs that describe how Ansible finds roles).

A few questions:

• Which directory are you running the AWS provision playbook from? It should be the "root" of selfhost, possibly /mnt/c/Users/johnc/selfhost/ in your environment.
• What version of Ansible are you running (ansible --version), and how did you install it?
• what Linux distribution are you using with WSL2?

You might be our first Selfhoster using Windows/WSL2, so thank you for trying this out!

1. I'm running from the */selfhost folder

2. This is the output

   ansible [core 2.11.6] 
   config file = None
   configured module search path = ['/home/john/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
   ansible python module location = /home/john/.local/lib/python3.6/site-packages/ansible
   ansible collection location = /home/john/.ansible/collections:/usr/share/ansible/collections
   executable location = /home/john/.local/bin/ansible
   python version = 3.6.9 (default, Jan 26 2021, 15:33:00) [GCC 8.4.0]
   jinja version = 3.0.3
   libyaml = True

   And I suppose I installed it using the Semi-automated setup in the README

3. This is the Linux I'm running:

   No LSB modules are available.
   Distributor ID: Ubuntu
   Description:    Ubuntu 18.04.6 LTS
   Release:        18.04
   Codename:       bionic

[andygeorge]

Can you post the entire Ansible output from the playbook run?

[JohnCido]

[WARNING]: Ansible is being run in a world writable directory (/mnt/c/Users/johnc/selfhost), ignoring it as an ansible.cfg source. For more information see https://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-world-writable-dir
[DEPRECATION WARNING]: Ansible will require Python 3.8 or newer on the controller starting with Ansible 2.12. Current version: 3.6.9 (default, Jan 26 2021, 15:33:00) [GCC 8.4.0]. This feature will be removed from ansible-core in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
ERROR! the role 'preflight' was not found in amazon.aws:community.aws:community.general:ansible.legacy:/mnt/c/Users/johnc/selfhost/playbooks/providers/roles:/home/john/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/mnt/c/Users/johnc/selfhost/playbooks/providers

The error appears to be in '/mnt/c/Users/johnc/selfhost/playbooks/providers/aws.yml': line 24, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  roles:
    - preflight
      ^ here

[andygeorge]

I wonder if your environment is not treating roles_path = ./roles/ correctly. Can you try setting roles_path in ansible.cfg to the full path of your local selfhost/roles/ folder? Guessing that'd be:

roles_path = /mnt/c/Users/johnc/selfhost/roles/

[andygeorge]

Another thought: since it looks like you've downloaded the selfhost repository to Windows directly, but are running Ansible through WSL2 Ubuntu and accessing your Windows C: drive mount, I wonder if that's causing issues.

I would recommend downloading the repository directly into your WSL2 Ubuntu filesystem and trying to provision that way.

[JohnCido]

I've tried to explicitly set the roles_path and move the project folder under ~/, the error persists.

[andygeorge]

Please add verbose logging to the playbook run:

ansible-playbook -i inventory/forem/setup.yml playbooks/providers/aws.yml -vvvvv

...and then paste the entire output of the playbook run (including the run command) here.

[bangden07]

Please add verbose logging to the playbook run:

ansible-playbook -i inventory/forem/setup.yml playbooks/providers/aws.yml -vvvvv

...and then paste the entire output of the playbook run (including the run command) here.

I also have the same problem

[WARNING]: Ansible is being run in a world writable directory (/mnt/d/project/selfhost), ignoring it as an ansible.cfg
source. For more information see https://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-world-
writable-dir
ansible-playbook [core 2.11.6]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/bangden/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/bangden/.local/lib/python3.8/site-packages/ansible
  ansible collection location = /home/bangden/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/bangden/.local/bin/ansible-playbook
  python version = 3.8.10 (default, Sep 28 2021, 16:10:42) [GCC 9.3.0]
  jinja version = 2.10.1
  libyaml = True
Using /etc/ansible/ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /mnt/d/project/selfhost/inventory/forem/setup.yml as it did not pass its verify_file() method
toml declined parsing /mnt/d/project/selfhost/inventory/forem/setup.yml as it did not pass its verify_file() method
[WARNING]:  * Failed to parse /mnt/d/project/selfhost/inventory/forem/setup.yml with script plugin: problem running
/mnt/d/project/selfhost/inventory/forem/setup.yml --list ([Errno 8] Exec format error:
'/mnt/d/project/selfhost/inventory/forem/setup.yml')
  File "/home/bangden/.local/lib/python3.8/site-packages/ansible/inventory/manager.py", line 290, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/home/bangden/.local/lib/python3.8/site-packages/ansible/plugins/inventory/script.py", line 169, in parse
    raise AnsibleParserError(to_native(e))
[WARNING]:  * Failed to parse /mnt/d/project/selfhost/inventory/forem/setup.yml with auto plugin: We were unable to
read either as JSON nor YAML, these are the errors we got from each: JSON: Expecting value: line 1 column 1 (char 0)
Syntax Error while loading YAML.   could not find expected ':'  The error appears to be in
'/mnt/d/project/selfhost/inventory/forem/setup.yml': line 88, column 11, but may be elsewhere in the file depending on
the exact syntax problem.  The offending line appears to be:            $ANSIBLE_VAULT;1.1;AES256
30633232646163653761613336313565333564376562633963633837643966373739376334306665           ^ here
  File "/home/bangden/.local/lib/python3.8/site-packages/ansible/inventory/manager.py", line 290, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/home/bangden/.local/lib/python3.8/site-packages/ansible/plugins/inventory/auto.py", line 40, in parse
    config_data = loader.load_from_file(path, cache=False)
  File "/home/bangden/.local/lib/python3.8/site-packages/ansible/parsing/dataloader.py", line 97, in load_from_file
    parsed_data = self.load(data=file_data, file_name=file_name, show_content=show_content, json_only=json_only)
  File "/home/bangden/.local/lib/python3.8/site-packages/ansible/parsing/dataloader.py", line 80, in load
    return from_yaml(data, file_name, show_content, self._vault.secrets, json_only=json_only)
  File "/home/bangden/.local/lib/python3.8/site-packages/ansible/parsing/utils/yaml.py", line 82, in from_yaml
    _handle_error(json_exc, yaml_exc, file_name, show_content)
  File "/home/bangden/.local/lib/python3.8/site-packages/ansible/parsing/utils/yaml.py", line 43, in _handle_error
    raise AnsibleParserError(n_err_msg, obj=err_obj, show_content=show_content, orig_exc=yaml_exc)
[WARNING]:  * Failed to parse /mnt/d/project/selfhost/inventory/forem/setup.yml with yaml plugin: We were unable to
read either as JSON nor YAML, these are the errors we got from each: JSON: Expecting value: line 1 column 1 (char 0)
Syntax Error while loading YAML.   could not find expected ':'  The error appears to be in
'/mnt/d/project/selfhost/inventory/forem/setup.yml': line 88, column 11, but may be elsewhere in the file depending on
the exact syntax problem.  The offending line appears to be:            $ANSIBLE_VAULT;1.1;AES256
30633232646163653761613336313565333564376562633963633837643966373739376334306665           ^ here
  File "/home/bangden/.local/lib/python3.8/site-packages/ansible/inventory/manager.py", line 290, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/home/bangden/.local/lib/python3.8/site-packages/ansible/plugins/inventory/yaml.py", line 105, in parse
    raise AnsibleParserError(e)
[WARNING]:  * Failed to parse /mnt/d/project/selfhost/inventory/forem/setup.yml with ini plugin: Invalid host pattern
'---' supplied, '---' is normally a sign this is a YAML file.
  File "/home/bangden/.local/lib/python3.8/site-packages/ansible/inventory/manager.py", line 290, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/home/bangden/.local/lib/python3.8/site-packages/ansible/plugins/inventory/ini.py", line 136, in parse
    raise AnsibleParserError(e)
[WARNING]: Unable to parse /mnt/d/project/selfhost/inventory/forem/setup.yml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match
'all'
Loading collection community.general from /home/bangden/.local/lib/python3.8/site-packages/ansible_collections/community/general
Loading collection community.digitalocean from /home/bangden/.ansible/collections/ansible_collections/community/digitalocean
ERROR! the role 'preflight' was not found in community.general:community.digitalocean:ansible.legacy:/mnt/d/project/selfhost/playbooks/providers/roles:/home/bangden/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/mnt/d/project/selfhost/playbooks/providers

The error appears to be in '/mnt/d/project/selfhost/playbooks/providers/digitalocean.yml': line 57, column 5, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  roles:
  - preflight
    ^ here

[JohnCido]

In my case, the logging:

[WARNING]: Ansible is being run in a world writable directory (/home/john/selfhost), ignoring it as an ansible.cfg source. For more information see https://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-world-writable-dir
[DEPRECATION WARNING]: Ansible will require Python 3.8 or newer on the controller starting with Ansible 2.12. Current version: 3.6.9 (default, Jan 26 2021, 15:33:00) [GCC 8.4.0]. This feature will be removed from ansible-core in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
ansible-playbook [core 2.11.6] 
  config file = None
  configured module search path = ['/home/john/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/john/.local/lib/python3.6/site-packages/ansible
  ansible collection location = /home/john/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/john/.local/bin/ansible-playbook
  python version = 3.6.9 (default, Jan 26 2021, 15:33:00) [GCC 8.4.0]
  jinja version = 3.0.3
  libyaml = True
No config file found; using defaults
setting up inventory plugins
host_list declined parsing /home/john/selfhost/inventory/forem/setup.yml as it did not pass its verify_file() method
Skipping empty key (hosts) in group (all)
Parsed /home/john/selfhost/inventory/forem/setup.yml inventory source with yaml plugin
Loading collection amazon.aws from /home/john/.local/lib/python3.6/site-packages/ansible_collections/amazon/aws
Loading collection community.aws from /home/john/.local/lib/python3.6/site-packages/ansible_collections/community/aws
Loading collection community.general from /home/john/.local/lib/python3.6/site-packages/ansible_collections/community/general
ERROR! the role 'preflight' was not found in amazon.aws:community.aws:community.general:ansible.legacy:/home/john/selfhost/playbooks/providers/roles:/home/john/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/john/selfhost/playbooks/providers

The error appears to be in '/home/john/selfhost/playbooks/providers/aws.yml': line 24, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  roles:
    - preflight
      ^ here

[andygeorge]

There might be an issue with your inventory file, as mentioned here:

Failed to parse /mnt/d/project/selfhost/inventory/forem/setup.yml with auto plugin: We were unable to
read either as JSON nor YAML, these are the errors we got from each: JSON: Expecting value: line 1 column 1 (char 0)
Syntax Error while loading YAML.   could not find expected ':'  The error appears to be in
'/mnt/d/project/selfhost/inventory/forem/setup.yml': line 88, column 11, but may be elsewhere in the file depending on
the exact syntax problem.  The offending line appears to be:            $ANSIBLE_VAULT;1.1;AES256
30633232646163653761613336313565333564376562633963633837643966373739376334306665           ^ here

Any chance you could email me your selfhost/inventory/forem/setup.yml file (ageorge [AT] forem.com)? The vaulted strings can't be decrypted without your ~/.config/forem/selfhost_ansible_vault_password, which only you should have, I am just curious if there are any formatting issues.

Outside of that, though, this is still a problem:

ERROR! the role 'preflight' was not found in amazon.aws:community.aws:community.general:ansible.legacy:/home/john/selfhost/playbooks/providers/roles:/home/john/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/john/selfhost/playbooks/providers

...as it does not appear Ansible is correctly finding the selfhost/roles/ folder.

Any chance you could try running Ansible on a native Linux host, versus WSL2? We haven't tested this under WSL2, but I would not be surprised if Ansible doesn't play nice there.

[JohnCido]

@andygeorge I've sent you the setup.yml file. And I'm sorry that I don't have a machine with native Linux. And I don't know if understand it correctly. The selfhost folder is located on a machine that controls the AWS EC2 using the CLI right? If I have to use an instance on EC2 to manage another EC2 instance, it appears too troublesome to me since what I want is a simple forum server for a small group of people.

[andygeorge]

Thank you for your setup.yml file. The formatting seems fine.

The selfhost folder is located on a machine that controls the AWS EC2 using the CLI right?

Yes; more specifically, the selfhost repository should be on host you're using to run Ansible, like a home computer.

If I have to use an instance on EC2 to manage another EC2 instance, it appears too troublesome to me

You are correct: you do not have to do this. I was wondering if you had access to a host with native Linux, such as a home computer, or a VM running on your home Windows PC.

The selfhost/roles/preflight/ role folder should be automatically found by Ansible when running a playbook. As I mentioned before, it is very possible that Ansible running on WSL2 is not functioning as intended, so my next recommended step would be running Ansible natively on Linux.

Please note that Ansible itself does not natively support running on Windows or WSL2.

[citizen428]

I have set up a a self-hosted Forem instance from within WSL2 (it's also where I developed the shells script that's now part of the repository) and so has at least one community contributor. Apart from the lack of systemd and a kernel compiled by MS there are no major differences between running a distribution inside or outside WSL, at least for normal CLI usage.

One thing of note is that I manually updated the Ubuntu version from LTS, I think at the time of my selfhost installation I was on 21.04. That said, 18.04 is a relatively old version, it isn't even the most recent LTS, which would be 20.04. I don't have access to my Windows machine right now, but I'm reasonably sure that I installed Ansible via PIP, not a distribution package because I wanted to keep everything in the same virtualenv.

@JohnCido How did you install Ansible, via a distribution package or via pip? If the former, can you maybe uninstall it again and try to install a current Ansible version with PIP? To be clear: our setup script does install Ansible via PIP, I just wanted to make sure you don't also have it installed directly in Ubuntu, potentially in a location where it gets picked up first from your $PATH.

[JohnCido]

I have set up a a self-hosted Forem instance from within WSL2 (it's also where I developed the shells script that's now part of the repository) and so has at least one community contributor. Apart from the lack of systemd and a kernel compiled by MS there are no major differences between running a distribution inside or outside WSL, at least for normal CLI usage.

One thing of note is that I manually updated the Ubuntu version from LTS, I think at the time of my selfhost installation I was on 21.04. That said, 18.04 is a relatively old version, it isn't even the most recent LTS, which would be 20.04. I don't have access to my Windows machine right now, but I'm reasonably sure that I installed Ansible via PIP, not a distribution package because I wanted to keep everything in the same virtualenv.

@JohnCido How did you install Ansible, via a distribution package or via pip? If the former, can you maybe uninstall it again and try to install a current Ansible version with PIP? To be clear: our setup script does install Ansible via PIP, I just wanted to make sure you don't also have it installed directly in Ubuntu, potentially in a location where it gets picked up first from your $PATH.

I've updated my WSL to Ubuntu 20.04.3 and installed Ansible using pip3, and this is the terminal output:

[WARNING]: Ansible is being run in a world writable directory (/home/john/selfhost), ignoring it as an ansible.cfg
source. For more information see https://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-world-
writable-dir
ansible-playbook [core 2.12.1]
  config file = None
  configured module search path = ['/home/john/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/john/.local/lib/python3.8/site-packages/ansible
  ansible collection location = /home/john/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/john/.local/bin/ansible-playbook
  python version = 3.8.10 (default, Sep 28 2021, 16:10:42) [GCC 9.3.0]
  jinja version = 3.0.3
  libyaml = True
No config file found; using defaults
setting up inventory plugins
host_list declined parsing /home/john/selfhost/inventory/forem/setup.yml as it did not pass its verify_file() method
Skipping empty key (hosts) in group (all)
Parsed /home/john/selfhost/inventory/forem/setup.yml inventory source with yaml plugin
Loading collection amazon.aws from /home/john/.local/lib/python3.8/site-packages/ansible_collections/amazon/aws
Loading collection community.aws from /home/john/.local/lib/python3.8/site-packages/ansible_collections/community/aws
Loading collection community.general from /home/john/.local/lib/python3.8/site-packages/ansible_collections/community/general
ERROR! the role 'preflight' was not found in amazon.aws:community.aws:community.general:ansible.legacy:/home/john/selfhost/playbooks/providers/roles:/home/john/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/john/selfhost/playbooks/providers

The error appears to be in '/home/john/selfhost/playbooks/providers/aws.yml': line 24, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  roles:
    - preflight
      ^ here

[jdoss]

Are you running the setup command or calling Ansible directly? Are you in the selfhost directory? The roles path is set here:

https://github.com/forem/selfhost/blob/main/ansible.cfg#L11

and if you are not in the selfhost directory it won't be able to find it as it won't load the ansible.cfg file with the settings. Based on the output you posted above, this is saying config file = None which makes me think you are not in the right directory.

[jdoss]

I am going to close this issue for now. Please let us know if you can answer the questions I posted back in February and we can reopen it.