search

Rudd-O / ansible-qubes

Qubes OS DevOps automation
55 stars 12 forks source link

Getting Non-Fatal Error Message on a lot of items #20

Open ProfessorManhattan opened 2 years ago

ProfessorManhattan commented 2 years ago

Hey, I'm trying to clean up the logging that this project outputs. I'm getting the following error but I'm unsure of how to fix it:

TypeError: a bytes-like object is required, not 'str'

This error comes from tasks that use copy with inline data. Here's an example:

- name: Save meta information about the version of AriaNG that was installed
  copy:
    dest: ~/.config/megabytelabs/ariang
    mode: 0600
    content: |
      {{ ariang_latest_release_tag.json.tag_name }}
  when: install_ariang

Any idea on how to fix this?

Rudd-O commented 2 years ago

Not sure, we need a traceback. Run ansible with -vvvvv to get the traceback and let's use that to figure out what is up.

ProfessorManhattan commented 2 years ago

Hey @Rudd-O -- I ran it with the -vvvvv. Here's some of the output --- you can see a lot of META: noop and Pipelining is enabled. as well as some of that TypeError message:

META: noop

TASK [roles/services/elasticagent : Find the latest Elastic Agent version] *****
task path: /etc/ansible/roles/services/elasticagent/tasks/install-Linux.yml:30
Wednesday 02 November 2022  23:03:59 -0400 (0:00:00.878)       0:46:53.513 **** 
ok: [whonix-ws-16] => changed=false 
  ansible_facts:
    elasticagent_latest_release: 8.5.0
META: noop

TASK [roles/services/elasticagent : Determine whether or not the latest version of Elastic Agent is already installed] ***
task path: /etc/ansible/roles/services/elasticagent/tasks/install-Linux.yml:34
Wednesday 02 November 2022  23:03:59 -0400 (0:00:00.075)       0:46:53.589 **** 
ok: [whonix-ws-16] => changed=false 
  ansible_facts:
    install_elasticagent: true
META: noop

TASK [roles/services/elasticagent : Fetch CHECKSUMS file of Elastic Agent installer] ***
task path: /etc/ansible/roles/services/elasticagent/tasks/install-Linux.yml:39
Wednesday 02 November 2022  23:03:59 -0400 (0:00:00.072)       0:46:53.662 **** 
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: user
<localhost> EXEC /bin/sh -c 'echo ~user && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/user/.ansible/tmp `"&& mkdir "` echo /home/user/.ansible/tmp/ansible-tmp-1667444639.279162-24173-280944725315199 `" && echo ansible-tmp-1667444639.279162-24173-280944725315199="` echo /home/user/.ansible/tmp/ansible-tmp-1667444639.279162-24173-280944725315199 `" ) && sleep 0'
Using module file /usr/lib/python3/dist-packages/ansible/modules/uri.py
Pipelining is enabled.
<localhost> EXEC /bin/sh -c 'sudo -H -S -n  -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-bqnnjbfgcsgeednwogmcwooxtdqdcssq ; /usr/bin/python3'"'"' && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/user/.ansible/tmp/ansible-tmp-1667444639.279162-24173-280944725315199/ > /dev/null 2>&1 && sleep 0'
ok: [whonix-ws-16] => changed=false 
  accept_ranges: bytes
  age: '0'
  alt_svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
  cache_control: public, max-age=600
  connection: close
  content: |-
    0cca2b56aed453ea03775fdcc7345332f404d1cc41864addfb6d8df7cc8ea0a09cecb0b8201742fab3e0bd59de0d4ba33ca2d132fec7d6cb5a178ee7b6e382f2  elastic-agent-8.5.0-linux-x86_64.tar.gz
  content_length: '170'
  content_type: binary/octet-stream
  cookies: {}
  cookies_string: ''
  date: Thu, 03 Nov 2022 03:04:00 GMT
  elapsed: 0
  etag: '"4ba5fbdd131435a6374138160381c5ae"'
  invocation:
    module_args:
      attributes: null
      body: null
      body_format: raw
      client_cert: null
      client_key: null
      creates: null
      dest: null
      follow_redirects: safe
      force: false
      force_basic_auth: false
      group: null
      headers: {}
      http_agent: ansible-httpget
      method: GET
      mode: null
      owner: null
      remote_src: false
      removes: null
      return_content: true
      selevel: null
      serole: null
      setype: null
      seuser: null
      src: null
      status_code:
      - 200
      timeout: 30
      unix_socket: null
      unsafe_writes: false
      url: https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.5.0-linux-x86_64.tar.gz.sha512
      url_password: null
      url_username: null
      use_proxy: true
      validate_certs: true
  last_modified: Tue, 01 Nov 2022 11:52:04 GMT
  msg: OK (170 bytes)
  redirected: false
  status: 200
  strict_transport_security: max-age=31536000; includeSubDomains
  url: https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.5.0-linux-x86_64.tar.gz.sha512
  via: 1.1 google
META: noop

TASK [roles/services/elasticagent : Find the Checksum of Elastic Agent installer] ***
task path: /etc/ansible/roles/services/elasticagent/tasks/install-Linux.yml:48
Wednesday 02 November 2022  23:03:59 -0400 (0:00:00.474)       0:46:54.136 **** 
ok: [whonix-ws-16] => changed=false 
  ansible_facts:
    elasticagent_checksum: 0cca2b56aed453ea03775fdcc7345332f404d1cc41864addfb6d8df7cc8ea0a09cecb0b8201742fab3e0bd59de0d4ba33ca2d132fec7d6cb5a178ee7b6e382f2
META: noop

TASK [roles/services/elasticagent : Acquire Elastic Agent's latest installer] ***
task path: /etc/ansible/roles/services/elasticagent/tasks/install-Linux.yml:54
Wednesday 02 November 2022  23:03:59 -0400 (0:00:00.055)       0:46:54.191 **** 
skipping: [whonix-ws-16] => changed=false 
  skip_reason: Conditional result was False
META: noop

TASK [roles/services/elasticagent : Acquire Elastic Agent's latest installer (for Qubes Whonix)] ***
task path: /etc/ansible/roles/services/elasticagent/tasks/install-Linux.yml:64
Wednesday 02 November 2022  23:03:59 -0400 (0:00:00.041)       0:46:54.233 **** 
Using module file /usr/lib/python3/dist-packages/ansible/modules/get_url.py
Pipelining is enabled.
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: user
<localhost> EXEC /bin/sh -c 'sudo -H -S -n  -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-syuledurhbmbbfrbniesraninmtsajpd ; /usr/bin/python3'"'"' && sleep 0'
ok: [whonix-ws-16] => changed=false 
  checksum_dest: null
  checksum_src: null
  dest: /tmp/elasticagent-8.5.0.tar.bz2
  elapsed: 0
  gid: 0
  group: root
  invocation:
    module_args:
      attributes: null
      backup: false
      checksum: sha512:0cca2b56aed453ea03775fdcc7345332f404d1cc41864addfb6d8df7cc8ea0a09cecb0b8201742fab3e0bd59de0d4ba33ca2d132fec7d6cb5a178ee7b6e382f2
      client_cert: null
      client_key: null
      dest: /tmp/elasticagent-8.5.0.tar.bz2
      force: false
      force_basic_auth: false
      group: null
      headers: null
      http_agent: ansible-httpget
      mode: null
      owner: null
      selevel: null
      serole: null
      setype: null
      seuser: null
      sha256sum: ''
      timeout: 10
      tmp_dest: null
      unsafe_writes: false
      url: https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.5.0-linux-x86_64.tar.gz
      url_password: null
      url_username: null
      use_proxy: true
      validate_certs: true
  mode: '0644'
  msg: file already exists
  owner: root
  size: 395510548
  state: file
  uid: 0
  url: https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.5.0-linux-x86_64.tar.gz
META: noop

TASK [roles/services/elasticagent : Ensure Elastic Agent is extracted] *********
task path: /etc/ansible/roles/services/elasticagent/tasks/install-Linux.yml:75
Wednesday 02 November 2022  23:04:00 -0400 (0:00:00.887)       0:46:55.121 **** 
skipping: [whonix-ws-16] => changed=false 
  skip_reason: Conditional result was False
META: noop

TASK [roles/services/elasticagent : Ensure Elastic Agent is extracted (for Qubes Whonix)] ***
task path: /etc/ansible/roles/services/elasticagent/tasks/install-Linux.yml:86
Wednesday 02 November 2022  23:04:00 -0400 (0:00:00.036)       0:46:55.158 **** 
Using module file /usr/lib/python3/dist-packages/ansible/modules/stat.py
Pipelining is enabled.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: a bytes-like object is required, not 'str'
Using module file /usr/lib/python3/dist-packages/ansible/modules/unarchive.py
Pipelining is enabled.

changed: [whonix-ws-16] => changed=true 
  dest: /usr/local/src/elastic
  extract_results:
    cmd:
    - /usr/bin/tar
    - --extract
    - -C
    - /usr/local/src/elastic
    - -z
    - --show-transformed-names
    - --strip-components=1
    - -f
    - /home/user/.ansible/tmp/ansible-tmp-1667444641.0826766-24214-125972086714396/source
    err: ''
    out: ''
    rc: 0
  gid: 0
  group: root
  handler: TgzArchive
  invocation:
    module_args:
      attributes: null
      creates: null
      dest: /usr/local/src/elastic
      exclude: []
      extra_opts:
      - --strip-components=1
      group: null
      keep_newer: false
      list_files: false
      mode: null
      owner: null
      remote_src: false
      selevel: null
      serole: null
      setype: null
      seuser: null
      src: /home/user/.ansible/tmp/ansible-tmp-1667444641.0826766-24214-125972086714396/source
      unsafe_writes: false
      validate_certs: true
  mode: '0755'
  owner: root
  size: 4096
  src: /home/user/.ansible/tmp/ansible-tmp-1667444641.0826766-24214-125972086714396/source
  state: directory
  uid: 0
META: noop

TASK [roles/services/elasticagent : Ensure Elastic Agent is installed] *********
task path: /etc/ansible/roles/services/elasticagent/tasks/install-Linux.yml:96
Wednesday 02 November 2022  23:04:12 -0400 (0:00:11.993)       0:47:07.151 **** 
skipping: [whonix-ws-16] => changed=false 
  skip_reason: Conditional result was False
META: noop

TASK [roles/services/elasticagent : Save meta information about the version of Elastic Agent that was installed] ***
task path: /etc/ansible/roles/services/elasticagent/tasks/install-Linux.yml:108
Wednesday 02 November 2022  23:04:12 -0400 (0:00:00.051)       0:47:07.203 **** 
skipping: [whonix-ws-16] => changed=false 
  skip_reason: Conditional result was False
META: noop

TASK [roles/services/portmaster : Include variables based on the operating system] ***
task path: /etc/ansible/roles/services/portmaster/tasks/main.yml:2
Wednesday 02 November 2022  23:04:12 -0400 (0:00:00.041)       0:47:07.245 **** 
looking for "Debian.yml" at "/etc/ansible/roles/services/portmaster/vars/Debian.yml"
ok: [whonix-ws-16] => changed=false 
  ansible_facts:
    portmaster_dependencies:
    - libappindicator3-1
    - libnetfilter-queue1
    portmaster_url: https://updates.safing.io/latest/linux_amd64/packages/portmaster-installer.deb
  ansible_included_var_files:
  - /etc/ansible/roles/services/portmaster/vars/Debian.yml
META: noop

TASK [roles/services/portmaster : include_tasks] *******************************
task path: /etc/ansible/roles/services/portmaster/tasks/main.yml:13
Wednesday 02 November 2022  23:04:12 -0400 (0:00:00.078)       0:47:07.323 **** 
included: /etc/ansible/roles/services/portmaster/tasks/install-Debian.yml for whonix-ws-16
META: noop

TASK [roles/services/portmaster : Ensure apt cache is updated] *****************
task path: /etc/ansible/roles/services/portmaster/tasks/install-Debian.yml:2
Wednesday 02 November 2022  23:04:12 -0400 (0:00:00.049)       0:47:07.372 **** 
Using module file /usr/lib/python3/dist-packages/ansible/modules/apt.py
Pipelining is enabled.

The full traceback is:
  File "/tmp/ansible_apt_payload_m_x3c7uj/ansible_apt_payload.zip/ansible/modules/apt.py", line 1128, in main
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 551, in update
    with _WrappedLock(apt_pkg.config.find_dir("Dir::State::Lists")):
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 74, in __enter__
    raise LockFailedException(("Failed to lock directory %s: %s") %
fatal: [whonix-ws-16]: FAILED! => changed=false 
  invocation:
    module_args:
      allow_unauthenticated: false
      autoclean: false
      autoremove: false
      cache_valid_time: 0
      deb: null
      default_release: null
      dpkg_options: force-confdef,force-confold
      force: false
      force_apt_get: false
      install_recommends: null
      only_upgrade: false
      package: null
      policy_rc_d: null
      purge: false
      state: present
      update_cache: true
      update_cache_retries: 5
      update_cache_retry_max_delay: 12
      upgrade: null
  msg: 'Failed to lock apt for exclusive operation: Failed to lock directory /var/lib/apt/lists/: E:Could not get lock /var/lib/apt/lists/lock. It is held by process 35999 (apt-get.anondis)'
...ignoring
META: noop

TASK [roles/services/portmaster : Ensure Portmaster's dependencies are installed] ***
task path: /etc/ansible/roles/services/portmaster/tasks/install-Debian.yml:7
Wednesday 02 November 2022  23:04:13 -0400 (0:00:00.966)       0:47:08.339 **** 
Using module file /usr/lib/python3/dist-packages/ansible/modules/apt.py
Pipelining is enabled.

changed: [whonix-ws-16] => changed=true 
  cache_update_time: 1667444745
  cache_updated: false
  diff: {}
  invocation:
    module_args:
      allow_unauthenticated: false
      autoclean: false
      autoremove: false
      cache_valid_time: 0
      deb: null
      default_release: null
      dpkg_options: force-confdef,force-confold
      force: false
      force_apt_get: false
      install_recommends: null
      name:
      - libappindicator3-1
      - libnetfilter-queue1
      only_upgrade: false
      package:
      - libappindicator3-1
      - libnetfilter-queue1
      policy_rc_d: null
      purge: false
      state: present
      update_cache: null
      update_cache_retries: 5
      update_cache_retry_max_delay: 12
      upgrade: null
  stderr: ''
  stderr_lines: <omitted>
  stdout: |-
    Reading package lists...
    Building dependency tree...
    Reading state information...
    The following packages were automatically installed and are no longer required:
      libbotan-2-17 libtspi1
    Use 'sudo apt autoremove' to remove them.
    The following additional packages will be installed:
      libayatana-ido3-0.4-0 libayatana-indicator3-7 libdbusmenu-glib4
      libdbusmenu-gtk3-4
    The following NEW packages will be installed:
      libayatana-appindicator3-1 libayatana-ido3-0.4-0 libayatana-indicator3-7
      libdbusmenu-glib4 libdbusmenu-gtk3-4 libnetfilter-queue1
    0 upgraded, 6 newly installed, 0 to remove and 0 not upgraded.
    Need to get 377 kB of archives.
    After this operation, 868 kB of additional disk space will be used.
    Get:1 tor+https://deb.debian.org/debian bullseye/main amd64 libayatana-ido3-0.4-0 amd64 0.8.2-1 [74.9 kB]
    Get:2 tor+https://deb.debian.org/debian bullseye/main amd64 libayatana-indicator3-7 amd64 0.8.4-1+deb11u2 [96.3 kB]
    Get:3 tor+https://deb.debian.org/debian bullseye/main amd64 libdbusmenu-glib4 amd64 18.10.20180917~bzr492+repack1-2 [58.4 kB]
    Get:4 tor+https://deb.debian.org/debian bullseye/main amd64 libdbusmenu-gtk3-4 amd64 18.10.20180917~bzr492+repack1-2 [46.1 kB]
    Get:5 tor+https://deb.debian.org/debian bullseye/main amd64 libayatana-appindicator3-1 amd64 0.5.5-2+deb11u2 [86.5 kB]
    Get:6 tor+https://deb.debian.org/debian bullseye/main amd64 libnetfilter-queue1 amd64 1.0.5-2 [14.5 kB]
    Fetched 377 kB in 2s (215 kB/s)
    Selecting previously unselected package libayatana-ido3-0.4-0:amd64.
    (Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 62774 files and directories currently installed.)
    Preparing to unpack .../0-libayatana-ido3-0.4-0_0.8.2-1_amd64.deb ...
    Unpacking libayatana-ido3-0.4-0:amd64 (0.8.2-1) ...
    Selecting previously unselected package libayatana-indicator3-7:amd64.
    Preparing to unpack .../1-libayatana-indicator3-7_0.8.4-1+deb11u2_amd64.deb ...
    Unpacking libayatana-indicator3-7:amd64 (0.8.4-1+deb11u2) ...
    Selecting previously unselected package libdbusmenu-glib4:amd64.
    Preparing to unpack .../2-libdbusmenu-glib4_18.10.20180917~bzr492+repack1-2_amd64.deb ...
    Unpacking libdbusmenu-glib4:amd64 (18.10.20180917~bzr492+repack1-2) ...
    Selecting previously unselected package libdbusmenu-gtk3-4:amd64.
    Preparing to unpack .../3-libdbusmenu-gtk3-4_18.10.20180917~bzr492+repack1-2_amd64.deb ...
    Unpacking libdbusmenu-gtk3-4:amd64 (18.10.20180917~bzr492+repack1-2) ...
    Selecting previously unselected package libayatana-appindicator3-1.
    Preparing to unpack .../4-libayatana-appindicator3-1_0.5.5-2+deb11u2_amd64.deb ...
    Unpacking libayatana-appindicator3-1 (0.5.5-2+deb11u2) ...
    Selecting previously unselected package libnetfilter-queue1:amd64.
    Preparing to unpack .../5-libnetfilter-queue1_1.0.5-2_amd64.deb ...
    Unpacking libnetfilter-queue1:amd64 (1.0.5-2) ...
    Setting up libdbusmenu-glib4:amd64 (18.10.20180917~bzr492+repack1-2) ...
    Setting up libayatana-ido3-0.4-0:amd64 (0.8.2-1) ...
    Setting up libnetfilter-queue1:amd64 (1.0.5-2) ...
    Setting up libayatana-indicator3-7:amd64 (0.8.4-1+deb11u2) ...
    Setting up libdbusmenu-gtk3-4:amd64 (18.10.20180917~bzr492+repack1-2) ...
    Setting up libayatana-appindicator3-1 (0.5.5-2+deb11u2) ...
    Processing triggers for libc-bin (2.31-13+deb11u5) ...
    [ Rootkit Hunter version 1.4.6 ]
    File updated: searched for 182 files, found 144
  stdout_lines: <omitted>
META: noop

TASK [roles/services/portmaster : Ensure Portmaster is installed] **************
task path: /etc/ansible/roles/services/portmaster/tasks/install-Debian.yml:12
Wednesday 02 November 2022  23:04:28 -0400 (0:00:14.639)       0:47:22.978 **** 
skipping: [whonix-ws-16] => changed=false 
  skip_reason: Conditional result was False
META: noop

TASK [roles/services/portmaster : Download Portmaster's deb installer file (for Qubes Whonix)] ***
task path: /etc/ansible/roles/services/portmaster/tasks/install-Debian.yml:18
Wednesday 02 November 2022  23:04:28 -0400 (0:00:00.058)       0:47:23.037 **** 
Using module file /usr/lib/python3/dist-packages/ansible/modules/get_url.py
Pipelining is enabled.
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: user
<localhost> EXEC /bin/sh -c 'sudo -H -S -n  -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-msdodekodccuisoryrmqdvmqlxxdrumm ; /usr/bin/python3'"'"' && sleep 0'
ok: [whonix-ws-16] => changed=false 
  dest: /tmp/portainer.deb
  elapsed: 0
  gid: 0
  group: root
  invocation:
    module_args:
      attributes: null
      backup: false
      checksum: ''
      client_cert: null
      client_key: null
      dest: /tmp/portainer.deb
      force: false
      force_basic_auth: false
      group: null
      headers: null
      http_agent: ansible-httpget
      mode: null
      owner: null
      selevel: null
      serole: null
      setype: null
      seuser: null
      sha256sum: ''
      timeout: 10
      tmp_dest: null
      unsafe_writes: false
      url: https://updates.safing.io/latest/linux_amd64/packages/portmaster-installer.deb
      url_password: null
      url_username: null
      use_proxy: true
      validate_certs: true
  mode: '0644'
  msg: 'HTTP Error 304: Not Modified'
  owner: root
  size: 10204272
  state: file
  status_code: 304
  uid: 0
  url: https://updates.safing.io/latest/linux_amd64/packages/portmaster-installer.deb
META: noop

TASK [roles/services/portmaster : Copy Portmaster's deb installer file (for Qubes Whonix)] ***
task path: /etc/ansible/roles/services/portmaster/tasks/install-Debian.yml:25
Wednesday 02 November 2022  23:04:29 -0400 (0:00:00.767)       0:47:23.804 **** 
Using module file /usr/lib/python3/dist-packages/ansible/modules/stat.py
Pipelining is enabled.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: a bytes-like object is required, not 'str'
Using module file /usr/lib/python3/dist-packages/ansible/modules/copy.py
Pipelining is enabled.

changed: [whonix-ws-16] => changed=true 
  checksum: 259745f587d8d5066a53e4d2683f52a6f41418ff
  dest: /tmp/portainer.deb
  diff: []
  gid: 0
  group: root
  invocation:
    module_args:
      _original_basename: portainer.deb
      attributes: null
      backup: false
      checksum: 259745f587d8d5066a53e4d2683f52a6f41418ff
      content: null
      dest: /tmp/portainer.deb
      directory_mode: null
      follow: false
      force: true
      group: null
      local_follow: null
      mode: null
      owner: null
      remote_src: null
      selevel: null
      serole: null
      setype: null
      seuser: null
      src: /home/user/.ansible/tmp/ansible-tmp-1667444669.73386-24312-66667816203684/source
      unsafe_writes: false
      validate: null
  md5sum: 2a23d5041eb4e723e86f2515e54bd58b
  mode: '0644'
  owner: root
  size: 10204272
  src: /home/user/.ansible/tmp/ansible-tmp-1667444669.73386-24312-66667816203684/source
  state: file
  uid: 0
META: noop

TASK [roles/services/portmaster : Ensure Portmaster is installed (on Qubes Whonix)] ***
task path: /etc/ansible/roles/services/portmaster/tasks/install-Debian.yml:31
Wednesday 02 November 2022  23:04:30 -0400 (0:00:01.062)       0:47:24.867 **** 
Using module file /usr/lib/python3/dist-packages/ansible/modules/apt.py
Pipelining is enabled.
Rudd-O commented 1 year ago

That looks like a bug in the stat.py module built into Ansible.