ansible / awx

AWX provides a web-based user interface, REST API, and task engine built on top of Ansible. It is one of the upstream projects for Red Hat Ansible Automation Platform.
Other
14.11k stars 3.43k forks source link

Job output not refreshing automatically in AWX 24.1.0 wit operator 2.14.0 #15071

Closed sysadmin-info closed 6 months ago

sysadmin-info commented 7 months ago

Please confirm the following

Bug Summary

HI,

the bug is identical like the one described here: https://github.com/ansible/awx/issues/9747

No above solution works. I mean this functionality just does not work and it seems it is related to websocket.

Operator version 2.14.0 AWX version 24.1.0 Database clean installation: PostgreSQL 15 k3s version: v1.28.8+k3s1 OS: Debian 12 with kernel: 6.1.0-18-amd64

I see identical errors that the websocket is not working as expected like it is described in the URL I provided.

The additional error:

The error Uncaught DOMException: Failed to execute 'send' on 'WebSocket': Still in CONNECTING state

indicates that the code is trying to send a message through the WebSocket before the connection has been fully established. This happens because the send method is called immediately after creating the WebSocket, but before the connection has completed the handshake with the server.

To resolve this issue, you should ensure that the send operation in the ws.onopen callback occurs only after the WebSocket connection is fully open. The onopen event is designed to handle this scenario, as it's triggered when the WebSocket connection has been established successfully.

Here's how you can modify the code to ensure the send method is called at the right time:

let ws;

export default function connectJobSocket({ type, id }, onMessage) {
  ws = new WebSocket(
    `${window.location.protocol === 'http:' ? 'ws:' : 'wss:'}//${
      window.location.host
    }${window.location.pathname}websocket/`
  );

  ws.onopen = () => {
    console.debug('WebSocket connection established.');

    const xrftoken = `; ${document.cookie}`
      .split('; csrftoken=')
      .pop()
      .split(';')
      .shift();
    const eventGroup = `${type}_events`;

    // Ensure the connection is open before sending data
    if (ws.readyState === WebSocket.OPEN) {
      ws.send(
        JSON.stringify({
          xrftoken,
          groups: { jobs: ['summary', 'status_changed'], [eventGroup]: [id] },
        })
      );
    }
  };

  ws.onmessage = (e) => {
    onMessage(JSON.parse(e.data));
  };

  ws.onclose = (e) => {
    console.debug('WebSocket closed:', e.code);
    if (e.code !== 1000) {
      console.debug('Reconnecting WebSocket...');
      setTimeout(() => {
        connectJobSocket({ type, id }, onMessage);
      }, 1000);
    }
  };

  ws.onerror = (err) => {
    console.debug('WebSocket error:', err);
    ws.close();
  };
}

export function closeWebSocket() {
  if (ws) {
    ws.close();
  }
}

Version AWX 23.9.0 with Operator 2.12.2 and PostgreSQL 13 is working without any issue. It does not matter do you use NGINX Proxy Manager, ingress or without ingress or do you modify the coreDNS it just works. So I decided to not upgrade to the newest AWX operator, AWX and PostgreSQL.

AWX version

24.1.0

Select the relevant components

Installation method

kubernetes

Modifications

yes

Ansible version

2.14.3

Operating system

Debian 12

Web browser

Chrome

Steps to reproduce

---
- name: Install AWX
  hosts: localhost
  become: yes
  vars:
    awx_namespace: awx
    project_directory: /var/lib/awx/projects
    storage_size: 2Gi

  tasks:
    - name: Download Kustomize with curl
      ansible.builtin.shell:
        cmd: curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash
        creates: /usr/local/bin/kustomize

    - name: Move Kustomize to the /usr/local/bin directory
      ansible.builtin.shell:
        cmd: mv kustomize /usr/local/bin
      args:
        creates: /usr/local/bin/kustomize

    - name: Ensure namespace {{ awx_namespace }} exists
      ansible.builtin.shell:
        cmd: kubectl create namespace {{ awx_namespace }} --dry-run=client -o yaml | kubectl apply -f -

    - name: Generate AWX resource file
      ansible.builtin.copy:
        dest: "./awx.yaml"
        content: |
          ---
          apiVersion: awx.ansible.com/v1beta1
          kind: AWX
          metadata:
            name: awx
          spec:
            service_type: nodeport
            nodeport_port: 30060
            projects_persistence: true
            projects_existing_claim: awx-projects-claim          

    - name: Fetch latest release tag of AWX Operator
      ansible.builtin.shell:
        cmd: curl -s https://api.github.com/repos/ansible/awx-operator/releases/latest | grep tag_name | cut -d '"' -f 4
      register: release_tag
      changed_when: false

    - name: Generate PV and PVC resource files
      ansible.builtin.copy:
        dest: "{{ item.dest }}"
        content: "{{ item.content }}"
      loop:
        - dest: "./pv.yml"
          content: |
            ---
            apiVersion: v1
            kind: PersistentVolume
            metadata:
              name: awx-projects-volume
            spec:
              accessModes:
                - ReadWriteOnce
              persistentVolumeReclaimPolicy: Retain
              capacity:
                storage: {{ storage_size }}
              storageClassName: awx-projects-volume
              hostPath:
                path: {{ project_directory }}            
        - dest: "./pvc.yml"
          content: |
            ---
            apiVersion: v1
            kind: PersistentVolumeClaim
            metadata:
              name: awx-projects-claim
            spec:
              accessModes:
                - ReadWriteOnce
              volumeMode: Filesystem
              resources:
                requests:
                  storage: {{ storage_size }}
              storageClassName: awx-projects-volume            

    - name: Create kustomization.yaml
      ansible.builtin.copy:
        dest: "./kustomization.yaml"
        content: |
          ---
          apiVersion: kustomize.config.k8s.io/v1beta1
          kind: Kustomization
          resources:
            - github.com/ansible/awx-operator/config/default?ref={{ release_tag.stdout }}
            - pv.yml
            - pvc.yml
            - awx.yaml
          images:
            - name: quay.io/ansible/awx-operator
              newTag: {{ release_tag.stdout }}
          namespace: {{ awx_namespace }}          

    - name: Apply Kustomize configuration
      ansible.builtin.shell:
        cmd: kustomize build . | kubectl apply -f -

Perform a check using http://IP_ADDRESS:30060

Expected results

Working standard output in jobs view.

Actual results

standard output in jobs does not work and it does not scroll the results of running ansible playbook. Websocket returns the same error like it is presented in the URL I provided.

Additional information

connectJobSocket.js:17 Uncaught DOMException: Failed to execute 'send' on 'WebSocket': Still in CONNECTING state.
    at WebSocket.onopen (https://awx.sysadmin.homes/static/js/main.13ab8549.js:2:3449626)
onopen  @   connectJobSocket.js:17
react-dom.production.min.js:54 Uncaught Error: Minified React error #188; visit https://reactjs.org/docs/error-decoder.html?invariant=188 for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
    at react-dom.production.min.js:54:67
    at Ze (react-dom.production.min.js:55:217)
    at t.findDOMNode (react-dom.production.min.js:295:212)
    at t.value (CellMeasurer.js:102:33)
    at CellMeasurer.js:48:41    

See logs from awx-web pod

10.42.0.1 - - [05/Apr/2024:15:53:26 +0000] "GET /api/v2/jobs/39/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 HTTP/1.1" 200 52 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 21|app: 0|req: 204/1288] 10.42.0.1 () {70 vars in 1359 bytes} [Fri Apr  5 15:53:25 2024] GET /api/v2/jobs/39/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 => generated 52 bytes in 376 msecs (HTTP/1.1 200) 15 headers in 600 bytes (1 switches on core 0)
127.0.0.1:54360 - - [05/Apr/2024:15:45:19] "WSCONNECTING /websocket/" - -
127.0.0.1:54360 - - [05/Apr/2024:15:45:19] "WSCONNECT /websocket/" - -
127.0.0.1:54308 - - [05/Apr/2024:15:45:28] "WSDISCONNECT /websocket/" - -
127.0.0.1:54360 - - [05/Apr/2024:15:45:28] "WSDISCONNECT /websocket/" - -
127.0.0.1:49622 - - [05/Apr/2024:15:45:28] "WSCONNECTING /websocket/" - -
127.0.0.1:49622 - - [05/Apr/2024:15:45:28] "WSCONNECT /websocket/" - -
127.0.0.1:49622 - - [05/Apr/2024:15:45:30] "WSDISCONNECT /websocket/" - -
127.0.0.1:49636 - - [05/Apr/2024:15:45:30] "WSCONNECTING /websocket/" - -
127.0.0.1:49636 - - [05/Apr/2024:15:45:30] "WSCONNECT /websocket/" - -
127.0.0.1:49638 - - [05/Apr/2024:15:45:31] "WSCONNECTING /websocket/" - -
127.0.0.1:49638 - - [05/Apr/2024:15:45:31] "WSDISCONNECT /websocket/" - -
127.0.0.1:49652 - - [05/Apr/2024:15:45:32] "WSCONNECTING /websocket/" - -
127.0.0.1:49652 - - [05/Apr/2024:15:45:32] "WSCONNECT /websocket/" - -
127.0.0.1:49664 - - [05/Apr/2024:15:45:32] "WSCONNECTING /websocket/" - -
127.0.0.1:49664 - - [05/Apr/2024:15:45:32] "WSCONNECT /websocket/" - -
127.0.0.1:49664 - - [05/Apr/2024:15:45:36] "WSDISCONNECT /websocket/" - -
127.0.0.1:49202 - - [05/Apr/2024:15:45:37] "WSCONNECTING /websocket/" - -
127.0.0.1:49202 - - [05/Apr/2024:15:45:37] "WSCONNECT /websocket/" - -
127.0.0.1:49208 - - [05/Apr/2024:15:45:37] "WSCONNECTING /websocket/" - -
127.0.0.1:49208 - - [05/Apr/2024:15:45:37] "WSCONNECT /websocket/" - -
127.0.0.1:49202 - - [05/Apr/2024:15:45:37] "WSDISCONNECT /websocket/" - -
127.0.0.1:49208 - - [05/Apr/2024:15:45:37] "WSDISCONNECT /websocket/" - -
127.0.0.1:49216 - - [05/Apr/2024:15:45:38] "WSCONNECTING /websocket/" - -
127.0.0.1:49216 - - [05/Apr/2024:15:45:38] "WSCONNECT /websocket/" - -
127.0.0.1:49232 - - [05/Apr/2024:15:45:38] "WSCONNECTING /websocket/" - -
127.0.0.1:49232 - - [05/Apr/2024:15:45:38] "WSCONNECT /websocket/" - -
127.0.0.1:49636 - - [05/Apr/2024:15:46:15] "WSDISCONNECT /websocket/" - -
127.0.0.1:49232 - - [05/Apr/2024:15:46:15] "WSDISCONNECT /websocket/" - -
127.0.0.1:57920 - - [05/Apr/2024:15:46:17] "WSCONNECTING /websocket/" - -
127.0.0.1:57920 - - [05/Apr/2024:15:46:17] "WSCONNECT /websocket/" - -
127.0.0.1:57920 - - [05/Apr/2024:15:46:20] "WSDISCONNECT /websocket/" - -
127.0.0.1:57934 - - [05/Apr/2024:15:46:20] "WSCONNECTING /websocket/" - -
127.0.0.1:57934 - - [05/Apr/2024:15:46:20] "WSCONNECT /websocket/" - -
127.0.0.1:57934 - - [05/Apr/2024:15:48:20] "WSDISCONNECT /websocket/" - -
127.0.0.1:39102 - - [05/Apr/2024:15:48:20] "WSCONNECTING /websocket/" - -
127.0.0.1:39102 - - [05/Apr/2024:15:48:20] "WSCONNECT /websocket/" - -
127.0.0.1:39102 - - [05/Apr/2024:15:48:48] "WSDISCONNECT /websocket/" - -
127.0.0.1:47586 - - [05/Apr/2024:15:48:48] "WSCONNECTING /websocket/" - -
127.0.0.1:47586 - - [05/Apr/2024:15:48:48] "WSCONNECT /websocket/" - -
127.0.0.1:47586 - - [05/Apr/2024:15:48:55] "WSDISCONNECT /websocket/" - -
127.0.0.1:47592 - - [05/Apr/2024:15:48:55] "WSCONNECTING /websocket/" - -
127.0.0.1:47592 - - [05/Apr/2024:15:48:55] "WSCONNECT /websocket/" - -
127.0.0.1:45654 - - [05/Apr/2024:15:48:56] "WSCONNECTING /websocket/" - -
127.0.0.1:45654 - - [05/Apr/2024:15:48:56] "WSCONNECT /websocket/" - -
127.0.0.1:45666 - - [05/Apr/2024:15:48:56] "WSCONNECTING /websocket/" - -
127.0.0.1:45654 - - [05/Apr/2024:15:48:56] "WSDISCONNECT /websocket/" - -
127.0.0.1:45666 - - [05/Apr/2024:15:48:56] "WSDISCONNECT /websocket/" - -
127.0.0.1:45672 - - [05/Apr/2024:15:48:57] "WSCONNECTING /websocket/" - -
127.0.0.1:45672 - - [05/Apr/2024:15:48:57] "WSCONNECT /websocket/" - -
127.0.0.1:45686 - - [05/Apr/2024:15:48:57] "WSCONNECTING /websocket/" - -
127.0.0.1:45686 - - [05/Apr/2024:15:48:57] "WSCONNECT /websocket/" - -
127.0.0.1:47592 - - [05/Apr/2024:15:51:04] "WSDISCONNECT /websocket/" - -
127.0.0.1:45686 - - [05/Apr/2024:15:51:04] "WSDISCONNECT /websocket/" - -
127.0.0.1:47042 - - [05/Apr/2024:15:51:05] "WSCONNECTING /websocket/" - -
127.0.0.1:47042 - - [05/Apr/2024:15:51:05] "WSCONNECT /websocket/" - -
127.0.0.1:47042 - - [05/Apr/2024:15:51:07] "WSDISCONNECT /websocket/" - -
127.0.0.1:36972 - - [05/Apr/2024:15:51:07] "WSCONNECTING /websocket/" - -
127.0.0.1:36972 - - [05/Apr/2024:15:51:07] "WSCONNECT /websocket/" - -
127.0.0.1:36972 - - [05/Apr/2024:15:51:16] "WSDISCONNECT /websocket/" - -
127.0.0.1:50854 - - [05/Apr/2024:15:51:16] "WSCONNECTING /websocket/" - -
127.0.0.1:50854 - - [05/Apr/2024:15:51:16] "WSCONNECT /websocket/" - -
127.0.0.1:50854 - - [05/Apr/2024:15:51:21] "WSDISCONNECT /websocket/" - -
127.0.0.1:50860 - - [05/Apr/2024:15:51:22] "WSCONNECTING /websocket/" - -
127.0.0.1:50860 - - [05/Apr/2024:15:51:22] "WSCONNECT /websocket/" - -
127.0.0.1:50862 - - [05/Apr/2024:15:51:23] "WSCONNECTING /websocket/" - -
127.0.0.1:50862 - - [05/Apr/2024:15:51:23] "WSCONNECT /websocket/" - -
127.0.0.1:50864 - - [05/Apr/2024:15:51:23] "WSCONNECTING /websocket/" - -
127.0.0.1:50864 - - [05/Apr/2024:15:51:23] "WSCONNECT /websocket/" - -
127.0.0.1:50862 - - [05/Apr/2024:15:51:23] "WSDISCONNECT /websocket/" - -
127.0.0.1:50864 - - [05/Apr/2024:15:51:23] "WSDISCONNECT /websocket/" - -
127.0.0.1:50872 - - [05/Apr/2024:15:51:24] "WSCONNECTING /websocket/" - -
127.0.0.1:50872 - - [05/Apr/2024:15:51:24] "WSCONNECT /websocket/" - -
127.0.0.1:50878 - - [05/Apr/2024:15:51:24] "WSCONNECTING /websocket/" - -
127.0.0.1:50878 - - [05/Apr/2024:15:51:24] "WSCONNECT /websocket/" - -
127.0.0.1:50860 - - [05/Apr/2024:15:51:35] "WSDISCONNECT /websocket/" - -
127.0.0.1:50878 - - [05/Apr/2024:15:51:35] "WSDISCONNECT /websocket/" - -
127.0.0.1:38166 - - [05/Apr/2024:15:51:35] "WSCONNECTING /websocket/" - -
127.0.0.1:38166 - - [05/Apr/2024:15:51:35] "WSCONNECT /websocket/" - -
127.0.0.1:38166 - - [05/Apr/2024:15:51:38] "WSDISCONNECT /websocket/" - -
127.0.0.1:52776 - - [05/Apr/2024:15:51:38] "WSCONNECTING /websocket/" - -
127.0.0.1:52776 - - [05/Apr/2024:15:51:38] "WSCONNECT /websocket/" - -
127.0.0.1:52776 - - [05/Apr/2024:15:52:45] "WSDISCONNECT /websocket/" - -
127.0.0.1:54772 - - [05/Apr/2024:15:52:45] "WSCONNECTING /websocket/" - -
127.0.0.1:54772 - - [05/Apr/2024:15:52:45] "WSCONNECT /websocket/" - -
127.0.0.1:54772 - - [05/Apr/2024:15:52:51] "WSDISCONNECT /websocket/" - -
127.0.0.1:43496 - - [05/Apr/2024:15:52:52] "WSCONNECTING /websocket/" - -
127.0.0.1:43496 - - [05/Apr/2024:15:52:52] "WSCONNECT /websocket/" - -
127.0.0.1:43502 - - [05/Apr/2024:15:52:53] "WSCONNECTING /websocket/" - -
127.0.0.1:43502 - - [05/Apr/2024:15:52:53] "WSCONNECT /websocket/" - -
127.0.0.1:43518 - - [05/Apr/2024:15:52:53] "WSCONNECTING /websocket/" - -
127.0.0.1:43518 - - [05/Apr/2024:15:52:53] "WSCONNECT /websocket/" - -
127.0.0.1:43502 - - [05/Apr/2024:15:52:53] "WSDISCONNECT /websocket/" - -
127.0.0.1:43518 - - [05/Apr/2024:15:52:53] "WSDISCONNECT /websocket/" - -
127.0.0.1:43522 - - [05/Apr/2024:15:52:54] "WSCONNECTING /websocket/" - -
127.0.0.1:43522 - - [05/Apr/2024:15:52:54] "WSCONNECT /websocket/" - -
127.0.0.1:43538 - - [05/Apr/2024:15:52:54] "WSCONNECTING /websocket/" - -
127.0.0.1:43538 - - [05/Apr/2024:15:52:54] "WSCONNECT /websocket/" - -
127.0.0.1:43538 - - [05/Apr/2024:15:53:19] "WSDISCONNECT /websocket/" - -
127.0.0.1:36982 - - [05/Apr/2024:15:53:21] "WSCONNECTING /websocket/" - -
127.0.0.1:36982 - - [05/Apr/2024:15:53:21] "WSCONNECT /websocket/" - -
127.0.0.1:36992 - - [05/Apr/2024:15:53:21] "WSCONNECTING /websocket/" - -
127.0.0.1:36992 - - [05/Apr/2024:15:53:21] "WSCONNECT /websocket/" - -
127.0.0.1:43496 - - [05/Apr/2024:15:53:22] "WSDISCONNECT /websocket/" - -
127.0.0.1:36992 - - [05/Apr/2024:15:53:22] "WSDISCONNECT /websocket/" - -
127.0.0.1:37002 - - [05/Apr/2024:15:53:22] "WSCONNECTING /websocket/" - -
127.0.0.1:37002 - - [05/Apr/2024:15:53:22] "WSCONNECT /websocket/" - -
127.0.0.1:37002 - - [05/Apr/2024:15:53:24] "WSDISCONNECT /websocket/" - -
127.0.0.1:37008 - - [05/Apr/2024:15:53:24] "WSCONNECTING /websocket/" - -
127.0.0.1:37008 - - [05/Apr/2024:15:53:24] "WSCONNECT /websocket/" - -
127.0.0.1:37022 - - [05/Apr/2024:15:53:25] "WSCONNECTING /websocket/" - -
127.0.0.1:37022 - - [05/Apr/2024:15:53:25] "WSCONNECT /websocket/" - -
127.0.0.1:37022 - - [05/Apr/2024:15:53:25] "WSDISCONNECT /websocket/" - -
10.42.0.1 - - [05/Apr/2024:15:53:29 +0000] "GET /websocket/ HTTP/1.1" 101 207 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:53:30 +0000] "GET /websocket/ HTTP/1.1" 101 29 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:53:30 +0000] "GET /websocket/ HTTP/1.1" 101 29 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:53:31 +0000] "GET /api/v2/jobs/39/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 HTTP/1.1" 200 4049 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 20|app: 0|req: 218/1289] 10.42.0.1 () {70 vars in 1359 bytes} [Fri Apr  5 15:53:30 2024] GET /api/v2/jobs/39/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 => generated 4049 bytes in 731 msecs (HTTP/1.1 200) 15 headers in 602 bytes (1 switches on core 0)

logs from google chrome browser console in developer tools

Failed to load resource: the server responded with a status of 404 ()
/api/v2/workflow_jobs/62/workflow_nodes/?page=1&page_size=200:1   

Failed to load resource: the server responded with a status of 404 ()
/api/v2/workflow_jobs/62/workflow_nodes/?page=1&page_size=200:1 

Failed to load resource: the server responded with a status of 404 ()
requestCommon.ts:20 

GET https://awx.sysadmin.homes/api/v2/workflow_jobs/62/workflow_nodes/?page=1&page_size=200 404 (Not Found)
He @ requestCommon.ts:20
(anonimowa) @ useGet.tsx:55
t @ index.mjs:653
(anonimowa) @ index.mjs:671
(anonimowa) @ index.mjs:613
(anonimowa) @ index.mjs:245
(anonimowa) @ index.mjs:398
t.isPaused.A.revalidateOnFocus.A.revalidateOnReconnect.t.onErrorRetry.retryCount @ index.mjs:329
Pokaż jeszcze 6 ramek
Pokaż mniej
requestCommon.ts:20 

GET https://awx.sysadmin.homes/api/v2/workflow_jobs/62/workflow_nodes/?page=1&page_size=200 404 (Not Found)

See logs from k3s cluster pod for awx-web

10.42.0.1 - - [05/Apr/2024:14:59:39 +0000] "GET /websocket/ HTTP/1.1" 404 3878 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:14:59:50 +0000] "GET /websocket/ HTTP/1.1" 101 322 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:14:59:50 +0000] "OPTIONS /api/v2/unified_jobs/ HTTP/1.1" 200 11658 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 20|app: 0|req: 141/842] 10.42.0.1 () {72 vars in 1293 bytes} [Fri Apr  5 14:59:50 2024] OPTIONS /api/v2/unified_jobs/ => generated 11658 bytes in 194 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:14:59:50 +0000] "OPTIONS /api/v2/inventory_sources/ HTTP/1.1" 200 24146 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 24|app: 0|req: 191/843] 10.42.0.1 () {72 vars in 1303 bytes} [Fri Apr  5 14:59:50 2024] OPTIONS /api/v2/inventory_sources/ => generated 24146 bytes in 410 msecs (HTTP/1.1 200) 14 headers in 586 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:14:59:51 +0000] "GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 HTTP/1.1" 200 27264 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 260/844] 10.42.0.1 () {70 vars in 1369 bytes} [Fri Apr  5 14:59:50 2024] GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 => generated 27264 bytes in 585 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:14:59:56 +0000] "GET /api/v2/jobs/9/relaunch/ HTTP/1.1" 200 68 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 261/845] 10.42.0.1 () {70 vars in 1254 bytes} [Fri Apr  5 14:59:56 2024] GET /api/v2/jobs/9/relaunch/ => generated 68 bytes in 341 msecs (HTTP/1.1 200) 14 headers in 583 bytes (1 switches on core 0)
2024-04-05 14:59:57,986 INFO     [e7d81c60ac1a4b08af2363573ae59f3a] awx.analytics.job_lifecycle job-11 created
10.42.0.1 - - [05/Apr/2024:14:59:58 +0000] "POST /api/v2/jobs/9/relaunch/ HTTP/1.1" 201 2937 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 262/846] 10.42.0.1 () {76 vars in 1374 bytes} [Fri Apr  5 14:59:56 2024] POST /api/v2/jobs/9/relaunch/ => generated 2937 bytes in 2210 msecs (HTTP/1.1 201) 15 headers in 618 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:14:59:58 +0000] "GET /websocket/ HTTP/1.1" 101 350 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:14:59:59 +0000] "GET /api/v2/unified_jobs/?id__in=11&not__launch_type=sync&order_by=-finished&page=1&page_size=20 HTTP/1.1" 200 2884 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 20|app: 0|req: 142/847] 10.42.0.1 () {70 vars in 1389 bytes} [Fri Apr  5 14:59:58 2024] GET /api/v2/unified_jobs/?id__in=11&not__launch_type=sync&order_by=-finished&page=1&page_size=20 => generated 2884 bytes in 577 msecs (HTTP/1.1 200) 14 headers in 579 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:14:59:59 +0000] "GET /api/v2/unified_jobs/?id=11 HTTP/1.1" 200 2884 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 263/848] 10.42.0.1 () {70 vars in 1259 bytes} [Fri Apr  5 14:59:58 2024] GET /api/v2/unified_jobs/?id=11 => generated 2884 bytes in 688 msecs (HTTP/1.1 200) 14 headers in 579 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:14:59:59 +0000] "GET /api/v2/jobs/11/ HTTP/1.1" 200 3026 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 264/849] 10.42.0.1 () {70 vars in 1238 bytes} [Fri Apr  5 14:59:59 2024] GET /api/v2/jobs/11/ => generated 3026 bytes in 331 msecs (HTTP/1.1 200) 14 headers in 587 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:00 +0000] "OPTIONS /api/v2/jobs/11/job_events/ HTTP/1.1" 200 12315 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 265/850] 10.42.0.1 () {72 vars in 1305 bytes} [Fri Apr  5 14:59:59 2024] OPTIONS /api/v2/jobs/11/job_events/ => generated 12315 bytes in 356 msecs (HTTP/1.1 200) 15 headers in 603 bytes (1 switches on core 0)
[pid: 21|app: 0|req: 139/851] 10.42.0.1 () {70 vars in 1358 bytes} [Fri Apr  5 15:00:00 2024] GET /api/v2/jobs/11/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 => generated 52 bytes in 351 msecs (HTTP/1.1 200) 15 headers in 600 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:00 +0000] "GET /api/v2/jobs/11/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 HTTP/1.1" 200 52 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 266/852] 10.42.0.1 () {72 vars in 1293 bytes} [Fri Apr  5 15:00:00 2024] OPTIONS /api/v2/unified_jobs/ => generated 11658 bytes in 393 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:00 +0000] "OPTIONS /api/v2/unified_jobs/ HTTP/1.1" 200 11658 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:00:00 +0000] "OPTIONS /api/v2/inventory_sources/ HTTP/1.1" 200 24146 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 22|app: 0|req: 115/853] 10.42.0.1 () {72 vars in 1303 bytes} [Fri Apr  5 15:00:00 2024] OPTIONS /api/v2/inventory_sources/ => generated 24146 bytes in 646 msecs (HTTP/1.1 200) 14 headers in 586 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:01 +0000] "GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 HTTP/1.1" 200 32260 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 20|app: 0|req: 143/854] 10.42.0.1 () {70 vars in 1369 bytes} [Fri Apr  5 15:00:00 2024] GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 => generated 32260 bytes in 1164 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:30 +0000] "GET /websocket/ HTTP/1.1" 101 251 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 267/855] 10.42.0.1 () {70 vars in 1260 bytes} [Fri Apr  5 15:00:30 2024] GET /api/v2/project_updates/12/ => generated 5601 bytes in 567 msecs (HTTP/1.1 200) 14 headers in 587 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:31 +0000] "GET /api/v2/project_updates/12/ HTTP/1.1" 200 5601 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:00:31 +0000] "OPTIONS /api/v2/project_updates/12/events/ HTTP/1.1" 200 12382 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 20|app: 0|req: 144/856] 10.42.0.1 () {72 vars in 1319 bytes} [Fri Apr  5 15:00:31 2024] OPTIONS /api/v2/project_updates/12/events/ => generated 12382 bytes in 264 msecs (HTTP/1.1 200) 15 headers in 603 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:31 +0000] "GET /api/v2/project_updates/12/events/?not__stdout=&order_by=counter&page=1&page_size=50 HTTP/1.1" 200 32639 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 22|app: 0|req: 116/857] 10.42.0.1 () {70 vars in 1373 bytes} [Fri Apr  5 15:00:31 2024] GET /api/v2/project_updates/12/events/?not__stdout=&order_by=counter&page=1&page_size=50 => generated 32639 bytes in 342 msecs (HTTP/1.1 200) 15 headers in 603 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:40 +0000] "GET /websocket/ HTTP/1.1" 101 195 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:00:40 +0000] "OPTIONS /api/v2/unified_jobs/ HTTP/1.1" 200 11658 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 21|app: 0|req: 140/858] 10.42.0.1 () {72 vars in 1293 bytes} [Fri Apr  5 15:00:40 2024] OPTIONS /api/v2/unified_jobs/ => generated 11658 bytes in 299 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
[pid: 22|app: 0|req: 117/859] 10.42.0.1 () {72 vars in 1303 bytes} [Fri Apr  5 15:00:40 2024] OPTIONS /api/v2/inventory_sources/ => generated 24146 bytes in 546 msecs (HTTP/1.1 200) 14 headers in 586 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:40 +0000] "OPTIONS /api/v2/inventory_sources/ HTTP/1.1" 200 24146 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:00:41 +0000] "GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 HTTP/1.1" 200 32590 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 268/860] 10.42.0.1 () {70 vars in 1368 bytes} [Fri Apr  5 15:00:40 2024] GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 => generated 32590 bytes in 1479 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:42 +0000] "GET /websocket/ HTTP/1.1" 101 237 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:00:43 +0000] "GET /api/v2/jobs/11/ HTTP/1.1" 200 6618 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 24|app: 0|req: 192/861] 10.42.0.1 () {70 vars in 1238 bytes} [Fri Apr  5 15:00:42 2024] GET /api/v2/jobs/11/ => generated 6618 bytes in 732 msecs (HTTP/1.1 200) 14 headers in 587 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:43 +0000] "OPTIONS /api/v2/jobs/11/job_events/ HTTP/1.1" 200 12315 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 269/862] 10.42.0.1 () {72 vars in 1305 bytes} [Fri Apr  5 15:00:43 2024] OPTIONS /api/v2/jobs/11/job_events/ => generated 12315 bytes in 232 msecs (HTTP/1.1 200) 15 headers in 603 bytes (1 switches on core 0)
[pid: 23|app: 0|req: 270/863] 10.42.0.1 () {70 vars in 1359 bytes} [Fri Apr  5 15:00:43 2024] GET /api/v2/jobs/11/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 => generated 179025 bytes in 167 msecs (HTTP/1.1 200) 15 headers in 604 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:43 +0000] "GET /api/v2/jobs/11/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 HTTP/1.1" 200 179025 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:01:28 +0000] "GET /websocket/ HTTP/1.1" 101 235 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:01:30 +0000] "GET /websocket/ HTTP/1.1" 101 223 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:01:30 +0000] "OPTIONS /api/v2/unified_jobs/ HTTP/1.1" 200 11658 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 24|app: 0|req: 193/864] 10.42.0.1 () {72 vars in 1293 bytes} [Fri Apr  5 15:01:30 2024] OPTIONS /api/v2/unified_jobs/ => generated 11658 bytes in 408 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:01:30 +0000] "OPTIONS /api/v2/inventory_sources/ HTTP/1.1" 200 24146 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 271/865] 10.42.0.1 () {72 vars in 1303 bytes} [Fri Apr  5 15:01:30 2024] OPTIONS /api/v2/inventory_sources/ => generated 24146 bytes in 569 msecs (HTTP/1.1 200) 14 headers in 586 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:01:31 +0000] "GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 HTTP/1.1" 200 32615 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 21|app: 0|req: 141/866] 10.42.0.1 () {70 vars in 1369 bytes} [Fri Apr  5 15:01:30 2024] GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 => generated 32615 bytes in 775 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)

The log snippets indicate interactions with various API endpoints and websocket connections under the /websocket/ path. Here's an analysis of the relevant parts and possible issues:

Websocket Connection Codes (101): The HTTP status code 101 in your logs suggests that websocket connections are being initiated and possibly established ("Switching Protocols"). However, there's an initial 404 error for the websocket endpoint, which could indicate a misconfiguration at some point in time.

Initial 404 Error: The first attempt to access /websocket/ resulted in a 404 (Not Found) error. This could imply that either the websocket service wasn't ready at that moment or there was a misconfiguration in the URL or routing.

Successful Connections: Subsequent attempts to connect to the websocket endpoint returned a 101 status, which means the server accepted the request to switch protocols from HTTP to WebSockets.

Other API Requests: The log shows successful HTTP requests (status 200) to various API endpoints, indicating that the regular HTTP API seems to be functioning well.

kurokobo commented 7 months ago

Perhaps dup of: https://github.com/ansible/awx/issues/15038 It's already addressed and the fix will be shipped with the next versioned release.

rexberg commented 7 months ago

This was addressed in the 24.2.0 release.

bbourchanin commented 7 months ago

Still have "Minified React error #188;" on 24.2.0.

When i append url with "?lang=fr" or "?lang=en", or "?no_stdout=" ... it works

Edit: Ok i have resolved my pb by deleting on ansible.cfg

display_ok_hosts=false

display_skipped_hosts=false

fosterseth commented 7 months ago

@sysadmin-info are you still having this issue? if not, we can go ahead and close this ticket. Thanks!

sysadmin-info commented 7 months ago

Hi, I have not checked it yet. Close it. Until next week I will not have time to perform a check. Kind regards

On Fri, Apr 12, 2024 at 7:44 PM Seth Foster @.***> wrote:

@sysadmin-info https://github.com/sysadmin-info are you still having this issue? if not, we can go ahead and close this ticket. Thanks!

— Reply to this email directly, view it on GitHub https://github.com/ansible/awx/issues/15071#issuecomment-2052203130, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFULAOTUKG6G3MM35PSFNVLY5AMQTAVCNFSM6AAAAABF2H7QLSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJSGIYDGMJTGA . You are receiving this because you were mentioned.Message ID: @.***>

TheRealHaoLiu commented 6 months ago

@sysadmin-info thanks for the extremely detailed issue, please check the latest release to see if the problem can be reproduced

as for the UI change please submit a PR.

closing this issue for now since its been more than a month and other have reported the root cause has been addressed