[Enhancement] [Epic] [zos_job_submit, zos_job_query, zos_job_output] support Ansible async controls

ddimatos commented 9 months ago

Is there an existing issue for this?

[X] There are no existing issues.

Ansible module

zos_job_submit

Enhancement or feature description

The collection does not explicitly support Ansible async , when trying to use it with zos_job_submit the error async is not supported for this task appears. Further evaluation, to support async plugins specifically have to declare self._supports_async = True but its more than simply this entry, the module itself has to deal with running in the async setting.

This epic should have issues created:

Research usage and impact of self._supports_async = True and answer the question if our current architecture can support this and if not what needs to change.
Research how supporting async might impact a future enhancement where module zos_job_submit could accept a list of jobs to submit , the JSON responses were designed to accommodate more than one job submission, it would be good to understand if such a the multi job submission feature were might be limited by ansync, for example if both async and multiple jobs were supported we might need to disable async when running more than one job else asynchronously looping over a list of jobs might not lend itself to poll.
Issues to track the development if research determines architecture supports asynch

For reference:

see the services.py module:
messages from Brian
Comments on this PR
Documentation using async
Review lopp_var controls , will help with evaluation

An example which I can't fully evaluate because of the error: zos_job_submit_loop_async.yml

---
- hosts: zvm
  collections:
    - ibm.ibm_zos_core
  gather_facts: false

  environment: "{{ environment_vars }}"

  tasks:

    - name: Create 14 more copies of 1.jcl for demonstration
      ansible.builtin.copy:
        dest: "/tmp/{{ item }}.jcl"
        src: /tmp/1.jcl
      with_sequence: start=2 end=15
      delegate_to: localhost

    - name: Submit jobs asynchronously
      vars:
        jobs:
          - /tmp/1.jcl
          - /tmp/2.jcl
          - /tmp/3.jcl
          - /tmp/4.jcl
          - /tmp/5.jcl
          - /tmp/6.jcl
          - /tmp/7.jcl
          - /tmp/8.jcl
          - /tmp/9.jcl
          - /tmp/10.jcl
          - /tmp/11.jcl
          - /tmp/12.jcl
          - /tmp/13.jcl
          - /tmp/14.jcl
          - /tmp/15.jcl
        job: "{{ item }}"
      include_tasks: submit_jobs.yml
      loop: "{{ jobs | batch(5) | list }}"

submit_jobs.yml


    - name: Submit local jcl found on controller
      zos_job_submit:
        src: "{{ async_item }}"
        location: LOCAL
        encoding:
          from: ISO8859-1
          to: IBM-1047
        wait_time_s: 1
        use_template: false
      async: 600
      poll: 0
      loop: "{{ job }}"
      loop_control:
        loop_var: "async_item"
      ignore_errors: true
      register: result

    # - name: Wait until job completes
    #   zos_job_query:
    #     job_id: "{{ result.job_id }}"
    #   register: result
    #   until: result.jobs[0].ret_code.code == 0
    #   retries: 100
    #   delay: 2

    # - name: Results
    #   debug:
    #     var: result
    #   when: result.jobs[0].ret_code.code == 0

Error:

failed: [zvm] (item=/tmp/3.jcl) => {
    "ansible_loop_var": "async_item",
    "async_item": "/tmp/3.jcl",
    "msg": "async is not supported for this task."
}

Related issues:

[ ] Issue [Enabler] [Research] [zos_job_submit, zos_job_query, zos_job_output] support Ansible async controls #1743

ddimatos commented 5 months ago

2 work arounds for this issue, one has a 5 second async delay which uses ansyc_status, while the other with no delay does not use async_status (more of a submit and forget model).

- hosts: zvm
  collections:
    - ibm.ibm_zos_core
  gather_facts: false
  vars:
    files:
      - file_name : "UPTIME"
      - file_name : "hello.jcl"

  environment: "{{ environment_vars }}"

  tasks:

    - name: Copy files from control node to USS
      ibm.ibm_zos_core.zos_copy:
        src: "{{playbook_dir}}/files/{{ item.file_name }}"
        dest: "/tmp/{{ item.file_name }}"
        remote_src: false
        force: true
      register: result
      loop: "{{ files }}"

    - name: Result
      ansible.builtin.debug:
        var: result

    - name: Use ZOAU jsub to submit jobs asynchronously, place result in async_submit
      ansible.builtin.shell: jsub -f "/tmp/{{ item.file_name }}"
      register: async_submit
      loop: "{{ files }}"
      async: 0
      poll: 0

    - name: Print complete result for async task registered with var async_submit
      ansible.builtin.debug:
        var: async_submit.results

    - name: Loop over the Job IDs and capture them into a var called job
      zos_job_output:
        job_id: "{{ item.stdout }}"
      register: jobs
      loop: "{{ async_submit.results }}"

    - name: Print all async jobs
      ansible.builtin.debug:
        var: jobs

and

---
- hosts: zvm
  collections:
    - ibm.ibm_zos_core
  gather_facts: false
  vars:
    files:
      - file_name : "UPTIME"
      - file_name : "hello.jcl"

  environment: "{{ environment_vars }}"

  tasks:

    - name: Copy files from control node to USS
      ibm.ibm_zos_core.zos_copy:
        src: "{{playbook_dir}}/files/{{ item.file_name }}"
        dest: "/tmp/{{ item.file_name }}"
        remote_src: false
        force: true
      register: result
      loop: "{{ files }}"

    - name: Result
      ansible.builtin.debug:
        var: result

    - name: Use ZOAU jsub to submit jobs asynchronously, place result in async_submit
      ansible.builtin.shell: jsub -f "/tmp/{{ item.file_name }}"
      register: async_submit
      loop: "{{ files }}"
      async: 5
      poll: 0

    - name: Print complete result for async task registered with var async_submit
      ansible.builtin.debug:
        var: async_submit.results

    - name: Print the async_submit async ansible_job_id, special var used for async call
      ansible.builtin.debug:
        msg: "ansible_job_id is --> {{ item.ansible_job_id }}"
      loop: "{{ async_submit.results }}"

    - name: Monitor when the async jsub completes and capture jsub JCL Batch Job ID, try this up to 10 times
      async_status:
        jid: "{{ item.ansible_job_id }}"
      register: async_result
      until: async_result.finished
      retries: 10
      delay: 10
      loop: "{{ async_submit.results }}"

    - name: Print the async_result, this should now contain the job ID
      ansible.builtin.debug:
        var: async_result

    - name: Loop over the Job IDs and capture them into a var called job
      zos_job_output:
        job_id: "{{ item.stdout }}"
      register: jobs
      loop: "{{ async_result.results }}"

    - name: Print all async jobs
      ansible.builtin.debug:
        var: jobs

ddimatos commented 3 months ago

My concern is how async will impact our future work items:

Being able to submit more than one job
Being able to submit a job with more than on job card – this work has been started

In summary, what we need to determine is:

How async might impact bullets (1) or (2) above
If there is impact, do we see the need for either of those functions, maybe when a list of jobs is provided, the async code be disabled
Do we really need bullet (1) if we are able to use async in a loop, it is submitting multiple jobs so maybe it is not necessary ( am leaning towards this being true that async support will negate the need for bullet (1))

ansible-collections / ibm_zos_core