cubewise-code / tm1py

TM1py is a Python package that wraps the TM1 REST API in a simple to use library.
http://tm1py.readthedocs.io/en/latest/
MIT License
191 stars 111 forks source link

Support for Decoupled Async Execution and Polling #963

Closed rclapp closed 10 months ago

rclapp commented 1 year ago

When operating in async mode TM1py manages the entire lifecycle of an TM1 async operation. The library

  1. Adds a prefer-async header to the request
  2. Receives a Job ID for the request by parsing the response header
  3. Polls TM1 for the status of the JobID
  4. Then when the operation is complete, returns the full response to the original requesting method

This method works for some applications, but is not ideal for serverless implementations like AWS Lamdba. In these use cases we want to reduce the total runtime of the function be separating the TM1py call from polling for the status of the request. This feature will expose the async job id to the request creator and allow them to deal with polling and the subsequent response.

MariusWirtz commented 10 months ago

Feature is implemented for execute_process_with_return function with #1000 and #1035

This will be rolled out to other functions gradually over the next releases.

Example:

with TM1Service(**tm1params) as tm1:
    async_id = tm1.processes.execute_with_return(
        process_name="}bedrock.server.wait",
        pWaitSec=10,
        return_async_id=True)

    done = None
    while not done:
        time.sleep(1)
        done = tm1.processes.poll_execute_with_return(async_id)

    success, status, error_log_file = done
    print(success, status, error_log_file)
macsir commented 3 months ago

This is cool. However, there is a limitation that this async job ID can't be shared across TM1 sessions. In order to decouple, we need to pass the session id as well with async id after creating this job in one AWS lambda so that another lambda can pick them up for polling in the same session.