search

zowe / zowe-cli

Zowe CLI
Eclipse Public License 2.0
108 stars 85 forks source link

Allow passing a destination URL while submitting a job #595

Open Alexandru-Dumitru opened 4 years ago

Alexandru-Dumitru commented 4 years ago

Restjobs PUT method (used in submitting a job) has a custom header, X-IBM-Notification-URL, which allows passing a destination URL for receiving an HTTP POST when the job is no longer eligible for execution (reached output or purge queue).

This feature might prove valuable for other applications built on top of Zowe CLI, because there is no need to poll for job status anymore, if you can consume a REST call.

Implementation details here. Configuration pre-req details here.

martin-pala-broadcom commented 4 years ago

+1

The current client-side busy-wait for --wait-for-output or --view-all-spool-content option is not very resource friendly both on client and server side ... if you run for example 8 sessions with --wait-for-output you run 8 busy-wait loops, which are polling status every 3 seconds.

zFernand0 commented 4 years ago

Hi @martin-pala-broadcom, What if you could configure the polling yourself by specifying a delay time (of 10 seconds instead of 3) or maybe less retries (6 instead of 100)

I believe @tjohnsonBCM was looking into this recently : )
See #549 for more details

martin-pala-broadcom commented 4 years ago

@zFernand0 the asynchronous notification via X-IBM-Notification-URL callback would be much more effective:

  1. speed: the X-IBM-Notification-URL will allow one to get the notification immediately when the job finished (for example if the job finished within 0.1s after submission, you can continue immediately vs. e.g. 10s poll cycle of the busy-wait loop)

  2. resources: even if you'll rise the poll cycle to 10s and the job will be hanging in the queue for 5 minutes, you'll perform 6*5=30 HTTP requests. The X-IBM-Notification-URL will save this overhead: just one zowe->mainframe HTTP request for job submission and one HTTP request from mainframe->zowe-webservice for the job completion notification. If e.g. 50 developers will run their jobs like this, it'll waste lots of resources on the mainframe side.

  3. parallelism: if you need to submit e.g. 10 jobs with --wait-for-output in parallel, you run 10 busy-wait loops, growing the load with O(n) on the client side.

The #549 is example of the job, where the busy-wait wastes resources both on client and server side and where X-IBM-Notification-URL would be much better approach.

zFernand0 commented 4 years ago

@martin-pala-broadcom Sorry about that. I should have said that that would be a temporary solution.

I do agree that async operations are often better. Thanks for the informational reply. 😄

github-actions[bot] commented 1 year ago

Thank you for raising this enhancement request. The community has 90 days to vote on it. If the enhancement receives at least 5 upvotes, it is added to our development backlog. If it receives fewer votes, the issue is closed.