google / weather-tools

Tools to make weather data accessible and useful.
https://weather-tools.readthedocs.io/
Apache License 2.0
208 stars 39 forks source link

Add new functionality (--async) in weather-dl to terminate tool after dataflow job launched. #373

Open dabhicusp opened 1 year ago

dabhicusp commented 1 year ago

Let's add a new flag like --async to weather-dl that enables the tool to automatically terminate and exit gracefully after getting the specific keyword from the log such as JOB_IS_STARTED.

Presently, upon completion of the entire dataflow job, our program terminates, resulting in significant time consumption. However, with the above modifications, our program exits as soon as the data flow job is launched.