search

inductiva / wind-tunnel

Example of a virtual Wind Tunnel implementation with Inductiva API.
0 stars 2 forks source link

Script to monitor and download tasks #4

Closed AugustoPeres closed 7 months ago

AugustoPeres commented 8 months ago

Using the tasks_metadata files automatically logged by the api this script can monitor the status of the tasks and download their outputs.

Here I made the choice of downloading their outputs to the folder wind_tunnel_input__TASK_NUMBER. Even though it is strange to download tasks to a file with inputs this has the advantage that we use our file manager to keep the outputs of the task and its inputs together without any extra code. This way, later when post-processing, we can simply look for the downloaded_outputs folder inside each task and apply any processing steps that we may want.

AugustoPeres commented 8 months ago

I have one doubt! Imagine that I run this script but I still have some tasks running. The download of the success tasks proceeds and goes well. Now, I want to check the rest of the tasks that didn't had finished yet, but now already have. Won't I download everything again?

You would download that again. Just set the variable DOWNLOAD_TASKS=False until you are ready for download