inductiva / wind-tunnel

Example of a virtual Wind Tunnel implementation with Inductiva API.
0 stars 2 forks source link

Add large scale job submission #3

Closed AugustoPeres closed 2 months ago

AugustoPeres commented 8 months ago

Contents

This pull request adds the first part of submitting a large job to the API through the script job_submission.py. Essentially the user gives it a path to a folder with .obj and the script will run num_simulations_per_object experiments.

Experiment management

While there is no automatic experiment management provided natively through the api the user can use the variableworking_dir. This sets the inductiva.working_dir variable ensuring all the API logs are sent to that directory. This logging is already done automatically through the API.

For example, setting WORKING_DIR=../../wind_tunnel_experiments/experiment_1 will create under the given working dir:

task_metadata.json  wind_tunnel_input__138  wind_tunnel_input__178  wind_tunnel_input__23   wind_tunnel_input__63
wind_tunnel_input   wind_tunnel_input__139  wind_tunnel_input__179

Where task_metadata.json contains information about all simulations submited.

~### Extras~

~* Changed the __init__ and added a setup.py file to make the make lib instalable with pip install -e .;~

~### Simplifications with respect to previous iterations~

~Here the submission code is much cleaner as the API is already storing the logs and the inputs to each task. This removes the extra necessary code for experiment management. To get a better understanding of what is no longer necessary look here, here and here~

rcvalerio commented 2 months ago

This pull request is using a very outdated version of the code. With the current codebase large scale job submission is very easy to do, see batch_run.py