sbatch: error: Batch job submission failed: Socket timed out on send/recv operation

[delgadocc]

Dear IMI team,

First of all I wanted to congratulate you for the splendid job you have done developing IMI. This is a very valuable ready-to-run tool for early-career researchers like me.

I recently started doing the tutorial that you have in the documentation. I was able to create the instance on AWS and run the example for the Permian Basin with rectangular grid (CreateAutomaticRectilinearStateVectorFile: true) without much hassle.

However, I have tried to reproduce the same example by first creating the state vector from the Shapefile (Jupyter notebook) and then running the code with the option CreateAutomaticRectilinearStateVectorFile: false. Everything goes well until "SUBMITTING POSTERIOR SIMULATION" where the simulation crash with the following message: "sbatch: error: Batch job submission failed: Socket timed out on send/recv operation".

I did the tests on both IMI versions (1.0 and 1.1) without any success. Attached you will find the config and log files config.txt imi_output.log

Thanks in advance

[laestrada]

Hi @delgadocc,

Based on the output log you attached, it appears that your EC2 instance has run out of storage space (check the last line in the output file). I suspect that this is then causing the sbatch command to fail. You can expand the root storage volume on your instance by following these instructions. I recommend expanding by 200GB to be safe.

It looks like all the other steps of the IMI succeeded except for the posterior simulation, so in your config file you can turn off the preliminary setup and run steps like so:

SetupTemplateRundir: false 
SetupSpinupRun: false
SetupJacobianRuns: false
SetupInversion: false
SetupPosteriorRun: false

## Run modules
##   Turn on/off different steps in performing the inversion
RunSetup: false
DoSpinup: false
DoJacobian: false
DoInversion: false
DoPosterior: true

Thanks for writing in and we hope you continue using the IMI for your research needs! Let us know if the above works so we can offer additional assistance or close the issue.