search

geodesymiami / rsmas_insar

RSMAS InSAR code
https://rsmas-insar.readthedocs.io/
GNU General Public License v3.0
62 stars 23 forks source link

launcher Bus error although job was successful #456

Open falkamelung opened 3 years ago

falkamelung commented 3 years ago

Occasionally I get a launcher Bus error but when I run again it works just fine. I checked the memory using free -g and there was 60GB free.

cat run_05_fullBurst_resample_3_7157479.e
using /tmp/launcher.7157479.hostlist.imUn1ez1 to get hosts
starting job on c506-083
/opt/apps/launcher/launcher-3.7/launcher: line 93: 285520 Bus error               SentinelWrapper.py -c /scratch/05861/tg851601/wd2KokoxiliBigChunk36SenDT48/configs/config_fullBurst_resample_20190727 > /scratch/05861/tg851601/wd2KokoxiliBigChunk36SenDT48/run_files/run_05_fullBurst_resample_3_20190727_$LAUNCHER_JID.o 2> /scratch/05861/tg851601/wd2KokoxiliBigChunk36SenDT48/run_files/run_05_fullBurst_resample_3_20190727_$LAUNCHER_JID.e
ll *05*715*.e
-rw-rw---- 1 tg851601 G-820134  84 Jan 21 05:05 run_05_fullBurst_resample_0_7157476.e
-rw-rw---- 1 tg851601 G-820134  84 Jan 21 06:05 run_05_fullBurst_resample_0_7157566.e
-rw-rw---- 1 tg851601 G-820134  84 Jan 21 05:06 run_05_fullBurst_resample_1_7157477.e
-rw-rw---- 1 tg851601 G-820134  84 Jan 21 05:15 run_05_fullBurst_resample_2_7157478.e
-rw-rw---- 1 tg851601 G-820134 521 Jan 21 05:17 run_05_fullBurst_resample_3_7157479.e
-rw-rw---- 1 tg851601 G-820134  84 Jan 21 06:20 run_05_fullBurst_resample_3_7157623.e
-rw-rw---- 1 tg851601 G-820134  84 Jan 21 05:20 run_05_fullBurst_resample_4_7157480.e
-rw-rw---- 1 tg851601 G-820134  84 Jan 21 05:22 run_05_fullBurst_resample_5_7157481.e
sacctj 7157479
  NNodes  Timelimit   Reserved    Elapsed      State 
-------- ---------- ---------- ---------- ---------- 
       1   00:20:00   00:13:09   00:12:43  COMPLETED 
       1                         00:12:43  COMPLETED 
       1                         00:00:01  COMPLETED 
//login3/scratch/05861/tg851601/wd2KokoxiliBigChunk36SenDT48/run_files[1119] sacctj 7157623
  NNodes  Timelimit   Reserved    Elapsed      State 
-------- ---------- ---------- ---------- ---------- 
       1   00:20:00   00:00:03   00:12:38  COMPLETED 
       1                         00:12:38  COMPLETED 
       1                         00:00:02  COMPLETED