Run `Models --apply` and `Models` interactively for few samples

[DininduSenanayake]

First step is to make sure we can run at least few samples.

1. Start with opening a Slurm interactive session srun --cpus-per-task 8 --mem 24G --time 04:00:00 --pty bash

2. Working directory is /agr/persist/projects/2024_apsim_improvements/apsim-simulations/ConfigFiles

3. APSIM version : 2024.09.7579.0

#!/bin/bash -e

module load Apptainer
export APPTAINER_BIND="/agr/scratch,/agr/persist"

export APPTAINER_CMD="apptainer exec /agr/persist/projects/2024_apsim_improvements/apsim-simulations/container/apsim-2024.09.7579.0.aimg"

${APPTAINER_CMD} Models --apply Ahuriri_7a1_13223ConfigFile.txt
${APPTAINER_CMD} Models --apply Ahuriri_7a1_19487ConfigFile.txt
${APPTAINER_CMD} Models --apply Ailsa_5a1_13223ConfigFile.txt

I can see the .apsimx files were generated for each file as per ( Ignore the )

└$ ls *.apsimx
13223_Ahuriri_7a1.apsimx  13223_Ailsa_5a1.apsimx  19487_Ahuriri_7a1.apsimx

However, it creates the .db files as part of --apply which I have treated as a place holder as they do not carry any weight

$ find ./ -name "*.db" -type f -exec du -h {} + | sort -rh

32K ./19487_Ahuriri_7a1.db
32K ./13223_Ailsa_5a1.db
32K ./13223_Ahuriri_7a1.db

Then I have ran ${APPTAINER_CMD} Models to three .apsimx files without deleteing the .db place holder files

#!/bin/bash -e

module load Apptainer
export APPTAINER_BIND="/agr/scratch,/agr/persist"

export APPTAINER_CMD="apptainer exec /agr/persist/projects/2024_apsim_improvements/apsim-simulations/container/apsim-2024.09.7579.0.aimg"

${APPTAINER_CMD} Models 13223_Ahuriri_7a1.apsimx
${APPTAINER_CMD} Models 13223_Ailsa_5a1.apsimx
${APPTAINER_CMD} Models 19487_Ahuriri_7a1.apsimx

All looks good based on the file size

$ find ./ -name "*.db" -type f -exec du -h {} + | sort -rh
27M ./19487_Ahuriri_7a1.db
27M ./13223_Ahuriri_7a1.db
26M ./13223_Ailsa_5a1.db

[DininduSenanayake]

Part 2 : Run the same set of commands as sbatch

Since we want to make sure both Models --apply and Models works via sbatch ( verifying it isn't subject hangs,etc), I have used the same set of samples and fed it through a Slurm script

NOTE - Made sure to delete all of the existing .apsimx and .db files from previous run

#!/bin/bash -e

#SBATCH --job-name=apsim_models
#SBATCH --output=slurmlogs/%A_%a.out
#SBATCH --cpus-per-task=4
#SBATCH --mem=18G
#SBATCH --time=00:20:00

module load Apptainer
export APPTAINER_BIND="/agr/scratch,/agr/persist"

export APPTAINER_CMD="apptainer exec /agr/persist/projects/2024_apsim_improvements/apsim-simulations/container/apsim-2024.09.7579.0.aimg"

#create .apsimx files : serially
${APPTAINER_CMD} Models --apply Ahuriri_7a1_13223ConfigFile.txt
${APPTAINER_CMD} Models --apply Ahuriri_7a1_19487ConfigFile.txt
${APPTAINER_CMD} Models --apply Ailsa_5a1_13223ConfigFile.txt

#populate .db files : serially
${APPTAINER_CMD} Models 13223_Ahuriri_7a1.apsimx
${APPTAINER_CMD} Models 13223_Ailsa_5a1.apsimx
${APPTAINER_CMD} Models 19487_Ahuriri_7a1.apsimx

• Confirming the .db files carry the expected weight

❯ find ./ -name "*.db" -type f -exec du -h {} + | sort -rh
27M ./19487_Ahuriri_7a1.db
27M ./13223_Ahuriri_7a1.db
26M ./13223_Ailsa_5a1.db

• Quick check on job stats

❯ seff 985625
Job ID: 985625
Cluster: eri
User/Group: dinindu/dinindu
State: COMPLETED (exit code 0)
Nodes: 1
Cores per node: 4
CPU Utilized: 00:21:03
CPU Efficiency: 77.01% of 00:27:20 core-walltime
Job Wall-clock time: 00:06:50
Memory Utilized: 1.75 GB
Memory Efficiency: 9.71% of 18.00 GB