Open first ticket in HPC for PEMA

[savvas-paragkamian]

Put in action the script https://github.com/savvas-paragkamian/isd-crete/blob/main/scripts/isd_crete_hpc_job.sh

[savvas-paragkamian]

A preliminary job was executed using PEMA using 10 files, 5 samples. The job was active for 13 hours and the output was 5gb in storage.

In total, there are 280 files for 140 samples, so to complete the PEMA analysis it will require 364 hours in a single BATCH node and 140 gb in storage. In 4 nodes it will take 5 days. To speed up the process I opened a ticket in the HCMR-HPC for 4 nodes and 6 days per node and 170 gb.

[savvas-paragkamian]

Created 4 directories

job1/mydata job2/mydata job3/mydata job4/mydata and in each directory added 70 files (25 samples)

ls -1 | gawk 'NR>=1 && NR<=70'| xargs -I{} cp "{}" ../job1/mydata/

Than run 4 jobs.

[savvas-paragkamian]

All jobs finished.

Storage: 204G Job1 : 411463 sec, 4.76 days, storage 42G Job2 : 400039 sec, 4.64 days, storage 41G Job3 : 468861 sec, 5.42 days, storage 48G Job4 : 507915 sec, 5.87 days, storage, 52G