rdirocco
commented
3 years ago for anyone that cares to do the same, here is some bash that you can use, loop it, cron it, whatever
#!/bin/bash
main () {
echo "Checking PIDs and Making Nice!"
nice_p1
nice_p234
}
function nice_p1 {
jobs=($(plotman status | awk {'print $1" "$6'} | grep "1:" | awk {'print $1'}))
if [[ ${#jobs[@]} -gt 0 ]]
then
echo "- Jobs in Phase 1:"
for i in $(echo ${jobs[@]}); do
pid=$(plotman details $i | grep "pid:" | cut -d ":" -f 2)
echo " -- PID: "$pid
echo " --- Checking NICE "
nice=$(ps --no-headers -o ni $pid)
if [[ $nice -ne -19 ]]
then
echo " ----"`renice --priority -19 --pid $pid`
else
echo " --- Good, continuing"
fi
done
fi
}
function nice_p234 {
jobs=($(plotman status | awk {'print $1" "$6'} | grep -v "1:" | awk {'print $1'} | tail -n +2))
if [[ ${#jobs[@]} -gt 0 ]]
then
echo "- Jobs in Phase 2/3/4:"
for i in $(echo ${jobs[@]}); do
pid=$(plotman details $i | grep "pid:" | cut -d ":" -f 2)
echo " -- PID: "$pid
echo " --- Checking Nice"
nice=$(ps --no-headers -o ni $pid)
if [[ $nice -ne 15 ]]
then
echo " ----"`renice --priority 15 --pid $pid`
else
echo " --- Good, continuing"
fi
done
fi
}
main"$@"
fer662
MartinEderer
altendky
Feature Request
Support configurable Phase1 and Phase2/3/4 NICE levels in the config Ex:
On launch of plotting processes set NICE of -19, as per example At Phase 1 completion, renice PID of respective job to 15, as per example
Background:
much of my plotting load testing has lead me to the conclusion that over-subscription of machine cores (threads) at a near 2:1 ratio nets optimal parallel plotting, while sacrificing a bit of phase1 speed. The objective is to minimize the peak/valley behavior and fully utilize the compute resources available, within the limits of the storage. Phase1 is multi-threaded and compared to the other phases, extremely CPU bound. In the example configuration 32 threads would be allocated to Phase1 with an additional 11 threads for the jobs in phase2/3/4. , 43 threads allocated on 32 threads, roughly a 34% over subscription.
Example Hardware Config:
Ryzen 5950x (16c / 32t) 64GB DDR4-3600 4 x 1TB NVME TMP (mdadm raid-0, xfs, zram logdev, noatime mounts, etc) 1 x 500GB SSD Destination
Example Plotman config:
4 max before stage 2:0, 10 second scan, 10 minute stagger, 15 max plots, memory 4096, threads 8
Data:
Testing has shown that for all PIDs of jobs in Phase1, setting a nice of -19 and all other jobs at a default 15 ensures that adequate (all the things...) CPU resources are allocated to quickly progress the P1 jobs to a state where everyone gets their own single physical core in 2+. In ~24 hours of testing this has resulted in about an 8% gain in reduction of total plot time.