realdevnullius
commented
3 years ago You could limit the amount of plots with the phase options, no?
# Optional: Allows the overriding of some scheduling characteristics of the
# tmp directories specified here.
# This contains a map of tmp directory names to attributes. If a tmp directory
# and attribute is not listed here, the default attribute setting from the main
# configuration will be used
#
# Currently support override parameters:
# - tmpdir_stagger_phase_major (requires tmpdir_stagger_phase_minor)
# - tmpdir_stagger_phase_minor (requires tmpdir_stagger_phase_major)
# - tmpdir_stagger_phase_limit
# - tmpdir_max_jobs
tmp_overrides:
# In this example, /mnt/tmp/00 is larger and faster than the
# other tmp dirs and it can hold more plots than the default,
# allowing more simultaneous plots, so they are being started
# earlier than the global setting above.
#"/mnt/tmp/00":
# tmpdir_stagger_phase_major: 1
# tmpdir_stagger_phase_minor: 5
# tmpdir_max_jobs: 5
# Here, /mnt/tmp/03 is smaller, so a different config might be
# to space the phase stagger further apart and only allow 2 jobs
# to run concurrently in it
# QUESTION HOW TO PLAY WITH THESE PHASES?? :(
#"/mnt/tmp/03":
# tmpdir_stagger_phase_major: 3
# tmpdir_stagger_phase_minor: 1
# tmpdir_max_jobs: 2
When configured without a destination the software leaves it in the temp directory it created it in. In my case this is 1 of 24x 10 SAS hard drives. Then the archiver comes along and transfers it to the remote system. While the archiver is trying to do its thing plotman says hey I got a free temp directory and starts up a new plot on that drive slamming the transfer rate of the archiver. This slows things WAY down. Maybe an option could be added that just checks if a .plot file exists in the temp directory consider it not ready.