tueda / formset

A "form.set" generator.
MIT License
2 stars 0 forks source link

How to find parameters for PBS cluster jobs? #7

Open ahamaline opened 11 months ago

ahamaline commented 11 months ago

Hi, this is not really an "issue" - just a request for guidance. At my university we have a high-power computing cluster, with jobs submitted via PBS as shell scripts, from a login node that we connect to using SSH. I have some heavy FORM calculations that I'd like to do on the cluster. How would I go about determining the optimal values for the various FORM parameters?

For example, does it make sense to get those values by submitting a job that runs Formset on the cluster?

tueda commented 11 months ago

A conservative answer is, well, probably it depends on your task and cluster, I think. You need some testing.

If each node of the cluster has enough memory and if your FORM jobs with the default parameters are not eating up the memory (and if your jobs handle gigantic expressions such that it causes many slow disk accesses), then increasing the sizes of the sort buffers may make a difference on the performance.

In such a case, if the cluster is heterogeneous and the memory size may be different for each node, running formset (to generate form.set in the working directory) before executing tform (which uses form.set in the current directory by default) in a batch job makes sense.

If the node memory size is always the same (and your jobs are well balanced), submitting one job to generate a common form.set is sufficient.