SMART-Lab / smartdispatch

An easy to use job launcher for supercomputers with PBS compatible job manager.
Do What The F*ck You Want To Public License
34 stars 18 forks source link

Change logging of smartdispatch #176

Open SinaHonari opened 6 years ago

SinaHonari commented 6 years ago

There are two log parts that ideally I would like to change:

1- The dir name inside SMART_DISPATCH_LOGS: I get names like this: 2017-10-2710-25-4931_AFLW_gan_test_gan_coef/
2017-10-2710-33-5631_AFLW_gan_test_gan_coef/
2017-10-2712-18-4831_AFLW_gan_test_gan_coef_c_smart/ 2017-10-2712-26-2531_AFLW_gan_test_gan_coef_c_smart/

So if I have different jobs they are sorted by date which is a bit difficult to find logs corresponding to different experiments, while I would sort them by name and if the date comes afterwards that would be fine (I would know the order of running jobs for a particular experiment). So I would prefer something like this instead: 31_AFLW_gan_test_gan_coef_2017-10-27_10-25-49/

2- the name of the err and out files: I have log files like this: 8e556a8dfdbc7bbad3cc64869d1d69d30a5a7b26e40939ea9339b77c7636a7d5.err c1d98dd2cf5e71b2a8b5140914943c6a756651dc52e386076afb0a7718f29cac.err
8e556a8dfdbc7bbad3cc64869d1d69d30a5a7b26e40939ea9339b77c7636a7d5.out c1d98dd2cf5e71b2a8b5140914943c6a756651dc52e386076afb0a7718f29cac.out

Now if I run 20 jobs, it is not easy to find them (checking logs corresponding to the configurations I run). Instead I would prefer them having the same base name and then having different suffix (e.g. _1 _2 _3 and so on). This is what I get on helios for job_dispatch and it is convenient. It has same base name and then the suffix is always ordered as the jobs are submitted to cluster. the first submitted jobs gets _1, the second gets _2 and so on.

I think the first dir name (since it is date based) makes jobs distinguishable and ordered well and the second just make it easier to see the logs corresponding to different configurations.

Having these two changed, makes navigating in the logs much easier.

bouthilx commented 6 years ago

I agree with both of your suggestions. @mgermain Do you have any objections?