search

aiidateam / aiida-core

The official repository for the AiiDA code
https://aiida-core.readthedocs.io
Other
434 stars 188 forks source link

lsf scheduler plugin time string parsing, time format differs #1709

Open broeder-j opened 6 years ago

broeder-j commented 6 years ago

On one of our machines running lsf time has the format '%b %d %H:%M:%S %Y ' which is not the expected format '%Y %b %d %H:%M L' by aiida....

Therefore, in the current implementation the lsf plugin won't work for this machine...

For our machine I have just hacked it. Through I do not know what other time formats might be out there and how we can in general cover for it... besides trying out a bunch of formats... From a quick check I have found also nothing in datetime that 'auto' recognizes the format, so I do not have a solution to push. Other schedulers plugins might have similar problems, I suspect...

Might be a thing for the coding week. Check how to parse in general different datetime strings. @giovannipizzi: Since you wrote it. Any suggestions?

giovannipizzi commented 6 years ago

Thanks @broeder-j for the report. First of all a note, I committed the LSF scheduler plugin but it was all work of @nmounet, who had the chance to test it only on one machine.

I see a few options to solve this (probably, in priority order according to me):

  1. ask LSF to always output in a specific format, if LSF has a way to control it. I couldn't find a reference on google on how to do it, but I know that the manpage of bjobs does not show all options, so maybe you can try out if you find a solution. We do this, e.g. with SLURM via an environment variable, see e.g. https://github.com/aiidateam/aiida_core/blob/ca9d1fde81b8038ff3cced626263d3522bb2f408/aiida/scheduler/plugins/slurm.py#L193
  2. try to extend the time parser to accept a list of formats rather than a single format, and parse them in order until a proper match is obtained (this assumes that all other formats fail, otherwise one gets a wrong time parsed)
  3. see if the dateutil package (it might be called python-dateutil) is able to automatically detect both formats (it provides an automatic detection of format)
  4. start implementing scheduler options (but this requires to extend a bit AiiDA, and makes it more complex to setup a new computer - so if it is possible to automate this with one of the approaches above, better).

If you can try if one of the above options could work (especially 1 and 2), it would be great (and in case we can discuss at the MaX Hackathon).

giovannipizzi commented 5 years ago

@broeder-j do you know if something like 1. above can be done? (i.e. if there is an ENVVAR that we can set to specify explicitly the time format?) I don't have a LSF cluster to test this.