search

radical-collaboration / hpc-workflows

NSF16514 EarthCube Project - Award Number:1639694
5 stars 0 forks source link

Change the default name of lsf job? #122

Closed wjlei1990 closed 3 years ago

wjlei1990 commented 4 years ago

Hi Can I change the name of the lsf job showing in the queue?

Say I have two jobs in the queue:

JOBID   USER       STAT   SLOTS    QUEUE       START_TIME    FINISH_TIME   JOB_NAME                      
91487   lei        RUN    211      batch       May 11 23:45  May 12 00:45  pilot.0000                    
91467   lei        PEND      -     batch             -             -       pilot.0000

They both have the name pilot.0000 and I can't differentiate between them...

andre-merzky commented 4 years ago

This is now implemented in radical-cybertools/radical.pilot/pull/2146

mturilli commented 4 years ago

This needs to be merged and then can be tried.

mturilli commented 4 years ago

Scheduled for July release

mturilli commented 4 years ago

Released and ready to be tested.

wjlei1990 commented 3 years ago

Hi, I justed update the radical stack and it seems adding the job_name didn't do the trick...

radical-stack

  python               : 3.7.6
  pythonpath           : /sw/summit/xalt/1.2.0/site:/sw/summit/xalt/1.2.0/libexec
  virtualenv           : summit-entk

  radical.analytics    : 1.5.0
  radical.entk         : 1.5.1
  radical.gtod         : 1.5.0
  radical.pilot        : 1.5.4
  radical.saga         : 1.5.4
  radical.utils        : 1.5.4

Here is the resource description:

    res_dict = {                                                                
        'resource': 'ornl.summit',                                              
        'project': 'GEO111',                                                    
        'schema': 'local',                                                      
        'job_name': 'test-w',                                                                                                   
        'walltime': 60,                                                         
        'gpus': ngpus,                                                          
        'cpus': ncpus,                                                          
        'queue': 'batch'                                                        
    }

The job name is still pilot.0000 in the job queue.

JOBID   USER       STAT   SLOTS    QUEUE       START_TIME    FINISH_TIME   JOB_NAME                      
367323  lei        RUN    85       batch       Sep 25 12:28  Sep 25 13:28  pilot.0000
wjlei1990 commented 3 years ago

I did another test today with updated radical-stack:

radical-stack

  python               : 3.7.6
  pythonpath           : /sw/summit/xalt/1.2.0/site:/sw/summit/xalt/1.2.0/libexec
  virtualenv           : summit-entk

  radical.analytics    : 1.5.0
  radical.entk         : 1.5.1
  radical.gtod         : 1.5.0
  radical.pilot        : 1.5.5
  radical.saga         : 1.5.6
  radical.utils        : 1.5.4

It seems the naming issue still persists.

andre-merzky commented 3 years ago

Hmm, I see this works in the last release:

(ve3)  login4  merzky1  …/world-shared/merzky1/gb20-covid19   $ bjobs
JOBID   USER       STAT   SLOTS    QUEUE       START_TIME    FINISH_TIME   JOB_NAME
396375  merzky1    PEND      -     batch             -             -       pilot.0000
399226  merzky1    PEND      -     batch             -             -       pilot.0000
400986  merzky1    PEND      -     batch             -             -       hey_andre

So it seems that EnTK is not forwarding that setting to RP. I opened an EnTK ticket(radical-cybertools/radical.entk/issues/484), lets see what @lee212 says about it...

andre-merzky commented 3 years ago

@lee212 , @iparask : ping on radical-cybertools/radical.entk/issues/484

wjlei1990 commented 3 years ago

Tested on Summit. The new stack of radical tools now support user-specific job name. Thanks for this improvement!

This issue can be closed at today's meeting.

I also noticed a change in the python script. Now entk requires the string type

t1.executable = './bin/xspecfem3D'

In the old version, it asks for a list:

t1.executable = ['./bin/xspecfem3D']
andre-merzky commented 3 years ago

I also noticed a change in the python script. Now entk requires the string type

That is correct. The list type has been phased out for quite a while though. Let us know if that transition poses a problem though, and we'll extend the backward compatibility for a while.

wjlei1990 commented 3 years ago

I just updated all the radical stack. No worries. It is a very easy change on our side.