search

xenon-middleware / xenon-cli

Perform files and jobs operations with Xenon library from command line
http://nlesc.github.io/Xenon/
Apache License 2.0
2 stars 3 forks source link

GE adaptor cannot determine version #71

Open arnikz opened 4 years ago

arnikz commented 4 years ago

I've tried to retrieve job accounting info on a GE cluster using the following command:

xenon -vvv scheduler gridengine --location local:// list --identifier [jobID]

but got this error

16:58:15.798 [main] DEBUG n.e.x.a.s.ScriptingScheduler - creating sub scheduler for gridengine adaptor at local://
16:58:15.807 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - Creating JobQueueScheduler for Adaptor local with multiQThreads: 4 and pollingDelay: 1000
16:58:15.831 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Submitting job
16:58:15.835 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Created Job local-0
16:58:15.836 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Submitting job to queue unlimited
16:58:15.836 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Waiting for interactive job to start.
16:58:15.907 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: getJobStatus for job local-0
16:58:15.907 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-0
16:58:15.908 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-0
16:58:15.908 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-0
16:58:15.908 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-0
16:58:15.908 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-0
16:58:15.908 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-0
16:58:15.908 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-0
16:58:15.909 [main] DEBUG n.e.x.a.s.RemoteCommandRunner - CommandRunner took 79 ms, executable = qconf, arguments = [-sql], exitcode = 0, stdout:
all.q
imd.q
test.q

stderr:

16:58:15.911 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Submitting job
16:58:15.911 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Created Job local-1
16:58:15.911 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Submitting job to queue unlimited
16:58:15.911 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Waiting for interactive job to start.
16:58:15.965 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: getJobStatus for job local-1
16:58:15.965 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-1
16:58:15.966 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-1
16:58:15.966 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-1
16:58:15.966 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-1
16:58:15.966 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-1
16:58:15.966 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-1
16:58:15.966 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-1
16:58:15.966 [main] DEBUG n.e.x.a.s.RemoteCommandRunner - CommandRunner took 55 ms, executable = qconf, arguments = [-sq, all.q,imd.q,test.q], exitcode = 0, stdout:
qname                 all.q
hostlist              @allhosts
seq_no                0,[@n_11_20=2],[@n_21_30=3],[@n_31_40=4],[@n_41_50=5], \
                      [@n_62_70=7],[@n_71_80=8],[@n_81_90=1],[@n_51_59=6], \
                      [n0065.compute.hpc=9],[n0061.compute.hpc=0]
load_thresholds       np_load_avg=3.0
suspend_thresholds    NONE
nsuspend              1
suspend_interval      00:05:00
priority              0
min_cpu_interval      00:05:00
processors            UNDEFINED
qtype                 BATCH INTERACTIVE,[n0061.compute.hpc=INTERACTIVE]
ckpt_list             NONE
pe_list               mpi threaded,[n0061.compute.hpc=NONE]
rerun                 FALSE
slots                 12,[@cores_36=36],[@cores_12=12],[@cores_48=48], \
                      [@cores_24=24],[n0081.compute.hpc=40], \
                      [n0082.compute.hpc=40],[n0083.compute.hpc=40], \
                      [n0084.compute.hpc=40],[n0085.compute.hpc=40], \
                      [n0086.compute.hpc=40],[n0087.compute.hpc=40], \
                      [n0088.compute.hpc=20],[n0089.compute.hpc=20], \
                      [n0090.compute.hpc=20],[n0091.compute.hpc=20], \
                      [n0092.compute.hpc=20],[n0093.compute.hpc=20], \
                      [n0094.compute.hpc=40]
tmpdir                /tmp
shell                 /bin/bash
prolog                root@/opt/sge/umc_scripts/prolog.sh
epilog                root@/opt/sge/umc_scripts/epilog.sh
shell_start_mode      posix_compliant
starter_method        NONE
suspend_method        NONE
resume_method         NONE
terminate_method      NONE
notify                00:00:60
owner_list            NONE
user_lists            NONE
xuser_lists           NONE
subordinate_list      NONE
complex_values        NONE
projects              NONE
xprojects             NONE
calendar              NONE
initial_state         disabled
s_rt                  INFINITY
h_rt                  2400:00:00
s_cpu                 INFINITY
h_cpu                 INFINITY
s_fsize               INFINITY
h_fsize               INFINITY
s_data                INFINITY
h_data                INFINITY
s_stack               INFINITY
h_stack               INFINITY
s_core                INFINITY
h_core                INFINITY
s_rss                 INFINITY
h_rss                 INFINITY
s_vmem                INFINITY
h_vmem                INFINITY
qname                 imd.q
hostlist              n0088.compute.hpc n0089.compute.hpc n0090.compute.hpc \
                      n0091.compute.hpc n0092.compute.hpc n0093.compute.hpc
seq_no                0
load_thresholds       np_load_avg=3.0
suspend_thresholds    NONE
nsuspend              1
suspend_interval      00:05:00
priority              0
min_cpu_interval      00:05:00
processors            UNDEFINED
qtype                 BATCH INTERACTIVE
ckpt_list             NONE
pe_list               mpi threaded
rerun                 FALSE
slots                 20
tmpdir                /tmp
shell                 /bin/bash
prolog                root@/opt/sge/umc_scripts/prolog.sh
epilog                root@/opt/sge/umc_scripts/epilog.sh
shell_start_mode      posix_compliant
starter_method        NONE
suspend_method        NONE
resume_method         NONE
terminate_method      NONE
notify                00:00:60
owner_list            NONE
user_lists            imaging_mrstat_demonstrator
xuser_lists           NONE
subordinate_list      NONE
complex_values        NONE
projects              NONE
xprojects             NONE
calendar              NONE
initial_state         disabled
s_rt                  INFINITY
h_rt                  2400:00:00
s_cpu                 INFINITY
h_cpu                 INFINITY
s_fsize               INFINITY
h_fsize               INFINITY
s_data                INFINITY
h_data                INFINITY
s_stack               INFINITY
h_stack               INFINITY
s_core                INFINITY
h_core                INFINITY
s_rss                 INFINITY
h_rss                 INFINITY
s_vmem                INFINITY
h_vmem                INFINITY
qname                 test.q
hostlist              n0007.compute.hpc
seq_no                100
load_thresholds       np_load_avg=1.05
suspend_thresholds    NONE
nsuspend              1
suspend_interval      00:05:00
priority              0
min_cpu_interval      00:05:00
processors            UNDEFINED
qtype                 BATCH INTERACTIVE
ckpt_list             NONE
pe_list               mpi threaded,[n0008.compute.hpc=NONE]
rerun                 FALSE
slots                 12,[@cores_36=36],[@cores_12=12],[@cores_48=48], \
                      [@cores_24=24],[@cores_40=40]
tmpdir                /tmp
shell                 /bin/bash
prolog                root@/opt/sge/umc_scripts/prologtest.sh
epilog                root@/opt/sge/umc_scripts/epilog.sh
shell_start_mode      posix_compliant
starter_method        NONE
suspend_method        NONE
resume_method         NONE
terminate_method      NONE
notify                00:00:60
owner_list            NONE
user_lists            testqusers
xuser_lists           NONE
subordinate_list      NONE
complex_values        NONE
projects              NONE
xprojects             NONE
calendar              NONE
initial_state         disabled
s_rt                  INFINITY
h_rt                  2400:0:0
s_cpu                 INFINITY
h_cpu                 INFINITY
s_fsize               INFINITY
h_fsize               INFINITY
s_data                INFINITY
h_data                INFINITY
s_stack               INFINITY
h_stack               INFINITY
s_core                INFINITY
h_core                INFINITY
s_rss                 INFINITY
h_rss                 INFINITY
s_vmem                INFINITY
h_vmem                INFINITY

stderr:

16:58:15.973 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Submitting job
16:58:15.974 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Created Job local-2
16:58:15.974 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Submitting job to queue unlimited
16:58:15.974 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Waiting for interactive job to start.
16:58:16.027 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: getJobStatus for job local-2
16:58:16.027 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-2
16:58:16.027 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-2
16:58:16.027 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-2
16:58:16.027 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-2
16:58:16.027 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-2
16:58:16.027 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-2
16:58:16.027 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-2
16:58:16.027 [main] DEBUG n.e.x.a.s.RemoteCommandRunner - CommandRunner took 54 ms, executable = qconf, arguments = [-spl], exitcode = 0, stdout:
make
mpi
smp
threaded

stderr:

16:58:16.028 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Submitting job
16:58:16.028 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Created Job local-3
16:58:16.028 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Submitting job to queue unlimited
16:58:16.028 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Waiting for interactive job to start.
16:58:16.093 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: getJobStatus for job local-3
16:58:16.094 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-3
16:58:16.094 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-3
16:58:16.094 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-3
16:58:16.094 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-3
16:58:16.094 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-3
16:58:16.094 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-3
16:58:16.094 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-3
16:58:16.095 [main] DEBUG n.e.x.a.s.RemoteCommandRunner - CommandRunner took 67 ms, executable = qconf, arguments = [-sp, make, -sp, mpi, -sp, smp, -sp, threaded], exitcode = 0, stdout:
pe_name            make
slots              1
user_lists         NONE
xuser_lists        NONE
start_proc_args    NONE
stop_proc_args     NONE
allocation_rule    $round_robin
control_slaves     TRUE
job_is_first_task  FALSE
urgency_slots      min
accounting_summary TRUE
qsort_args         NONE
pe_name            mpi
slots              250
user_lists         NONE
xuser_lists        NONE
start_proc_args    NONE
stop_proc_args     NONE
allocation_rule    $round_robin
control_slaves     TRUE
job_is_first_task  FALSE
urgency_slots      min
accounting_summary FALSE
qsort_args         NONE
pe_name            smp
slots              1
user_lists         NONE
xuser_lists        NONE
start_proc_args    NONE
stop_proc_args     NONE
allocation_rule    $pe_slots
control_slaves     TRUE
job_is_first_task  TRUE
urgency_slots      min
accounting_summary TRUE
qsort_args         NONE
pe_name            threaded
slots              6000
user_lists         NONE
xuser_lists        NONE
start_proc_args    NONE
stop_proc_args     NONE
allocation_rule    $pe_slots
control_slaves     FALSE
job_is_first_task  FALSE
urgency_slots      min
accounting_summary FALSE
qsort_args         NONE

stderr:

16:58:16.127 [main] DEBUG n.e.x.a.s.gridengine.GridEngineSetup - Created setup info, queues = {test.q=QueueInfo [name=test.q, slots=12, parallelEnvironments=[mpi, threaded]], imd.q=QueueInfo [name=imd.q, slots=20, parallelEnvironments=[mpi, threaded]], all.q=QueueInfo [name=all.q, slots=12, parallelEnvironments=[mpi, threaded]]}, parallel environments = {mpi=ParallelEnvironmentInfo [name=mpi, slots=250, allocationRule=ROUND_ROBIN, ppn=0], smp=ParallelEnvironmentInfo [name=smp, slots=1, allocationRule=PE_SLOTS, ppn=0], threaded=ParallelEnvironmentInfo [name=threaded, slots=6000, allocationRule=PE_SLOTS, ppn=0], make=ParallelEnvironmentInfo [name=make, slots=1, allocationRule=ROUND_ROBIN, ppn=0]}
16:58:16.127 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Submitting job
16:58:16.127 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Created Job local-4
16:58:16.127 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Submitting job to queue unlimited
16:58:16.127 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: Waiting for interactive job to start.
16:58:16.194 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: getJobStatus for job local-4
16:58:16.195 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-4
16:58:16.195 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-4
16:58:16.195 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-4
16:58:16.195 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: findJob for job local-4
16:58:16.195 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-4
16:58:16.195 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-4
16:58:16.195 [main] DEBUG n.e.x.a.schedulers.JobQueueScheduler - local: cleanupJob for job local-4
16:58:16.196 [main] DEBUG n.e.x.a.s.RemoteCommandRunner - CommandRunner took 68 ms, executable = qstat, arguments = [-xml], exitcode = 0, stdout:
<?xml version='1.0'?>
<job_info  xmlns:xsd="http://arc.liv.ac.uk/repos/darcs/sge/source/dist/util/resources/schemas/qstat/qstat.xsd">
  <queue_info>
  </queue_info>
  <job_info>
  </job_info>
</job_info>

stderr:

gridengine adaptor: cannot determine version, version attribute found: "http://arc.liv.ac.uk/repos/darcs/sge/source/dist/util/resources/schemas/qstat/qstat.xsd". Use the xenon.adaptors.schedulers.gridengine.ignore.version property to ignore this error
sverhoeven commented 4 years ago

Did you try xenon scheduler gridengine --prop xenon.adaptors.schedulers.gridengine.ignore.version=true ...?

arnikz commented 4 years ago

Yes, it works!

sverhoeven commented 4 years ago

Can you try again with Xenon cli 3.0.5