ComparativeGenomicsToolkit / cactus

Official home of genome aligner based upon notion of Cactus graphs
Other
529 stars 111 forks source link

BatchError because of LSF system #205

Open YPGG1234 opened 4 years ago

YPGG1234 commented 4 years ago

Hi , I submitted my jobs to LSF system with commad cactus --disableCaching --workDir /tmp --binariesMode local --batchSystem LSF jobStore cfg test.hal

But I got:

Root logger is at level 'INFO', 'toil' logger at level 'INFO'.
Root logger is at level 'INFO', 'toil' logger at level 'INFO'.
The workflow ID is: '7fd37406-ef95-403f-b4c1-4ae2ab6d0fe6'
No branch length for test1: setting to 1
No branch length for test2: setting to 1
Using config from path /home/cactus/lib/python2.7/site-packages/cactus/cactus_progressive_config.xml.
xmlRoot = <multi_cactus inputSequences="1.fa 2.fa">
    <tree>(Oar:1.0,Hu:1.0)Anc0;</tree>
    <cactus experiment_path="/tmp/tmpk9WIl_/progressiveAlignment/Anc0/Anc0_experiment.xml" name="Anc0" />
</multi_cactus>
Importing 2 sequences
Using the lsfbatch system
Limiting maxCores to CPU count of system (28).
Limiting maxMemory to physically available memory (270529912832).
Created the workflow directory at /tmp/toil-7fd37406-ef95-403f-b4c1-4ae2ab6d0fe6-72618347f6d5763816d1354d0000004c
Limiting maxDisk to physically available disk (176820895744).
Batch system does not support hot-deployment. The user script ModuleDescriptor(dirPath='/home/cactus/lib/python2.7/site-packages', name='cactus.progressive.cactus_progressive', fromVirtualEnv=False) will have to be present at the same location on every worker.
No user script to hot-deploy.
Written the environment for the jobs to the environment file
Caching all jobs in job store
0 jobs downloaded.
Running Toil version 3.14.0-b91dbf9bf6116879952f0a70f9a2fbbcae7e51b6.
Real-time logging disabled
(Re)building internal scheduler state
2020-04-08 00:39:14,490 - toil.toilState - INFO - (Re)building internal scheduler state
Found 1 jobs to start and 0 jobs with successors to run
2020-04-08 00:39:14,490 - toil.leader - INFO - Found 1 jobs to start and 0 jobs with successors to run
Checked batch system has no running jobs and no updated jobs
2020-04-08 00:39:14,490 - toil.leader - INFO - Checked batch system has no running jobs and no updated jobs
Starting the main loop
2020-04-08 00:39:14,491 - toil.leader - INFO - Starting the main loop
bparams: invalid option -- 'a'
Usage: bparams [-h] [-V] [-l]
Invalid command <showconf> 

Usage: lsadmin [-h] [-V] [command] [command_options] [command_args]

    where 'command' is:

    reconfig    Reconfigure jhscheduler
    ckconfig    Check configuration files
    limrestart  Restart LIM (Load Information Manager)
    limstartup  Startup LIM (Load Information Manager)
    limshutdown Shut down LIM (Load Information Manager)
    limlock     Lock LIM on local host
    limunlock   Unlock LIM on local host
    resrestart  Restart RES (Remote Execution Server)
    resstartup  Startup RES (Remote Execution Server)
    resshutdown Shut down RES (Remote Execution Server)
    reslogon    Log tasks executed by RES
    reslogoff   RES log off
    limdebug    Debug LIM
    limtime     Debug LIM timing message
    resdebug    Debug RES
    restime     Debug RES timing message
    help        Get help on commands
    ?           Get help on commands
    quit        Quit
bparams: invalid option -- 'a'
Usage: bparams [-h] [-V] [-l]
Invalid command <showconf> 

Usage: lsadmin [-h] [-V] [command] [command_options] [command_args]
......
......

When I checked my bparams, I got:

bparams -V
JH UniScheduler 3.2, Feb  2 2016
Copyright (C) 2013 jhlava foundation.
Copyright (C) 2011 openlava foundation.
Copyright (C) 2007 Platform Computing Inc.

I think the reason for this error may be that our bparams and other related software versions are too old, but I am not the root user and the server is difficult to upgrade in a short time, could you help me to solve it?

Thanks

diekhans commented 4 years ago

Look at the document ion on the current LSF, it does seem that the -a is a newer option.

You would have to modify the Toil LSF batch system to handle the older version. Support for HPC batch systems is difficult, do to all the variation in both software and configuration. We do not have the funding to support all of them.

May I suggest asking your admins if they would upgrade or find a different compute facility.

YPGG1234 notifications@github.com writes:

Hi , I submit my jobs to LSF system with commad cactus --disableCaching --workDir /tmp --binariesMode local --batchSystem LSF jobStore cfg test.hal

But I got:

Root logger is at level 'INFO', 'toil' logger at level 'INFO'.
Root logger is at level 'INFO', 'toil' logger at level 'INFO'.
The workflow ID is: '7fd37406-ef95-403f-b4c1-4ae2ab6d0fe6'
No branch length for test1: setting to 1
No branch length for test2: setting to 1
Using config from path /home/cactus/lib/python2.7/site-packages/cactus/cactus_progressive_config.xml.
xmlRoot = <multi_cactus inputSequences="1.fa 2.fa">
  <tree>(Oar:1.0,Hu:1.0)Anc0;</tree>
  <cactus experiment_path="/tmp/tmpk9WIl_/progressiveAlignment/Anc0/Anc0_experiment.xml" name="Anc0" />
</multi_cactus>
Importing 2 sequences
Using the lsfbatch system
Limiting maxCores to CPU count of system (28).
Limiting maxMemory to physically available memory (270529912832).
Created the workflow directory at /tmp/toil-7fd37406-ef95-403f-b4c1-4ae2ab6d0fe6-72618347f6d5763816d1354d0000004c
Limiting maxDisk to physically available disk (176820895744).
Batch system does not support hot-deployment. The user script ModuleDescriptor(dirPath='/home/cactus/lib/python2.7/site-packages', name='cactus.progressive.cactus_progressive', fromVirtualEnv=False) will have to be present at the same location on every worker.
No user script to hot-deploy.
Written the environment for the jobs to the environment file
Caching all jobs in job store
0 jobs downloaded.
Running Toil version 3.14.0-b91dbf9bf6116879952f0a70f9a2fbbcae7e51b6.
Real-time logging disabled
(Re)building internal scheduler state
2020-04-08 00:39:14,490 - toil.toilState - INFO - (Re)building internal scheduler state
Found 1 jobs to start and 0 jobs with successors to run
2020-04-08 00:39:14,490 - toil.leader - INFO - Found 1 jobs to start and 0 jobs with successors to run
Checked batch system has no running jobs and no updated jobs
2020-04-08 00:39:14,490 - toil.leader - INFO - Checked batch system has no running jobs and no updated jobs
Starting the main loop
2020-04-08 00:39:14,491 - toil.leader - INFO - Starting the main loop
bparams: invalid option -- 'a'
Usage: bparams [-h] [-V] [-l]
Invalid command <showconf> 

Usage: lsadmin [-h] [-V] [command] [command_options] [command_args]

    where 'command' is:

    reconfig    Reconfigure jhscheduler
    ckconfig    Check configuration files
    limrestart  Restart LIM (Load Information Manager)
    limstartup  Startup LIM (Load Information Manager)
    limshutdown Shut down LIM (Load Information Manager)
    limlock     Lock LIM on local host
    limunlock   Unlock LIM on local host
    resrestart  Restart RES (Remote Execution Server)
    resstartup  Startup RES (Remote Execution Server)
    resshutdown Shut down RES (Remote Execution Server)
    reslogon    Log tasks executed by RES
    reslogoff   RES log off
    limdebug    Debug LIM
    limtime     Debug LIM timing message
    resdebug    Debug RES
    restime     Debug RES timing message
    help        Get help on commands
    ?           Get help on commands
    quit        Quit
bparams: invalid option -- 'a'
Usage: bparams [-h] [-V] [-l]
Invalid command <showconf> 

Usage: lsadmin [-h] [-V] [command] [command_options] [command_args]
......
......

When I check my bparams, I got:

bparams -V
JH UniScheduler 3.2, Feb  2 2016
Copyright (C) 2013 jhlava foundation.
Copyright (C) 2011 openlava foundation.
Copyright (C) 2007 Platform Computing Inc.

I think the reason for this error may be that our bparams and other related software versions are too old, but I am not the root user and the server is difficult to upgrade in a short time, could you help me to solve it?

Thanks

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/ComparativeGenomicsToolkit/cactus/issues/205 Hi , I submit my jobs to LSF system with commad cactus --disableCaching --workDir /tmp --binariesMode local --batchSystem LSF jobStore cfg test.hal

But I got:

Root logger is at level 'INFO', 'toil' logger at level 'INFO'. Root logger is at level 'INFO', 'toil' logger at level 'INFO'. The workflow ID is: '7fd37406-ef95-403f-b4c1-4ae2ab6d0fe6' No branch length for test1: setting to 1 No branch length for test2: setting to 1 Using config from path /home/cactus/lib/python2.7/site-packages/cactus/cactus_progressive_config.xml. xmlRoot =

(Oar:1.0,Hu:1.0)Anc0;
    <cactus experiment_path="/tmp/tmpk9WIl_/progressiveAlignment/Anc0/Anc0_experiment.xml" name="Anc0" />

Importing 2 sequences Using the lsfbatch system Limiting maxCores to CPU count of system (28). Limiting maxMemory to physically available memory (270529912832). Created the workflow directory at /tmp/toil-7fd37406-ef95-403f-b4c1-4ae2ab6d0fe6-72618347f6d5763816d1354d0000004c Limiting maxDisk to physically available disk (176820895744). Batch system does not support hot-deployment. The user script ModuleDescriptor(dirPath='/home/cactus/lib/python2.7/site-packages', name='cactus.progressive.cactus_progressive', fromVirtualEnv=False) will have to be present at the same location on every worker. No user script to hot-deploy. Written the environment for the jobs to the environment file Caching all jobs in job store 0 jobs downloaded. Running Toil version 3.14.0-b91dbf9bf6116879952f0a70f9a2fbbcae7e51b6. Real-time logging disabled (Re)building internal scheduler state 2020-04-08 00:39:14,490 - toil.toilState - INFO - (Re)building internal scheduler state Found 1 jobs to start and 0 jobs with successors to run 2020-04-08 00:39:14,490 - toil.leader - INFO - Found 1 jobs to start and 0 jobs with successors to run Checked batch system has no running jobs and no updated jobs 2020-04-08 00:39:14,490 - toil.leader - INFO - Checked batch system has no running jobs and no updated jobs Starting the main loop 2020-04-08 00:39:14,491 - toil.leader - INFO - Starting the main loop bparams: invalid option -- 'a' Usage: bparams [-h] [-V] [-l] Invalid command

Usage: lsadmin [-h] [-V] [command] [command_options] [command_args]

where 'command' is:

reconfig    Reconfigure jhscheduler
ckconfig    Check configuration files
limrestart  Restart LIM (Load Information Manager)
limstartup  Startup LIM (Load Information Manager)
limshutdown Shut down LIM (Load Information Manager)
limlock     Lock LIM on local host
limunlock   Unlock LIM on local host
resrestart  Restart RES (Remote Execution Server)
resstartup  Startup RES (Remote Execution Server)
resshutdown Shut down RES (Remote Execution Server)
reslogon    Log tasks executed by RES
reslogoff   RES log off
limdebug    Debug LIM
limtime     Debug LIM timing message
resdebug    Debug RES
restime     Debug RES timing message
help        Get help on commands
?           Get help on commands
quit        Quit

bparams: invalid option -- 'a' Usage: bparams [-h] [-V] [-l] Invalid command

Usage: lsadmin [-h] [-V] [command] [command_options] [command_args] ...... ......

When I check my bparams, I got:

bparams -V JH UniScheduler 3.2, Feb 2 2016 Copyright (C) 2013 jhlava foundation. Copyright (C) 2011 openlava foundation. Copyright (C) 2007 Platform Computing Inc.

I think the reason for this error may be that our bparams and other related software versions are too old, but I am not the root user and the server is difficult to upgrade in a short time, could you help me to solve it?

Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.*