Batch mode on Mac

[raveenit]

I have around 2,000 abstracts to mining in my project, I succeed to run my individual abstract in classify.py mode. Therefore, I want to run in batch.py. I created "dir1" folder which containing "input1.tar.gz" and "input2.tar.gz" and I found this error;

python batch.py -i /dir1 -o /outdir -r '.input[1-2].tar.gz$' -n SLURM --limit 100 -c "python /ABSPATH_TO_TEES/classify.py -i %a -o %o/%b -m GE11" Psyco not installed New local connection connection=SLURM:debug=False /bin/sh: squeue: command not found Current jobs 0, max jobs 100, submitted jobs 0

I wonder that can we use batch.py on MacBook Air? Thank you in advance for any help

[jbjorne]

The "squeue: command not found" error indicates that the SLURM 'squeue' command cannot be found, possibly indicating that there is some problem with your SLURM installation.

If you are trying to use batch.py on your local machine, you probably don't want to use SLURM, but rather a "Unix" connection (which means simply parallel processing on a single machine). To use this type of connection, please use e.g. the connection parameters "connection=Unix:jobLimit=1".

[raveenit]

Many thank for your prompt response I tried ; python batch.py -i /dir1 -o /outdir -r '.input[1-2].tar.gz$' -n Unix:jobLimit=1 -c "python /ABSPATH_TO_TEES/classify.py -i %a -o %o/%b -m GE11"

I got ;

Psyco not installed Traceback (most recent call last): File "batch.py", line 209, in debug=options.debug, limit=options.limit, loop=options.loop) File "batch.py", line 143, in batch connection = getConnection(connection) File "/Users/raveenit/Desktop/TEEs/Utils/Connection/Connection.py", line 21, in getConnection connection = Parameters.get(connection, valueListKey="connection", valueTypes={"debug":[bool]}, defaults=defaultParams) File "/Users/raveenit/Desktop/TEEs/Utils/Parameters.py", line 108, in get raise Exception("Undefined parameter: " + key + " (parameters: " + str(parameters) + ", defaults: " + str(defaults) + ")") Exception: Undefined parameter: Unix (parameters: {'Unix': True, 'jobLimit': '1'}, defaults: {'account': None, 'workdir': None, 'settings': None, 'modules': None, 'jobLimit': None, 'connection': None, 'memory': None, 'debug': False, 'cores': None, 'preamble': None, 'wallTime': None})

What I did wrong? I have to sorry in an advance for asking you too much because I don't have never had experience in phyton.

[jbjorne]

It appears that you didn't use the full parameter set for the connection option ("-n connection=Unix:jobLimit=1") in your call to batch.py. For more details on parameter sets please refer to the documentation at https://github.com/jbjorne/TEES/wiki/TEES-Overview.

[jbjorne]

My mistake, the batch process itself is included in the jobLimit, so the minimum limit is actually 2. I put some sample data (tar.gz compressed directories of txt files) at /tmp/BatchTest/ and tested the batch.py command, so at least the following type of command works:

python batch.py -i /tmp/BatchTest/ -r '.*tar\.gz$' --limit 2 -n "connection=Unix:jobLimit=2" -c "python /home/jari/TEES/classify.py -i %a -o %a -m GE11"

In your case your input directory and TEES path will of course be different.

[raveenit]

Hi Jari Many thanks, it is also my mistake too. I just know that we have to change TEES path. I have already finished my project. ^-^ Thank you for your support. Ravee