kltpicker gpu capable does not run on cpu only node when cuda is present

[EricDeveaud]

Hello,

I tested kltpicker-1.2.8 on 3 different hardware. it was built with cupy-11.2 support on a node with gpu.

1) gpu node with cuda installed 2) cpu node without cuda 3) cpu node with cuda installed, no gpu cards

it runs fine on gpu node, see:

maestro-builder:kltpicker/1.2.8-gpu >  kltpicker -i ../../datas/kltpicker -o /opt/gensoft/tests/kltpicker/1.2.8-gpu -s 1 -p 1 -n 1

Running on 1 files.
Using GPUs 0.
/opt/gensoft/tests/kltpicker/1.2.8-gpu
Preprocessing (usually takes up to 1 minute)...

runs fine on cpu only node, see:

kltpicker -i ../../datas/kltpicker -o /local/gensoft2/tests/kltpicker/1.2.8-cpu -s 1 -p 1 -n 1

Running on 1 files.
/local/gensoft2/tests/kltpicker/1.2.8-cpu
Preprocessing (usually takes up to 1 minute)...
Preprocess finished. Picking particles...

but when I run it on a node with installed cuda and no gpu cards it fails. see:

[gensoft@223d0e9c227e kltpicker-1.2.8-cpu]$ kltpicker -h 
Traceback (most recent call last):
  File "/opt/gensoft/exe/kltpicker/1.2.8-gpu/bin/kltpicker", line 8, in <module>
    sys.exit(main())
  File "/opt/gensoft/exe/kltpicker/1.2.8-gpu/venv/lib/python3.8/site-packages/kltpicker/main.py", line 159, in main
    args = parse_args(HAS_CUPY)
  File "/opt/gensoft/exe/kltpicker/1.2.8-gpu/venv/lib/python3.8/site-packages/kltpicker/kltpicker_input.py", line 31, in parse_args
    parser.add_argument('--gpus', help='Indices of GPUs to be used. Valid indices: 0,...,%d. Enter -1 to use all available GPUS.'%(num_gpus-1), default=[-1], nargs='+', type=check_range_gpu)
NameError: name 'num_gpus' is not defined

indeed there is no graphics card installed so kltpicker_input.py fails silently here

try:
    import cupy
    num_gpus = cupy.cuda.runtime.getDeviceCount()
except:
    pass

IMHO it is legit to have cuda insatlled on system without gpu cards

I would suggest the follwoing fix that allows kltpicker to runn on all 3 conditions

just change main.py check for cupy import, to also test if there is any gpu card available. something like:

# Check if CuPy is installed AND we have gpu devices
try:
    import cupy as cp
    cp.cuda.runtime.getDeviceCount()
    HAS_CUPY = 1
except:
    HAS_CUPY = 0

whith this patch I was abble to succesfully run ltpicker on all my nodes

regards

Eric

[yoelsh]

Thank you very much for noting this. I believe I fixed the problem as you suggested. The repository https://github.com/ShkolniskyLab/kltpicker has been updated. Could you please verify that the updated version meets your expectations? If so, I will update the version on pypi. Thanks a lot! Yoel

On Tue, May 11, 2021 at 1:18 PM Eric Deveaud @.***> wrote:

Hello,

I tested kltpicker-1.2.8 on 3 different hardware. it was built with cupy-11.2 support on a node with gpu.

1. gpu node with cuda installed
2. cpu node without cuda
3. cpu node with cuda installed, no gpu cards

it runs fine on gpu node, see:

maestro-builder:kltpicker/1.2.8-gpu > kltpicker -i ../../datas/kltpicker -o /opt/gensoft/tests/kltpicker/1.2.8-gpu -s 1 -p 1 -n 1

Running on 1 files. Using GPUs 0. /opt/gensoft/tests/kltpicker/1.2.8-gpu Preprocessing (usually takes up to 1 minute)...

runs fine on cpu only node, see:

kltpicker -i ../../datas/kltpicker -o /local/gensoft2/tests/kltpicker/1.2.8-cpu -s 1 -p 1 -n 1

Running on 1 files. /local/gensoft2/tests/kltpicker/1.2.8-cpu Preprocessing (usually takes up to 1 minute)... Preprocess finished. Picking particles...

but when I run it on a node with installed cuda and no gpu cards it fails. see:

@.*** kltpicker-1.2.8-cpu]$ kltpicker -h Traceback (most recent call last): File "/opt/gensoft/exe/kltpicker/1.2.8-gpu/bin/kltpicker", line 8, in sys.exit(main()) File "/opt/gensoft/exe/kltpicker/1.2.8-gpu/venv/lib/python3.8/site-packages/kltpicker/main.py", line 159, in main args = parse_args(HAS_CUPY) File "/opt/gensoft/exe/kltpicker/1.2.8-gpu/venv/lib/python3.8/site-packages/kltpicker/kltpicker_input.py", line 31, in parse_args parser.add_argument('--gpus', help='Indices of GPUs to be used. Valid indices: 0,...,%d. Enter -1 to use all available GPUS.'%(num_gpus-1), default=[-1], nargs='+', type=check_range_gpu) NameError: name 'num_gpus' is not defined

indeed there is no graphics card installed so kltpicker_input.py fails silently here

try: import cupy num_gpus = cupy.cuda.runtime.getDeviceCount() except: pass

IMHO it is legit to have cuda insatlled on system without gpu cards

I would suggest the follwoing fix that allows kltpicker to runn on all 3 conditions

just change main.py check for cupy import, to also test if there is any gpu card available. something like:

Check if CuPy is installed AND we have gpu devices

try: import cupy as cp cp.cuda.runtime.getDeviceCount() HAS_CUPY = 1 except: HAS_CUPY = 0

whith this patch I was abble to succesfully run ltpicker on all my nodes

regards

Eric

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ShkolniskyLab/kltpicker/issues/1, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA52MT53VREE2W3HG5JWQ2DTNEABBANCNFSM44VAPKSA .

[EricDeveaud]

yes it does as I mentionned earlier.

thanks

Eric