Error/bug with device

[asmagen]

Error/bug with device:

pathflow-mixmatch register_images --im1 CD20_test.png --im2 CD3_test.png --fix_rotation False --output_dir ./ --gpu_device -1 --transform_type similarity --lr 0.01 --iterations 10 --min_object_size 50000 --no_segment_analysis True --black_background True
Loading images.
Performing regitration.
Traceback (most recent call last):
  File "/usr/local/bin/pathflow-mixmatch", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/pathflow_mixmatch/cli.py", line 522, in main
    fire.Fire(Commands)
  File "/usr/local/lib/python3.7/site-packages/fire/core.py", line 138, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/usr/local/lib/python3.7/site-packages/fire/core.py", line 468, in _Fire
    target=component.__name__)
  File "/usr/local/lib/python3.7/site-packages/fire/core.py", line 672, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pathflow_mixmatch/cli.py", line 519, in register_images
    pyramid=pyramid)
  File "/usr/local/lib/python3.7/site-packages/pathflow_mixmatch/cli.py", line 448, in register_images_
    new_img=displace_image(im2,affine_register(im1, im2, gpu_device=gpu_device, lr=lr, loss_fn=loss_fn, transform_type=transform_type, iterations=iterations, opt_cm=opt_cm, sigma=sigma, order=order, pyramid=pyramid)[0],gpu_device=gpu_device) # new tri, output he as well
  File "/usr/local/lib/python3.7/site-packages/pathflow_mixmatch/cli.py", line 148, in affine_register
    transformation = transforms[transform_type](*transform_args,**transform_opts, device=('cuda:{}'.format(gpu_device) if gpu_device>=0 else 'cpu'))
TypeError: __init__() got an unexpected keyword argument 'device'

[asmagen]

Also, using a GPU accelerated instance on AWS I got the ~6k x 6k pixel image to run in about 40 minutes using the code I extracted from your package and air lab. Not sure about the accuracy yet. But isn't it much more time than what we estimated? (and this size is for 5x while I want to do 10x)

[jlevy44]

Thanks, I just pushed another update

[jlevy44]

Hmm how many iterations did that run for? That does not sound right. What transformation did you use?

[jlevy44]

The time it took to run the algo is very surprising though. I think you did not specify a GPU to use and tried to fit using CPU. Try changing gpu_device to 0.

[jlevy44]

The number of iterations is far to small to achieve a good fit

[asmagen]

I used 1000 (not 10) iterations on that medium sized image (~6k). Is that too much? I was wondering why airlab doesn't have a termination strategy rather than a fixed # iterations. Not sure how big is the impact of additional 500 iterations. How long does it take you for such image? And what resources are you allocating? Do I still need a lot of RAM (>30GB) with a GPU instance? And does it utilize multiple GPUs? The idea is to run everything in parallel on about 100 images at a time so I hope this is all scalable and the resources allocation won't be a huge bottleneck.

[jlevy44]

I just checked out some of the stats from a previous run. I was using less than 11GB RAM, fitting images of size ~3-10k by 3-10k, with run times of 1-5 minutes per image with similar number of iterations, though I think it really depends on the task how long you want to iterate for and something I still need to fully explore for histopath image registration.

You'd want to deploy each image to separate GPU instances. But not sure why you were getting those runtimes.

[jlevy44]

I would check nvidia-smi -l while running a job.

[jlevy44]

https://github.com/biomedia-mira/drop2/issues/6#issuecomment-605113334

[jlevy44]

Going to close this since we patched this bug.