search

Project-MONAI / MONAILabel

MONAI Label is an intelligent open source image labeling and learning tool.
https://docs.monai.io/projects/label
Apache License 2.0
565 stars 185 forks source link

OHIF deepgrow model gives Nonetype foreground interactions #1654

Open Douwe-Spaanderman opened 3 months ago

Douwe-Spaanderman commented 3 months ago

Describe the bug When using monailabel with OHIF I receive Nonetypes in my interactions on the z-axis. This happens specifically after correctly segmenting a sequence and moving to the next sequence in the same patient. Note, clearing points/interactions, making sure to remove previous segmentation, etc. did not seem to fix the issue. However, was able to fix the issue by refreshing the patient. Bug occurs independent of sample, i.e. if I refresh the patient I am able to segment the specific sequence.

Server logs 'foreground': [[114.18579234972675, 212.28415300546447, 10], [154.40437158469945, 174.1639344262295, 10], [171.54098360655735, 221.37704918032787, 10], [150.55737704918033, 225.224043715847, 13], [120.48087431693988, 162.62295081967213, 13], [178.88524590163934, 159.82513661202185, 13], [109.57377049180327, 179.0601092896175, None], [154.68852459016392, 178.36065573770492, None], [137.55191256830602, 202.49180327868854, None], [120.76502732240435, 176.2622950819672, None], [154.68852459016392, 187.80327868852459, None], [129.1584699453552, 207.3879781420765, None]], 'background': [], 'result_extension': '.nrrd', 'result_dtype': 'uint16', 'result_compress': False, 'description': 'Volumetric Interactivenet'} [2024-03-15 13:24:41,491] [1] [MainThread] [INFO] (monailabel.interfaces.utils.transform:76) - PRE - Run Transform(s) [2024-03-15 13:24:41,491] [1] [MainThread] [INFO] (monailabel.interfaces.utils.transform:77) - PRE - Input Keys: ['device', 'model', 'image', 'foreground', 'background', 'result_extension', 'result_dtype', 'result_compress', 'description', 'image_path'] [2024-03-15 13:24:41,520] [1] [MainThread] [INFO] (monailabel.interfaces.utils.transform:122) - PRE - Transform (LoadImaged): Time: 0.0278; image: torch.Size([320, 320, 15])(torch.float32) [2024-03-15 13:24:41,521] [1] [MainThread] [ERROR] (uvicorn.error:434) - Exception in ASGI application Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 429, in run_asgi result = await app( # type: ignore[func-returns-value] File "/usr/local/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in call return await self.app(scope, receive, send) File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 276, in call await super().call(scope, receive, send) File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 122, in call await self.middleware_stack(scope, receive, send) File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in call raise exc File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in call await self.app(scope, receive, _send) File "/usr/local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 92, in call await self.simple_response(scope, receive, send, request_headers=headers) File "/usr/local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 147, in simple_response await self.app(scope, receive, send) File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 79, in call raise exc File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 68, in call await self.app(scope, receive, sender) File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in call raise e File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call await self.app(scope, receive, send) File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 718, in call await route.handle(scope, receive, send) File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 276, in handle await self.app(scope, receive, send) File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 66, in app response = await func(request) File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 237, in app raw_response = await run_endpoint_function( File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 163, in run_endpoint_function return await dependant.call(**values) File "/usr/local/lib/python3.9/site-packages/monailabel/endpoints/infer.py", line 180, in api_run_inference return run_inference(background_tasks, model, image, session_id, params, file, label, output) File "/usr/local/lib/python3.9/site-packages/monailabel/endpoints/infer.py", line 162, in run_inference result = instance.infer(request) File "/usr/local/lib/python3.9/site-packages/monailabel/interfaces/app.py", line 307, in infer result_file_name, result_json = task(request) File "/usr/local/lib/python3.9/site-packages/monailabel/tasks/infer/basic_infer.py", line 295, in call data = self.run_pre_transforms(data, pre_transforms) File "/usr/local/lib/python3.9/site-packages/monailabel/tasks/infer/basic_infer.py", line 386, in run_pre_transforms return run_transforms(data, transforms, log_prefix="PRE", use_compose=False) File "/usr/local/lib/python3.9/site-packages/monailabel/interfaces/utils/transform.py", line 106, in run_transforms data = t(data) File "/usr/local/lib/python3.9/site-packages/monai/apps/deepgrow/transforms.py", line 571, in call clicks = list(np.array(clicks, dtype=int)) TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

To Reproduce

  1. Foreground interactions on sequence X of patient Z
  2. Run deepgrow model
  3. Submit label
  4. Continue to sequence Y of patient Z (Note, in same patient) 5a. (Clear points / previous segmentation map) 5b. Create new segmentation map
  5. Foreground interactions on sequence Y of patient Z
  6. Run deepgrow model
  7. BUG -> Nonetype in foreground interactions

Expected behavior Should be able to work on multiple sequences in the same patient.

Environment pytorch=1.12.1+cu113 monai==1.2.0 monailabel==0.7.0

diazandr3s commented 2 months ago

Hi @Douwe-Spaanderman,

Thanks for reporting this. Which OHIF version are you using? Unfortunately, if you're using OHIFv3, DeepGrow is not supported on this version: https://github.com/Project-MONAI/MONAILabel/releases/tag/0.8.0 If you want to use DeepGrow on OHIF, I'd suggest using the V2.

Hope this helps,

Douwe-Spaanderman commented 2 months ago

We are using OHIF V2 as we were aware that OHIFv3 for DeepGrow was not yet supported, so that can't be the issue here unfortunately...

diazandr3s commented 2 months ago

Hi @Douwe-Spaanderman,

Sorry, I missed a part of your post. I see that you can run inference when refreshing the patient.

Also, checking the logs in more detail I see the None values in the foreground array:

'foreground': [[114.18579234972675, 212.28415300546447, 10], [154.40437158469945, 174.1639344262295, 10], [171.54098360655735, 221.37704918032787, 10], [150.55737704918033, 225.224043715847, 13], [120.48087431693988, 162.62295081967213, 13], [178.88524590163934, 159.82513661202185, 13], [109.57377049180327, 179.0601092896175, None], [154.68852459016392, 178.36065573770492, None], [137.55191256830602, 202.49180327868854, None], [120.76502732240435, 176.2622950819672, None], [154.68852459016392, 187.80327868852459, None], [129.1584699453552, 207.3879781420765, None]]

I'm not sure why this is happening. Are you using a modified version of the Radiology app?

'description': 'Volumetric Interactivenet'}