Failing to pull Girder Tile Source - CCIPD lab project

[VarunVerma7]

Here is sample code which is not working:

import girder_large_image
import girder_client
import large_image
import json
gc = girder_client.GirderClient(apiUrl='http://localhost:8080/api/v2')
gc.authenticate(username='admin', password='password')
item = gc.getItem('6204344429b98b9caa33c7ec')
print(item)
ts = girder_large_image.girder_tilesource.GirderTileSource(item)
ts_metadata = ts.getMetadata()
print(json.dumps(ts_metadata, indent=2))
is_wsi = ts_metadata['magnification'] is not None
print("WSI ", is_wsi)
print(ts.getSingleTile())

We used this svs file: https://styx.neurology.emory.edu/girder/#item/60df5a4c68f4fe34fc7e92d2

Here is the printout and error message:

{'_id': '6204344429b98b9caa33c7ec', '_modelType': 'item', 'baseParentId': '61dd35a75ef164ec2f932d35', 'baseParentType': 'collection', 'created': '2022-02-09T21:38:12.879000+00:00', 'creatorId': '61dd3564bd467a0ab2a0244c', 'description': '', 'folderId': '6204340929b98b9caa33c7ea', 'largeImage': {'fileId': '6204344429b98b9caa33c7ed', 'sourceName': 'openslide'}, 'meta': {}, 'name': 'girder-test.svs', 'size': 134394101, 'updated': '2022-02-09T21:38:12.879000+00:00'}

Using python for large_image caching

None

{ "levels": null, "sizeX": null, "sizeY": null, "tileWidth": null, "tileHeight": null, "magnification": null, "mm_x": null, "mm_y": null }

WSI False

Traceback (most recent call last):
  File "/Users/varunverma/Desktop/playground/test.py", line 23, in <module>
    print(ts.getSingleTile())
  File "/usr/local/lib/python3.9/site-packages/large_image/tilesource/base.py", line 2241, in getSingleTile
    return next(self.tileIterator(*args, kwargs), None)
  File "/usr/local/lib/python3.9/site-packages/large_image/tilesource/base.py", line 2202, in tileIterator
    iterInfo = self._tileIteratorInfo(format=iterFormat, resample=resample,
  File "/usr/local/lib/python3.9/site-packages/large_image/tilesource/base.py", line 467, in _tileIteratorInfo
    magLevel = self.getLevelForMagnification(magArgs)
  File "/usr/local/lib/python3.9/site-packages/large_image/tilesource/base.py", line 2010, in getLevelForMagnification
    mag = self.getMagnificationForLevel()
  File "/usr/local/lib/python3.9/site-packages/large_image/tilesource/base.py", line 1982, in getMagnificationForLevel
    if mag.get('level') == self.levels - 1:

TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

[dgutman]

So can you clarify--- your using a file from STYX.. but I assume that's uploaded to your own local server?

IF that's the case, I'd first try and list the Folder the image is in, and verify the itemID. It looks like there's something weird about this image being detected as largeImage.. or it wasn't able to read the data...

On Thu, Feb 10, 2022 at 2:17 AM Varun Verma @.***> wrote:

Here is are sample code which is not working:

import girder_large_image import girder_client import large_image import json gc = girder_client.GirderClient(apiUrl='http://localhost:8080/api/v2') gc.authenticate(username='admin', password='password') item = gc.getItem('6204344429b98b9caa33c7ec') print(item) ts = girder_large_image.girder_tilesource.GirderTileSource(item) ts_metadata = ts.getMetadata() print(json.dumps(ts_metadata, indent=2)) is_wsi = ts_metadata['magnification'] is not None print("WSI ", is_wsi) print(ts.getSingleTile())

We used this svs file: https://styx.neurology.emory.edu/girder/#item/60df5a4c68f4fe34fc7e92d2

Here is the printout and error message:

{'_id': '6204344429b98b9caa33c7ec', '_modelType': 'item', 'baseParentId': '61dd35a75ef164ec2f932d35', 'baseParentType': 'collection', 'created': '2022-02-09T21:38:12.879000+00:00', 'creatorId': '61dd3564bd467a0ab2a0244c', 'description': '', 'folderId': '6204340929b98b9caa33c7ea', 'largeImage': {'fileId': '6204344429b98b9caa33c7ed', 'sourceName': 'openslide'}, 'meta': {}, 'name': 'girder-test.svs', 'size': 134394101, 'updated': '2022-02-09T21:38:12.879000+00:00'}

Using python for large_image caching

None

{ "levels": null, "sizeX": null, "sizeY": null, "tileWidth": null, "tileHeight": null, "magnification": null, "mm_x": null, "mm_y": null }

WSI False

Traceback (most recent call last): File "/Users/varunverma/Desktop/playground/test.py", line 23, in print(ts.getSingleTile()) File "/usr/local/lib/python3.9/site-packages/large_image/tilesource/base.py", line 2241, in getSingleTile return next(self.tileIterator(*args, kwargs), None) File "/usr/local/lib/python3.9/site-packages/large_image/tilesource/base.py", line 2202, in tileIterator iterInfo = self._tileIteratorInfo(format=iterFormat, resample=resample, File "/usr/local/lib/python3.9/site-packages/large_image/tilesource/base.py", line 467, in _tileIteratorInfo magLevel = self.getLevelForMagnification(magArgs) File "/usr/local/lib/python3.9/site-packages/large_image/tilesource/base.py", line 2010, in getLevelForMagnification mag = self.getMagnificationForLevel() File "/usr/local/lib/python3.9/site-packages/large_image/tilesource/base.py", line 1982, in getMagnificationForLevel if mag.get('level') == self.levels - 1:

TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

— Reply to this email directly, view it on GitHub https://github.com/girder/large_image/issues/779, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFODTUZBG3MXOY4ASFOCK3U2NRAXANCNFSM5N7ZQ7MA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- David A Gutman, M.D. Ph.D. Associate Professor of Neurology Emory University School of Medicine

[cooperlab]

Something is wrong in creating the tile source. Levels should not be None.

@VarunVerma7 what happens if you use the girder and file on the Styx server? Connect your girder client to https://styx.neurology.emory.edu/girder/api/v1 and use item 60df5a4c68f4fe34fc7e92d2. That can rule out issues with your local instance.

[VarunVerma7]

So can you clarify--- your using a file from STYX.. but I assume that's uploaded to your own local server? IF that's the case, I'd first try and list the Folder the image is in, and verify the itemID. It looks like there's something weird about this image being detected as largeImage.. or it wasn't able to read the data... -- David A Gutman, M.D. Ph.D. Associate Professor of Neurology Emory University School of Medicine

Thank you for the reply. That is correct - file from STYX uploaded to my local instance. We know the itemId is valid because

item = gc.getItem('6204344429b98b9caa33c7ec')
print(item)

prints its corresponding attributes correctly, however when large image tries to pull it as a tile source, it fails

[VarunVerma7]

When I connect to https://styx.neurology.emory.edu/girder/api/v1 and swap out the item id with 60df5a4c68f4fe34fc7e92d2 (to match this itemId in production), I get the exact same output in my console.

This leads me to believe I am just using GirderTileSource incorrectly?

[manthey]

I think the confusion here is that the girder_client isn't the same as Girder. You are using girder_client to get information about an item on the girder server, but it doesn't download the associated file. The GirderTileSource expects that the item is in the locally running version of girder, but you are using a client, not on the girder server itself.

You can download the item (and associated file) and then use the non-girder tile source (large_image.open(<file>)). Or, you could mount the remote girder instance via something like https://github.com/manthey/girder-client-mount and the open the file there, still with large_image.open, not the GirderTileSource. This avoids downloading the file and only accesses the bytes needed for whatever analysis you are doing.

[cooperlab]

I think the confusion here is that the girder_client isn't the same as Girder. You are using girder_client to get information about an item on the girder server, but it doesn't download the associated file. The GirderTileSource expects that the item is in the locally running version of girder, but you are using a client, not on the girder server itself.

You can download the item (and associated file) and then use the non-girder tile source (large_image.open(<file>)). Or, you could mount the remote girder instance via something like https://github.com/manthey/girder-client-mount and the open the file there, still with large_image.open, not the GirderTileSource. This avoids downloading the file and only accesses the bytes needed for whatever analysis you are doing.

@manthey this is supposed to be running inside a slicer_web_cli algorithm. What is the best way to use large_image in that case? We are trying to pull a low-res image of the whole slide. We have been using the token passed to the algorithm with girder client and making API calls.

[manthey]

If your algorithm specifies the image (e.g., https://github.com/DigitalSlideArchive/HistomicsTK/blob/master/histomicstk/cli/NucleiDetection/NucleiDetection.xml#L14-L20), the girder_worker module ensures that the image is local to the running job and passes a path that will access it as a command line argument (as args.inputImageFile if you are using CLIArgumentParser().parse_args()). Inside the algorithm, you would then use large_image.open(args.inputImageFile).

[VarunVerma7]

We are using the pattern as shown the NucleiDetection example and using CLIArgumentArgumentParser() , however this error is thrown on this line of code img = large_image.open(args.inputImageFile) in the snippet below

this is the corresponding code

print("INPUT FILE ", args.inputImageFile)
gc = girder_client.GirderClient(apiUrl=args.girderApiUrl)
gc.authenticate(apiKey=args.girderApiKey)
img = large_image.open(args.inputImageFile)
print(f"LARGE IMAGE INPUT FILE {img}")
descend_folder(gc, parse_dir_input(args.directory), args)

print("INPUT FILE ", args.inputImageFile) ended up printing: INPUT FILE /mnt/girder_worker/761bfb53aaf04d9b98a55644b156d8d3/CMU-1.svs

so i believe this aspect was configured correctly.

[manthey]

Are you installing any sources in your python environment? That is, did you either do pip install large_image[sources] or specify an explicit source? I'm guessing that this might occur if no sources are installed.

[VarunVerma7]

Ignore the error message I posted here... it worked thanks!

I had to add pip install large-image-source-openslide

So my next step is to use large_image.open(args.inputImageFile) to get a reduced magnification version of the inputImageFile. I should be able to figure this out now.

[dgutman]

To clarify, are you trying to read the file directly from the file system via large_image or are you trying to grab the image from a girder server, and then grab a thumb nail via that connection...

Also just to clarify-- are you running this code from WITHIN a docker container? If so, what's your reference image that you are using.

On Thu, Feb 10, 2022 at 1:12 PM Varun Verma @.***> wrote:

Thank you for the help. No I did not. So I added RUN pip install large-image-source-openslide. Now, this error occurs.

[image: Screen Shot 2022-02-10 at 1 08 28 PM] https://user-images.githubusercontent.com/16344543/153469767-5d9834a5-46d8-45a0-8230-9b2fb9f11b78.png

I am little confused to when you say "the girder_worker module ensures that the image is local to the running job". Our purpose is too pull a low-res version of the whole slide, so it does not exist currently. In that case is large_image.open(args.inputImageFile) still what we are looking for?

— Reply to this email directly, view it on GitHub https://github.com/girder/large_image/issues/779#issuecomment-1035264449, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFODTXU26Q5URPB7BBC52TU2P5Y5ANCNFSM5N7ZQ7MA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

-- David A Gutman, M.D. Ph.D. Associate Professor of Neurology Emory University School of Medicine

[VarunVerma7]

We are trying to grab a lower res version of the image from a girder server.

We are running this code from within a docker container. The reference image I'm working within can be found here https://github.com/cooperlab/histoqc_plugin , though this does not include my most recent updates but is the scaffold from which I'm developing from.

[manthey]

I think this was resolved, so closing this issue.