Improvements for working with tiled segmentations

A major set of improvements for working with segmentations of TILED images:

Add support for passing a total pixel matrix of a segmentation to the constructor. Previously, the user had to handle the process of creating the tiles from a total pixel matrix themselves as well as creating the plane positions that go along with this. This process is tedious, error-prone, and computationally inefficient especially with very large arrays. This PR add a tile_input_array boolean parameter. If it is True, the user passes a single frame to the constructor representing the full total pixel matrix, and highdicom performs the tiling automatically. The user may specify a tile_size, otherwise it will be copied from the source image. This process avoids unnecessarily reshaping the input array and should therefore also be considerably more efficient than reshaping to a stack of tiled frames before compressing/storing.
Add support for the TILED_FULL dimension organization type upon construction. The user can specify the dimension_organization_type as TILED_FULL and, if the requirements are met for this to be possible, the segmentation will be stored in this way. Note that I anticipate this primarily being used with the aforementioned tile_input_array option, though that is not a requirement. The previous behaviour was to use TILED_SPARSE (although this was not actually stored in the dataset) and store the frames in an order that was different to that required for TILED_FULL. Given comments in @hackermd 's existing code, I believe this was a mistake and that the intention was that frames should always have been stored in an order that matches TILED_FULL. Therefore I chose to change the order of frames for all segmentations to match that required by TILED_FULL regardless of whether the segmentation is actually stored as TILED_FULL. I suppose you could consider this a backwards incompatible change, although hopefully no-one was relying on the frame ordering in a non TILED_FULL. Also note that a clarification of the standard regarding where segment number should live in the TILED_FULL frame order has been prepared by @dclunie and I have followed this convention (segment number varies more slowly than row and column position).
Add a new method get_total_pixel_matrix to the segmentation object. This returns the segmentation array of a segmentation object as the total pixel matrix rather than a stack of frames. It works for both TILED_FULL and TILED_SPARSE (or unspecified dimension organization type). Furthermore, the user can specify a rectangular sub-region of the total pixel matrix rather than the full array. Otherwise, behaviour is similar to the existing methods for accessing the segmentation mask (the user may specify segments of interest and use the combine_segments option to have a labelmap style array returned to them). I implemented this by generalizing the underlying efficient segmentation "reconstruction" routine so most of the code is shared with the implementation of the existing methods.
In order to achieve the above there were several changes to the functions in the utils module. Previously the compute_plane_position_slide_per_frame function returned a list of PlanePositionObjects but for the tasks above it is inefficient to do this as we just want the underlying values stored in the objects. Therefore that method has been refactored to call another public function (iter_tiled_full_frame_data) that just yields the underlying data (plane positions and x/y/z offsets).
Documentation and tests for all of the above, including new sections in the Segmentation page of the user guide.

Note that currently this PR contains all of the contents of #245 as I branched from there.

@fedorov @dclunie @DanielaSchacherer @curtislisle

[like] Schacherer, Daniela reacted to your message:

From: Chris Bridge @.> Sent: Monday, September 11, 2023 10:46:21 PM To: ImagingDataCommons/highdicom @.> Cc: Schacherer, Daniela @.>; Mention @.> Subject: [ImagingDataCommons/highdicom] Improvements for working with tiled segmentations (PR #248)

A major set of improvements for working with segmentations of TILED images:

Add support for passing a total pixel matrix of a segmentation to the constructor. Previously, the user had to handle the process of creating the tiles from a total pixel matrix themselves as well as creating the plane positions that go along with this. This process is tedious, error-prone, and computationally inefficient especially with very large arrays. This PR add a tile_input_array boolean parameter. If it is True, the user passes a single frame to the constructor representing the full total pixel matrix, and highdicom performs the tiling automatically. The user may specify a tile_size, otherwise it will be copied from the source image. This process avoids unnecessarily reshaping the input array and should therefore also be considerably more efficient than reshaping to a stack of tiled frames before compressing/storing.
Add support for the TILED_FULL dimension organization type upon construction. The user can specify the dimension_organization_type as TILED_FULL and, if the requirements are met for this to be possible, the segmentation will be stored in this way. Note that I anticipate this primarily being used with the aforementioned tile_input_array option, though that is not a requirement. The previous behaviour was to use TILED_SPARSE (although this was not actually stored in the dataset) and store the frames in an order that was different to that required for TILED_FULL. Given comments in @hackermdhttps://github.com/hackermd 's existing code, I believe this was a mistake and that the intention was that frames should always have been stored in an order that matches TILED_FULL. Therefore I chose to change the order of frames for all segmentations to match that required by TILED_FULL regardless of whether the segmentation is actually stored as TILED_FULL. I suppose you could consider this a backwards incompatible change, although hopefully no-one was relying on the frame ordering in a non TILED_FULL. Also note that a clarification of the standard regarding where segment number should live in the TILED_FULL frame order has been prepared by @dcluniehttps://github.com/dclunie and I have followed this convention (segment number varies more slowly than row and column position).
Add a new method get_total_pixel_matrix to the segmentation object. This returns the segmentation array of a segmentation object as the total pixel matrix rather than a stack of frames. It works for both TILED_FULL and TILED_SPARSE (or unspecified dimension organization type). Furthermore, the user can specify a rectangular sub-region of the total pixel matrix rather than the full array. Otherwise, behaviour is similar to the existing methods for accessing the segmentation mask (the user may specify segments of interest and use the combine_segments option to have a labelmap style array returned to them). I implemented this by generalizing the underlying efficient segmentation "reconstruction" routine so most of the code is shared with the implementation of the existing methods.
In order to achieve the above there were several changes to the functions in the utils module. Previously the compute_plane_position_slide_per_frame function returned a list of PlanePositionObjects but for the tasks above it is inefficient to do this as we just want the underlying values stored in the objects. Therefore that method has been refactored to call another public function (iter_tiled_full_frame_data) that just yields the underlying data (plane positions and x/y/z offsets).
Documentation and tests for all of the above, including new sections in the Segmentation page of the user guide.

Note that currently this PR contains all of the contents of #245https://github.com/ImagingDataCommons/highdicom/pull/245 as I branched from there.

@fedorovhttps://github.com/fedorov @dcluniehttps://github.com/dclunie @DanielaSchachererhttps://github.com/DanielaSchacherer @curtislislehttps://github.com/curtislisle

You can view, comment on, or merge this pull request online at:

https://github.com/ImagingDataCommons/highdicom/pull/248

Commit Summary

378c740https://github.com/ImagingDataCommons/highdicom/pull/248/commits/378c7406979516ca8d5b344935d9f5700d4f47c9 Add multithreading option to segmentation creation
be6963bhttps://github.com/ImagingDataCommons/highdicom/pull/248/commits/be6963bc2035327320df25f54cfdbc82e72da9ef Remove array copy, seems to be unnecessary
808af48https://github.com/ImagingDataCommons/highdicom/pull/248/commits/808af48384ded81ebefb489f4964d52242e5f6c2 Add test
9934170https://github.com/ImagingDataCommons/highdicom/pull/248/commits/9934170cd7373e44cc20fd37dceec99694fc0d76 Add section to docs on multiprocessing
08a82bahttps://github.com/ImagingDataCommons/highdicom/pull/248/commits/08a82ba960bfe95ba6a48ad864b766ed7dcfe101 Retrieve one segment array at a time: simpler and slightly faster
47286ddhttps://github.com/ImagingDataCommons/highdicom/pull/248/commits/47286dd113bfca1d047708574b79b1579765b000 Fix docstring for _get_segment_array
22e9024https://github.com/ImagingDataCommons/highdicom/pull/248/commits/22e9024da2385c10e3977f88762a57c4139bd30c add ability for user to pass their own process pool
2aee263https://github.com/ImagingDataCommons/highdicom/pull/248/commits/2aee2636b2035d244b272944438073e6a5fc97bf Add basic tiled_full option
29e45bfhttps://github.com/ImagingDataCommons/highdicom/pull/248/commits/29e45bfd6334265b6640711a70024993f208927e Fixes
74d5e83https://github.com/ImagingDataCommons/highdicom/pull/248/commits/74d5e834742e7a2e55f7037444133e2f2bbb81f1 Switch ordering of tiles to actually match TILED_FULL
cde407dhttps://github.com/ImagingDataCommons/highdicom/pull/248/commits/cde407df7226acc245aca33440be890247bd6fb7 Refactor utils code into iter_tiled_full_frame_data
8f71961https://github.com/ImagingDataCommons/highdicom/pull/248/commits/8f71961b5ca3210a77f47d9443aabb46233b5437 Fix types
7bbdc26https://github.com/ImagingDataCommons/highdicom/pull/248/commits/7bbdc26527e07712445c28e4532d20b765f9f5e3 Fix tests relating to order of dimension indices
b44ff78https://github.com/ImagingDataCommons/highdicom/pull/248/commits/b44ff78a46d41ca2602f296c8fc5c4b56fa74ec3 Basic implementation of autotiling. Need to add omitting frame logic
33f635ehttps://github.com/ImagingDataCommons/highdicom/pull/248/commits/33f635e72bda051a4e4bc2ab279fa339dae61633 Add frame omission logic and tests
f15aa1ahttps://github.com/ImagingDataCommons/highdicom/pull/248/commits/f15aa1aef25a4de118b5356c3f4277daac149fcc Work in progress implementation of constructing tiled outputs
6109eedhttps://github.com/ImagingDataCommons/highdicom/pull/248/commits/6109eed35588bff4bc6566f09e070caa83b273ca Working implementation of tiled outputs, need tests
c50614fhttps://github.com/ImagingDataCommons/highdicom/pull/248/commits/c50614ff40361241f0e2ae4815c5a61a8be877a4 Add tests, fix bug with padded tiles
402371ahttps://github.com/ImagingDataCommons/highdicom/pull/248/commits/402371a0e531bdaa45c88bb07e47e0bc73ebb6a3 Add doc sections on total pixel matrices for seg creation and parsing
f1a284bhttps://github.com/ImagingDataCommons/highdicom/pull/248/commits/f1a284bcd4912a81802bf3f1b1741a7c5fd7fa09 Add tiled full test image, links in docs

File Changes

(10 fileshttps://github.com/ImagingDataCommons/highdicom/pull/248/files)

M data/test_files/seg_image_sm_control.dcmhttps://github.com/ImagingDataCommons/highdicom/pull/248/files#diff-9bbb3fa19250ffc5249ae4a52711457ccf3a185b20cb274b03f5e4399ef08304 (0)
A data/test_files/seg_image_sm_dots_tiled_full.dcmhttps://github.com/ImagingDataCommons/highdicom/pull/248/files#diff-0c837ade48010958f211304a36370bb966dd676e9263a42c513d87415f6dd641 (0)
M docs/seg.rsthttps://github.com/ImagingDataCommons/highdicom/pull/248/files#diff-666b1f941dd7c1ef6f0ca73477d5beae5b4db4bb60b2522168d3f76e47180c6d (235)
M src/highdicom/content.pyhttps://github.com/ImagingDataCommons/highdicom/pull/248/files#diff-0fecc892b84534f041a43d6edf3d85b0fe9521d6dd8d2369c26f67177e2cc4ec (39)
M src/highdicom/seg/content.pyhttps://github.com/ImagingDataCommons/highdicom/pull/248/files#diff-bcc4271467fbf497ab022cf6ef97362c53369c31d4740a666d4d8b2c5db424be (28)
M src/highdicom/seg/sop.pyhttps://github.com/ImagingDataCommons/highdicom/pull/248/files#diff-0c88a539cb61a359ba6bc714ff7222138ffb09ba6cbb3474f732dc01299eee73 (1608)
M src/highdicom/utils.pyhttps://github.com/ImagingDataCommons/highdicom/pull/248/files#diff-3f0d84cb9cd39915f25355d2093ec6f90691872a3fcb3f228147e8581ad65c2c (382)
M tests/test_ann.pyhttps://github.com/ImagingDataCommons/highdicom/pull/248/files#diff-eb9fe7ab9a93c7126ea75284b0ee4bd18f9877eaa6839cb87b7bd0c58ae710ca (5)
M tests/test_seg.pyhttps://github.com/ImagingDataCommons/highdicom/pull/248/files#diff-90f5f84d2267e72c000d904b0da496442ab565e8a6d0d02574dde73d59598782 (275)
M tests/test_utils.pyhttps://github.com/ImagingDataCommons/highdicom/pull/248/files#diff-33c13e0b177bacd2f02e29bcb8aea5b49e7ce34901fd8f41fefb65defba1bd33 (66)

Patch Links:

— Reply to this email directly, view it on GitHubhttps://github.com/ImagingDataCommons/highdicom/pull/248, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACVDZQORSA7CTDIBZGXU22LXZ6ID3ANCNFSM6AAAAAA4EU3RUY. You are receiving this because you were mentioned.Message ID: @.***>

ImagingDataCommons / highdicom

Improvements for working with tiled segmentations #248