search

niaid / image_portal_workflows

Workflows related to project previously referred to as "Hedwig"
BSD 3-Clause "New" or "Revised" License
5 stars 1 forks source link

16 bit Tiffs are generating black thumbnails #466

Closed NetaFG closed 2 months ago

NetaFG commented 4 months ago

Hedwig URL : https://hedwig.niaid.nih.gov/sessions/U2Vzc2lvbjozMTQ0/samples#N4IgygpgzlCWD2A7AMrKAXWiDmIBcoAagKIBKYAkgPIBy+ATADQiECCyAqsfqAIYyxsiCABMw6XgDNJAMVgAbdBABO+ANogO9QgC8AxvXkA3AEYBrAFoAHEwAlS8vfIBCjxDVMAreAA8AsjoAigC8IAC6zFawcooq6hEgUBDyEHpKIgAqAJ5W0PHMSbzKegAW4spYuHggIMxGsBAA7tm5+AAMzEUQMUrKUGAl8I2I+OjKAK4QBfDKmDjivEo8iTPoAMIVvQgj1cj86BxWIosQACIntSuzp7DKqZhI+CCn0HoQiCKVIAC+v99AA== image

Please note that the 2D pipeline was updated formerly to support 16 bit files therefore these files should have A sample 16 bit files can be found in the Hedwig Teams channel: https://nih.sharepoint.com/:f:/r/sites/NIAID-NCATSCollaboration/Shared%20Documents/Hedwig%20V2/Sample%2016bit%20Files?csf=1&web=1&e=JO3RWJ

NetaFG commented 4 months ago

Pipeline run : http://prefect2.hedwig-workflow-api.niaidprod.net/flow-runs/flow-run/b5da08a8-f8bb-4916-8765-fbf75eea4cdc image

NetaFG commented 4 months ago

Please find attached a sample image showing how the thumbnails should look:

image

NetaFG commented 4 months ago

File path: : \ai-fas12.niaid.nih.gov\RMLEMHedwigProd\Projects\RTB\efischer\hansenbry-2024-0117-DMID\TEM-2024-0118-VLP-quant

blowekamp commented 4 months ago

The images come from a new microscope and with new tools which export the images to 16-bit TIFFs.

The tiff contains outlier pixel ( perhaps caused by X-rays ) for example 1632 value with a median of the image at 151. The range preserving approach of pytools.convert will not work with this.

A similar issue was previous encountered with 16-bit tiff exported from FIBSEM images. For this the 16-bit were preserved I the OME ZARR, but the intensity was windows with a MAD algorithm when viewing.

blowekamp commented 4 months ago

The command line from convert_3d_mrc_to_tiff was run on 4 sample input 16-bit tiff files. The conversion looked good. And Bryan said "those look good to me". This can address this issue.

However, it raises the question if these 16-bit TIFF need to be processed with this separate conversion, or if all can be run through the IMOD newstack command.

philipmac commented 2 months ago

468 should close this