search

Neural-Systems-at-UIO / nutil

https://nutil.readthedocs.io/en/latest/
GNU General Public License v3.0
12 stars 5 forks source link

Transform issue - some images fail to rotate #269

Closed sharoncy closed 2 years ago

sharoncy commented 2 years ago

Ulrike reported a possible bug. She used Transform to rotate an image series: it rotated about 90% of the images according to the parameters entered in the Nutil GUI, but around 10% did not rotate. She has provided files, so I will do some testing to figure out if this is a real Nutil bug, an error with the setup or something else.

sharoncy commented 2 years ago

I tested. The run malfunctioned. No error message but many of the thumbnails were missing and some were corrupted. I suspect it's because the images exceed the upper size limit of Nutil during the rotation process. As the images come from rat they are bigger than the mouse images we typically work with. Ulrike would really appreciate some advice on how best to proceed. Are there alternative software for batch processing? Possible to implement BigTiff support?

Uppersizelimit

Majpuc commented 2 years ago

Hi, that is indeed a limitation we currently have in Transform. Rotating big files is not trivial and exactly why we created -transform in the first time. Maybe the rotation could be done with ImageMagik. It would not be batch I guess... We can talk with @darwinjob and @leuat about this next week. We don't have a script for that right now.

darwinjob commented 2 years ago

Yep, this is about BigTIFF support. ImageMagick might be able to rotate the images but I have never tested it. My guess is that Nutil uses some sort of built-in library to read TIFFs, then the upgrade might be as simple as replacing that library with the latest one https://libtiff.gitlab.io/libtiff/ https://libtiff.gitlab.io/libtiff/specification/coverage-bigtiff.html Alternatively, if you use JPEG compression, you can try to increase the compression level making the resulting TIFFs smaller - under 4GB.

sharoncy commented 2 years ago

Thanks Dmitri. Let's discuss on Friday.

UlrikeS91 commented 2 years ago

@leuat suggested a solution offline, which works for my data. The solution was to change the compression from lzw (lossless) to jpeg (lossy). It creates smaller images and small enough images for my data so that Nutil Transform does not run into issue 🙂 Since my data has been exported lossy anyway, there is also no loss in quality after the transformations.

I ran one series now and that works well. Looks like everything is as it should be. The next one is running now. Looks like that one will be fine, too.

Majpuc commented 2 years ago

Thanks for the update @UlrikeS91. Do you mean that you have pre-processed your images before using Nutil Transform? Which SW or script did you use then ?

UlrikeS91 commented 2 years ago

No, I did not do any pre-processing. Just change the compression in the Nutil transform from lzw to jpeg.

image

lzw is lossless compression, but since my original images were exported lossy, there is no point in using the lossless compression for the output images. This just makes the images artificially "lossless" causing the storage size to go up significantly without actually improving the image quality. Meaning lossy images cannot be turned into actual lossless images this way.

Majpuc commented 2 years ago

Very good! Thanks for the precision. If you are satisfied, maybe we can close this ticket?

Majpuc commented 2 years ago

@sharoncy maybe a good item for the FAQ section in the RtDocs?

sharoncy commented 2 years ago

Yes I'll add this to the RtDocs

sharoncy commented 2 years ago

Info has been added to the RtDocs: to the "transform" section and to the "warning and error messages". Issue closed 26.09.22.