weird 4x4 patterns in raster image due to zfp compression

[kswang1029]

The default zfp compression level is 11 and usually it works fine without noticeable defects. However, in some cases (I am guessing it is related to the distribution of NaN pixels which are common in the moment 1 and 2 images (velocity field and velocity dispersion) defects are visible such as the following example which is a moment image (velocity field).

compression=11

compression=32

@kswang1029 should perform more experiments to see if we need a refined default compression level.

[kswang1029]

@veggiesaurus Is it possible to add the raster compression level to the settings of the image viewer (or the settings of the raster render config widget? probably image viewer setting is better) and if users change it we re-request tiles from the backend and re-render? The one in the preference still serves as the global preference and the proposed new one is applied immediately so that users can see the changes accordingly.

[veggiesaurus]

Wouldn't that be a step backwards from what we're trying to accomplish by persisting some of the render settings?

Those compression artefacts are pretty ugly. I wonder what's causing them

[kswang1029]

@veggiesaurus I will see what I will find out with some experiments. Hope it is just simply changing the default value.

[kswang1029]

@veggiesaurus @low-sky I have done some tests with few different zfp compression quality indices. Please find the attachment for the results. Still a puzzle here why we see defects on "continent" (such as the example here) too.... On small "island" (imagine NaN as Ocean/Sea), it makes senses to have visible defects. The extreme case is a single pixel island. CARTA_zfp_quality_test.pdf

[kswang1029]

Wonder if this is related to the pixel value? In the test example above, the pixel values are 1500-ish.

UPDATE: I have done another test by multiplying 0.001 to the test image so that pixel values are 1-ish. The symptom remains so probably it is not due to the range of pixel values.

UPDATE2: same issue with v1.3.1 and v1.2.2

[kswang1029]

Interestingly, with v1.1 (pre-tile-based rendering), there is no such issue. So it might be something to do with the tiles.

[veggiesaurus]

Perhaps my nan substitution stuff is buggy. I will explore

On Mon, 02 Nov 2020, 10:47 Kuo-Song Wang, notifications@github.com wrote:

Interestingly, with v1.1 (pre-tile-based rendering), there is no such issue. So it might be something to do with the tiles.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CARTAvis/carta-frontend/issues/1223#issuecomment-720329112, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEQU6BDRKW6LPXBQUEKIKTSNZWZRANCNFSM4SIFBSKQ .

[veggiesaurus]

Interestingly, with v1.1 (pre-tile-based rendering), there is no such issue. So it might be something to do with the tiles.

Is it possible that we didn't use compression on the desktop version for 1.1?

[kswang1029]

with LEL there is another way to trigger this issue. With M17_SWex.fits, we can try to do 1-"M17_SWex.fits"

[kswang1029]

@crocka based on my test (https://docs.google.com/spreadsheets/d/1jpSnsMRmyuOdZMgpLjfHsNuSB5uGxen0j1ca1tjAq10/edit#gid=0), let's try to do the following for now as a fix (we can have a discussion on a better generic solution on this during the f2f meeting): if BUNIT = km/s (or its variant like km s-1, km s^-1, etc), we internally request the image compression quality to be 20 (if the adopted image compression quality is less than 20). For example, by default the image compression quality is 11, then we use 20. If users change the default to 25, for example, then we use 25. "20" should be a safe choice for most of the use cases of visualizing a velocity map.