Low frequency banding in stitched volumes, likely a product of the destriping process

[galenlynch]

The stitched, destriped volumes have very pronounced low-frequency banding. I believe we are using the destriping parameters that Camilo and I found after testing for just an hour or two. This can be seen in most of the SmartSPIM processed volumes, for example in 655019. Screen shots to demonstrate:

Very pronounced low-frequency banding in stitched image

A raw tile from the same region without this banding:

[galenlynch]

IMO the raw data look better here.

[miketaormina]

I had also noticed this and assumed it was an optical effect, so thanks for bringing this up. It really had the appearance of lensing, so I thought that the hind brain with its more varied curvature was doing something funny. I hadn't looked back at the raw data about it, but you may be correct that the de-striper is to blame. It's been a while since I looked hard at raw data, I wonder if the de-striper is necessary? It can certainly make images "look better,' but the more important thing would be how algorithms like CCF registration and cell counting are impacted.

[miketaormina]

I believe we are using the destriping parameters that Camilo and I found after testing for just an hour or two.

I've been trying to figure out where these are specified in the pipeline, wondering how different they were from what the microscope manufacturer suggested, but don't think I'm seeing the correct thing in, e.g. ./processed/metadata/params/pystripe_params_Ex_488_Em_525.json ~I'm looking at one (597305) that has a sigma1 and sigma2 both set to 800, but these numbers sound way off so I don't think I'm looking in the correct place (I thought I remembered something in the 250 range and a second, smaller number)~. Looking at a more recent data set shows both as 256, so maybe I misremember the second number being smaller.

It does look, however, that there has been some variation in these over time. It might be a good idea to compile a list of affected and unaffected data and see if it correlates.

[miketaormina]

Briefly looking at the pystripe code, it seems that sigma1 == sigma2 is treated a little different from a sigma2 == 0 configuration: relevant line

[camilolaiton]

Hello @galenlynch @miketaormina, thanks for bringing this up! We will take a look at this issue.

[galenlynch]

I think this is a bug, not an enhancement.

[galenlynch]

@miketaormina I can compile a list of our brains that have this banding

[camilolaiton]

@galenlynch @miketaormina, what do you think about the new destriping? Were you able to find new issues? I am looking forward progressively improving this for you guys. This is the repo of the new destriping if you wanna take a look.

[camilolaiton]

@galenlynch @miketaormina, the new destriping algorithm seems to solve these issues for the new datasets. Feel free to reopen it or create a new one if you guys notice these issues again.