Is tractor model fitting biased?

[DriftingPig]

Obiwan simulation shows a bias in the final magnitude measurement. In this issue I am trying to identify where this issue come from and whether there is a good way to solve it. It is important to have an unbiased model with in ~0.05. Because The target selection varies a lot in such a small difference.

[DriftingPig]

For g magnitude, if we assume input flux cut is at 0.65, and extinction is 0.85, we get g magnitude of 22.79. This is way larger than ngc g magnitude lower limit cut (22.9NGC, 22.825SGC). This is the main reason of tractor bias: Most of the low flux targets get missing, and you only recover targets that originally have high flux value!

[DriftingPig]

nervermind this is my own flux cut limit...

[DriftingPig]

/global/u2/h/huikong/obiwan_codes/ipynb

[DriftingPig]

Here is an explanation: tractor solver is overall unbiased when there is only a single source in the image, even if the image is noisy. However, when sources are in blobs, other sources get fitted first, the result is that the second source fitting is not precise. And this will result from these sources having statistically lower flux.

[DriftingPig]

1st, large rhalf tend to have large scattering. 2nd, is the background negative on average?

[DriftingPig]

I tried setting all sources in an image (/global/project/projectdirs/cosmo/data/legacysurvey/dr3/tractor/133/tractor-1336p255.fits) to 0. Then I sum up the image. I get -1173. Which is 0.02 lower for a 15*15 patch. This might explain why obiwan sources are systematically fainter. How do I fix this?

[DriftingPig]

I'm gonna try this: from astropy.stats import sigma_clipped_stats mean, median, std = sigma_clipped_stats(dat, sigma=3.0)

And use the median as the background level per pixel. And see if the bias vanishes after I apply the modified background level for each image.

[DriftingPig]

Ok here is what I'm planning to do: I will make a delta_flux vs median plot for each brick. then hopefully they are linearly related. I will find the parameter and use it in the future fitting.

[DriftingPig]

a easier version: add up the pixels where a stamp get injected, normalized to a per pixel value, write this value to a text file, and cut off the tails in the file, find the mean. Use this mean as the 'offset value'.

[DriftingPig]

it sucks!!! I can never locate a good value to handle this!!!! It changes every time something is changed a little bit!!! Now I have a new idea: I can find the total flux in real image before the injection. (in dr3 tractor catalogue. ) It will be a size of 21*21 which I believe is a typical size of a ELG. Then hopefully the flux difference is linearly related with the total flux. I can do the linear correction in the output catalogue, and use the 'corrected flux' as my true output flux.

[DriftingPig]

I believe this is a good way to explain this: tractor is subtracting background with splinesky model. It first mask out galaxies and stars. However, some faint stars/galaxies remain in the background. The fitting in this area makes spline sky higher in these regions, thus background is lower in other regions. So this is truly a bias in tractor. A good way to quantify this is to measure the total flux in each stamp region.

[DriftingPig]

This plot shows that the flux difference and the background is correlated. However, the background still didn't explain the flux bias. The intercept of the fittings are negative, which means on average, zero flux sum would still make a negative bias. Details of my code is in: /global/cscratch1/sd/huikong/obiwan_Aug/repos_for_docker/obiwan_code/py/obiwan/Drones/obiwan_analysis/flux-analysis

[DriftingPig]

Now, another explanation is that simultaneous fitting of obiwan along with some other brighter sources would make obiwan source fainter. To test this I will make an image with zeros background, and simulated sources with the local psf. Then I will inject obiwan into this simuated image. If such bias still exists, it would mean that obiwan bias comes from nearby sources.

[DriftingPig]

I made a model image which is the same code used for obiwan. images are here: /global/cscratch1/sd/huikong/obiwan_Aug/repos_for_docker/obiwan_out/chunk23_test1/coadd/127/1273p255/more_rs666/legacysurvey-1273p255-image-g.fits.fz However, analysis gives me opposite conclusion:

[DriftingPig]

output flux goes higer in this analysis

[DriftingPig]

x axis is flux in grz band, y axis is input-output flux in grz band

[DriftingPig]

/global/u2/h/huikong/obiwan_codes/obiwan_Aug/obiwan_code/py/ipynb/source_viewer.ipynb

this notebook can view any source you want by specifying how much difference the source difference is.

[DriftingPig]

plots that explain sources that are significantly brighter than input. basically it's because of blending, and some of them are rendered as one source while they are actually multipole sources together.

[DriftingPig]

At this point I guess that because of the perfect background, most of the sources are fitted solely, and there are very few simultaneous fitting at this stage.

[DriftingPig]

some slightly less significant cases.

[DriftingPig]

more_rs668: noise image with ELGs. result is bascially unbiased

[DriftingPig]

more_rs666: model source & obiwan more_rs665:tractor model image & obiwan (bad!) more_rs667: model source & obiwan & noise more_rs668: obiwan & noise & zero image more_rs669:None more_rs700: tractor model image & obiwan & noise

[DriftingPig]

I think noise issume and blending issue compete with each other. When flux go higher, the blending is less severe as you have higher flux originally, the source at that sopt wouldn't give you much effect. The noise also get better and makes measured flux higher. The result of this two issue make the slope remain unknown.

[DriftingPig]

more_rs673: tractor model image & obiwan & noise2.2 more_rs674: obiwan & noise2.2 I see the same effect in 671,672, with 672 not having blending effect. Which means the bias mainly come from noise level being too high compared with obiwan.

[DriftingPig]

more_rs675: obiwan & noise*1.5-0.0003 more_rs676: obiwan & noise-0.000212

[DriftingPig]

more_rs677: obiwan & noise

[DriftingPig]

more_rs677: obiwan & noise mean flux diff is at 0.0018365685291761012. This is small enough that can be rendered as negligible.

[DriftingPig]

note that flux measurement is accurate within 0.2 in the case of no blending.

[DriftingPig]

more_rs678: obiwan & noise -0.00024 & tractor model image

[DriftingPig]

So my point is: noise increases the probability that the sources are within a blob. When they are in one blob, faint source usually get subtracted more flux than they actually are, and systematically get fainter. This effect is(might be) correlated with the noise level.

[DriftingPig]

To test my point, make a bunch if image levels from -0.0007 -- 0.0005, with an interval of 0.00005. find the bias on every image, and interpolate all the points in real image with this function, see if the calibrated points go to 0.

[DriftingPig]

I can't find a qulitative answer. Quantitatively, noise lavel varies on the image. Making things hard to get fitted.

[DriftingPig]

Coefficients: [[ 222.24795532]] intercept: [-0.05907201]

lvl mean k b value at that sopt -0.001200 -0.143611 171.694183 -0.068814 -0.274847 -0.001150 -0.131668 110.947876 -0.078247 -0.205837 -0.001100 -0.139668 150.915131 -0.065396 -0.231402 -0.001050 -0.149999 219.587189 -0.044106 -0.274672 -0.001000 -0.139870 199.471924 -0.047889 -0.247361 -0.000950 -0.140858 168.454208 -0.062804 -0.222836 -0.000900 -0.150726 244.275726 -0.040031 -0.259879 -0.000850 -0.135973 167.814423 -0.064639 -0.207282 -0.000800 -0.136712 194.909576 -0.055077 -0.211005 -0.000750 -0.124674 163.025070 -0.060954 -0.183223 -0.000700 -0.119034 305.524078 -0.005276 -0.219143 -0.000650 -0.103085 253.737686 -0.012025 -0.176954 -0.000600 -0.105170 278.157593 -0.009984 -0.176879 -0.000550 -0.108629 244.866623 -0.023324 -0.158001 -0.000500 -0.102978 217.944061 -0.034484 -0.143456 -0.000450 -0.092591 204.479019 -0.033134 -0.125150 -0.000400 -0.087876 187.499481 -0.036481 -0.111481 -0.000350 -0.071750 176.281021 -0.031435 -0.093133 -0.000300 -0.064643 167.023743 -0.026649 -0.076756 -0.000250 -0.054747 162.972565 -0.025268 -0.066011 -0.000200 -0.040052 163.450653 -0.017951 -0.050641 -0.000150 -0.028348 181.784348 -0.008276 -0.035543 -0.000100 -0.013104 199.794800 0.002470 -0.017510 -0.000050 -0.002374 190.127609 0.006364 -0.003142 0.000000 0.021579 219.245331 0.023479 0.023479 0.000050 0.030484 242.706848 0.028571 0.040706 0.000100 0.040703 226.054214 0.028201 0.050806 0.000150 0.055731 275.922028 0.030772 0.072160 0.000200 0.082742 318.852966 0.046612 0.110383 0.000250 0.083983 301.518768 0.044745 0.120125 0.000300 0.104455 343.823975 0.046302 0.149449 0.000350 0.126185 379.024445 0.049136 0.181795 0.000400 0.136149 293.003387 0.074280 0.191481 0.000450 0.145441 337.094849 0.069098 0.220791 0.000500 0.160212 303.591248 0.090373 0.242169

[DriftingPig]

The fitting is bascially in range. That's the best I can explain...

I didn't see strong correlation in noise, so the noise level should be the main contributing factor.