Color artefacts

[rolkar]

In this DPReview post some artefacts are evident.

http://www.dpreview.com/forums/post/56218184

[erikrk]

Looks like it could be ringing caused by bicubic interpolation. Maybe we could do bilinear instead, or would that compromise resolution too much? Remember that we are only dealing with chominance resolution. Luminance resolution is not affected by this interpolation. Any better suggestions when it comes to interpolation?

Do we have X3F for this or some other Quattro X3F with serious fringing issues?

On vie, 2015-07-31 at 04:23 -0700, Roland Karlsson wrote:

In this DPReview post some artefacts are evident.

http://www.dpreview.com/forums/post/56218184

— Reply to this email directly or view it on GitHub.

[erikrk]

Another thought, maybe resampling should not be done at the YUV level. Maybe it should be converted back before resampling and then converted to YUV again? Would that matter when it comes to colored ringing?

On vie, 2015-07-31 at 17:58 +0200, Erik Karlsson wrote:

Looks like it could be ringing caused by bicubic interpolation. Maybe we could do bilinear instead, or would that compromise resolution too much? Remember that we are only dealing with chominance resolution. Luminance resolution is not affected by this interpolation. Any better suggestions when it comes to interpolation?

Do we have X3F for this or some other Quattro X3F with serious fringing issues?

On vie, 2015-07-31 at 04:23 -0700, Roland Karlsson wrote:

In this DPReview post some artefacts are evident.

http://www.dpreview.com/forums/post/56218184

— Reply to this email directly or view it on GitHub.

[rolkar]

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then upscale by copying every pixel to 2x2 values and then use the topmost layer to modify the four pixels in all three layers.

/Roland

[erikrk]

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then upscale by copying every pixel to 2x2 values and then use the topmost layer to modify the four pixels in all three layers.

/Roland

— Reply to this email directly or view it on GitHub.

[erikrk]

I am doing some experimentation with various interpolation methods and dp2q_SDIM0003.X3F, but the differences are not very obvious. I would need an X3F file with more obvious fringing issues.

[rolkar]

Yes ... the first 5 to 20 MP upscaling is nearest neighbor. But, then you use the topmost layer to modify the result. The assumption being that all three layers have the same micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then upscale by copying every pixel to 2x2 values and then use the topmost layer to modify the four pixels in all three layers.

/Roland

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/Kalpanika/x3f/issues/67#issuecomment-126751928.

[erikrk]

Our Quattro expansion has another edge-related problem that is apparent in dp2q_SDIM0003.X3F. At the edge between the light blue sky and the dark gray building, there is a dark blue line. This is obviously a combination of the blue color of the sky and the darkness of the building. Changing the interpolation to bilinear (maybe necessary to get rid of fringing) unsurprisingly worsens this problem. Nearest neighbor makes it even worse.

SPP does not seem to have this problem at all. Neither does it have the fringing problem. Some trickery going on? Edge detection maybe?

[erikrk]

I have tested converting back from YUV to BMT before resampling too. There is no visible difference.

[erikrk]

As I mentioned, nearest neighbor does not seem to work very well at all. The problem is that if the luminance gradient is steeper than the chrominance gradient you will get color bleeding at edges. Interpolation algorithms that generate steep gradients unfortunately tend to overshoot and and ring though. You essentially have to choose between bleeding and fringing. SPP does neither. How is that possible?

On vie, 2015-07-31 at 10:44 -0700, Roland Karlsson wrote:

Yes ... the first 5 to 20 MP upscaling is nearest neighbor. But, then you use the topmost layer to modify the result. The assumption being that all three layers have the same micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then upscale by copying every pixel to 2x2 values and then use the topmost layer to modify the four pixels in all three layers.

/Roland

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/Kalpanika/x3f/issues/67#issuecomment-126751928.

— Reply to this email directly or view it on GitHub.

[rolkar]

I assume they have a pattern based algorithm. It is all about finding patterns for 4 pixels and their surrounding. It is possible to tabulate all alternatives or to find some simple rules.

In our case. It is not likely that a dark gray area has a one pixel dark blue border if the next pixel is a part of a light blue area. It is possible, but not likely. And - if that dark blue border disappears is much more agreeable than the upscaling inventing one.

Hmmmm ... this might explain why Quattro seems so sharp. All edges are generated by using some nice patterns.

/Roland

On 2015-07-31 20:11, Erik Karlsson wrote:

As I mentioned, nearest neighbor does not seem to work very well at all. The problem is that if the luminance gradient is steeper than the chrominance gradient you will get color bleeding at edges. Interpolation algorithms that generate steep gradients unfortunately tend to overshoot and and ring though. You essentially have to choose between bleeding and fringing. SPP does neither. How is that possible?

On vie, 2015-07-31 at 10:44 -0700, Roland Karlsson wrote:

Yes ... the first 5 to 20 MP upscaling is nearest neighbor. But, then you use the topmost layer to modify the result. The assumption being that all three layers have the same micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then upscale by copying every pixel to 2x2 values and then use the topmost layer to modify the four pixels in all three layers.

/Roland

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/Kalpanika/x3f/issues/67#issuecomment-126751928.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/Kalpanika/x3f/issues/67#issuecomment-126774070.

[erikrk]

In any case, our current approach of simply upsampling the chrominance and combining it with the luminance does not seem to perform very well at all at edges. I don't think this can be remedied just by interpolating better. The luminance most probably has to be taken into account when the chrominance is upsampled. If there is a sharp edge in the luminance, the edge in the chrominance should be in the same place and just as sharp.

On vie, 2015-07-31 at 11:48 -0700, Roland Karlsson wrote:

I assume they have a pattern based algorithm. It is all about finding patterns for 4 pixels and their surrounding. It is possible to tabulate all alternatives or to find some simple rules.

In our case. It is not likely that a dark gray area has a one pixel dark blue border if the next pixel is a part of a light blue area. It is possible, but not likely. And - if that dark blue border disappears is much more agreeable than the upscaling inventing one.

Hmmmm ... this might explain why Quattro seems so sharp. All edges are generated by using some nice patterns.

/Roland

On 2015-07-31 20:11, Erik Karlsson wrote:

As I mentioned, nearest neighbor does not seem to work very well at all. The problem is that if the luminance gradient is steeper than the chrominance gradient you will get color bleeding at edges. Interpolation algorithms that generate steep gradients unfortunately tend to overshoot and and ring though. You essentially have to choose between bleeding and fringing. SPP does neither. How is that possible?

On vie, 2015-07-31 at 10:44 -0700, Roland Karlsson wrote:

Yes ... the first 5 to 20 MP upscaling is nearest neighbor. But, then you use the topmost layer to modify the result. The assumption being that all three layers have the same micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then upscale by copying every pixel to 2x2 values and then use the topmost layer to modify the four pixels in all three layers.

/Roland

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub

https://github.com/Kalpanika/x3f/issues/67#issuecomment-126751928.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/Kalpanika/x3f/issues/67#issuecomment-126774070.

— Reply to this email directly or view it on GitHub.

[mmroden]

What about a variant of nearest neighbor, where the pixels just use the same information as it would if the pixels were all the same size? That is, for a given pixel, use the exact value of the layers at each point, no interpolation.

On Friday, July 31, 2015, Erik Karlsson notifications@github.com wrote:

In any case, our current approach of simply upsampling the chrominance and combining it with the luminance does not seem to perform very well at all at edges. I don't think this can be remedied just by interpolating better. The luminance most probably has to be taken into account when the chrominance is upsampled. If there is a sharp edge in the luminance, the edge in the chrominance should be in the same place and just as sharp.

On vie, 2015-07-31 at 11:48 -0700, Roland Karlsson wrote:

I assume they have a pattern based algorithm. It is all about finding patterns for 4 pixels and their surrounding. It is possible to tabulate all alternatives or to find some simple rules.

In our case. It is not likely that a dark gray area has a one pixel dark blue border if the next pixel is a part of a light blue area. It is possible, but not likely. And - if that dark blue border disappears is much more agreeable than the upscaling inventing one.

Hmmmm ... this might explain why Quattro seems so sharp. All edges are generated by using some nice patterns.

/Roland

On 2015-07-31 20:11, Erik Karlsson wrote:

As I mentioned, nearest neighbor does not seem to work very well at all. The problem is that if the luminance gradient is steeper than the chrominance gradient you will get color bleeding at edges. Interpolation algorithms that generate steep gradients unfortunately tend to overshoot and and ring though. You essentially have to choose between bleeding and fringing. SPP does neither. How is that possible?

On vie, 2015-07-31 at 10:44 -0700, Roland Karlsson wrote:

Yes ... the first 5 to 20 MP upscaling is nearest neighbor. But, then you use the topmost layer to modify the result. The assumption being that all three layers have the same micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then upscale by copying every pixel to 2x2 values and then use the topmost layer to modify the four pixels in all three layers.

/Roland

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub

https://github.com/Kalpanika/x3f/issues/67#issuecomment-126751928.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/Kalpanika/x3f/issues/67#issuecomment-126774070.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/Kalpanika/x3f/issues/67#issuecomment-126787124.

[erikrk]

I suppose you mean replicating each chroma pixel as a 2x2 square and then just adding the top layer as luminance. The problem with this is that all 4 pixels will have the same color but they may have very different luminances. This is exactly what causes the mentioned problem where a dark blue line appears between a dark gray area and a light blue area. This looks pretty ugly and it is not present in SPP output. Using bicubic interpolation reduces this problem, but it's still there to some extent, and you also get ringing.

On vie, 2015-07-31 at 12:28 -0700, mmroden wrote:

What about a variant of nearest neighbor, where the pixels just use the same information as it would if the pixels were all the same size? That is, for a given pixel, use the exact value of the layers at each point, no interpolation.

On Friday, July 31, 2015, Erik Karlsson notifications@github.com wrote:

In any case, our current approach of simply upsampling the chrominance and combining it with the luminance does not seem to perform very well at all at edges. I don't think this can be remedied just by interpolating better. The luminance most probably has to be taken into account when the chrominance is upsampled. If there is a sharp edge in the luminance, the edge in the chrominance should be in the same place and just as sharp.

On vie, 2015-07-31 at 11:48 -0700, Roland Karlsson wrote:

I assume they have a pattern based algorithm. It is all about finding patterns for 4 pixels and their surrounding. It is possible to tabulate all alternatives or to find some simple rules.

In our case. It is not likely that a dark gray area has a one pixel dark blue border if the next pixel is a part of a light blue area. It is possible, but not likely. And - if that dark blue border disappears is much more agreeable than the upscaling inventing one.

Hmmmm ... this might explain why Quattro seems so sharp. All edges are generated by using some nice patterns.

/Roland

On 2015-07-31 20:11, Erik Karlsson wrote:

As I mentioned, nearest neighbor does not seem to work very well at all. The problem is that if the luminance gradient is steeper than the chrominance gradient you will get color bleeding at edges. Interpolation algorithms that generate steep gradients unfortunately tend to overshoot and and ring though. You essentially have to choose between bleeding and fringing. SPP does neither. How is that possible?

On vie, 2015-07-31 at 10:44 -0700, Roland Karlsson wrote:

Yes ... the first 5 to 20 MP upscaling is nearest neighbor. But, then you use the topmost layer to modify the result. The assumption being that all three layers have the same micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then upscale by copying every pixel to 2x2 values and then use the topmost layer to modify the four pixels in all three layers.

/Roland

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub

https://github.com/Kalpanika/x3f/issues/67#issuecomment-126751928.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub

https://github.com/Kalpanika/x3f/issues/67#issuecomment-126774070.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/Kalpanika/x3f/issues/67#issuecomment-126787124.

— Reply to this email directly or view it on GitHub.

[mmroden]

We could experiment with wackiness-- a few different methods, then choose the median value. Or we could do an edge detection algorithm, and then a weighted average of one method vs another near strong luminance edges (something like (edge)(cubic interp) + (1-edge)(nearest neighbor)) to try to control ringing near edges, assuming ringing looks different in the different methods.

On Friday, July 31, 2015, Erik Karlsson notifications@github.com wrote:

I suppose you mean replicating each chroma pixel as a 2x2 square and then just adding the top layer as luminance. The problem with this is that all 4 pixels will have the same color but they may have very different luminances. This is exactly what causes the mentioned problem where a dark blue line appears between a dark gray area and a light blue area. This looks pretty ugly and it is not present in SPP output. Using bicubic interpolation reduces this problem, but it's still there to some extent, and you also get ringing.

On vie, 2015-07-31 at 12:28 -0700, mmroden wrote:

What about a variant of nearest neighbor, where the pixels just use the same information as it would if the pixels were all the same size? That is, for a given pixel, use the exact value of the layers at each point, no interpolation.

On Friday, July 31, 2015, Erik Karlsson <notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

In any case, our current approach of simply upsampling the chrominance and combining it with the luminance does not seem to perform very well at all at edges. I don't think this can be remedied just by interpolating better. The luminance most probably has to be taken into account when the chrominance is upsampled. If there is a sharp edge in the luminance, the edge in the chrominance should be in the same place and just as sharp.

On vie, 2015-07-31 at 11:48 -0700, Roland Karlsson wrote:

I assume they have a pattern based algorithm. It is all about finding patterns for 4 pixels and their surrounding. It is possible to tabulate all alternatives or to find some simple rules.

In our case. It is not likely that a dark gray area has a one pixel dark blue border if the next pixel is a part of a light blue area. It is possible, but not likely. And - if that dark blue border disappears is much more agreeable than the upscaling inventing one.

Hmmmm ... this might explain why Quattro seems so sharp. All edges are generated by using some nice patterns.

/Roland

On 2015-07-31 20:11, Erik Karlsson wrote:

As I mentioned, nearest neighbor does not seem to work very well at all. The problem is that if the luminance gradient is steeper than the chrominance gradient you will get color bleeding at edges. Interpolation algorithms that generate steep gradients unfortunately tend to overshoot and and ring though. You essentially have to choose between bleeding and fringing. SPP does neither. How is that possible?

On vie, 2015-07-31 at 10:44 -0700, Roland Karlsson wrote:

Yes ... the first 5 to 20 MP upscaling is nearest neighbor. But, then you use the topmost layer to modify the result. The assumption being that all three layers have the same micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then upscale by copying every pixel to 2x2 values and then use the topmost layer to modify the four pixels in all three layers.

/Roland

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub

https://github.com/Kalpanika/x3f/issues/67#issuecomment-126751928.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub

https://github.com/Kalpanika/x3f/issues/67#issuecomment-126774070.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/Kalpanika/x3f/issues/67#issuecomment-126787124.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/Kalpanika/x3f/issues/67#issuecomment-126794199.

[rolkar]

Yes, we can do some edge detection research :) Or, we can do the pattern based upscaling I was talking about. In any case, we cannot just use some simple non adaptive upscaling. This is actually kind of fun :)

[erikrk]

I flag this as important since it seems to be a general Quattro issue.