SignedDifference filter

[GoogleCodeExporter]

See http://www.aforgenet.com/forum/viewtopic.php?f=2&t=1785&p=4267#p4267

SignedDifference.cs and AddDifference.cs were originally created with svn copy, 
but svn diff was unhelpful, so I just attached the whole files instead. The two 
images are the results of the sample code listed in the XML-doc comments. All 
four files in the .zip will place themselves in the correct locations if you 
unzip into /trunk/. Obviously, though, they won't svn add themselves.

Original issue reported on code.google.com by dales...@gmail.com on 6 Nov 2010 at 4:06

Attachments:

• SignedDifference.patch
• Sources.zip

[GoogleCodeExporter]

Yes, I also thought about such type of image processing routine. It could be 
useful in certain cases.

However, there is an issue with it and current implementation. I think you know 
about it (however don’t see a coment/doc about it). What will be result of 
SignedDifference if two images are provided – black and white? OK, it is a 
rare case. But a case when one image has black pixel and another image has 
white pixel in the same location is quite often. Signed difference will provide 
either -255 or 255. Adding 128 will result in -127 or 383, where both don’t 
fit into [0, 255] range. So with this truncation the filter does not have any 
point in the case if absolute difference is greater than 127.

Any comments? ;)

Original comment by andrew.k...@gmail.com on 9 Nov 2010 at 3:43

• Changed state: Pending

[GoogleCodeExporter]

> However, there is an issue with it and current implementation. I think you 
know
> about it (however don’t see a coment/doc about it). What will be result of
> SignedDifference if two images are provided – black and white?

An image that, when passed correctly to AddDifference, will return a solid gray 
(128) image.
I did give this a passing mention in the documentation for AddDifference: 
"modulo any lost information due to clamping", but you are right, there should 
be a more prominent mention of this.
The output of SignedDifference is silently clamped to (notationally) -128..127, 
and the output of AddDifference is clamped back to 0..255.

In my imagined scenario, this would primarily be used for rudimentary video 
pre-compression: each frame is sent as a difference between it and the 
preceeding frame. I recognize that the clamping, however, does provide a 
side-effect — large changes can require two frames to transmit — but this 
seems like a reasonable trade-off in most circumstances.

This does cause a potential issue on the transmission side, as now the 
compressor has to decompress the stream too, and compare each new frame to what 
the decompressor has, rather than the simpler (and more obvious) operation of 
comparing to each frame to the previous one.

The plus side of this procedure is that selective compression of the 
transmitted frames could be used to maintain framerate. If a frame is the same 
as the previous, it can transmit details that were not present in the previous 
frame, while frames with large changes can compressed lossily, more like the 
first frame of a stream, and following frames can be used to transmit the 
missing information. (In this scenario, the presence of a change is more 
significant than the content of the change.)

In the case that taking two frames to transmit the difference is deemed not 
acceptable, another post-difference pass (that should also fix the 
compressor-must-decompress issue) could be used to transmit both the difference 
and the portions of the frame where the difference filter returned -128 or 127.

Original comment by dales...@gmail.com on 9 Nov 2010 at 10:21

[GoogleCodeExporter]

Well, not sure people will start inventing their own video compression when 
there are already dedicated project for this. FFMPEG for example. So although 
there may be a use case in video compression, the documentation should be 
clarified about all the aspect.

I can imagine another use case of SignedDifference - to show amount of 
difference. Current Difference class does this, but it does not show sign of 
difference. But SignedDifference could do this.

Original comment by andrew.k...@gmail.com on 10 Nov 2010 at 10:43

[GoogleCodeExporter]

By the way. Just as idea. These both classes may work a bit differently. Will 
clarify on signed diff example. Instead of writing this
v = (int) *ptr - (int) *ovr + 128;
we may try this:
v = 128 + ( (int) *ptr - (int) *ovr ) / 2;

In this case any difference fits the [0 , 255] range. Also user clearly can see 
difference :
-   128 (gray), no diff;
-   255 (white), max positive diff;
-   0 (black), max negative diff.
It also allows to turn from black to white frame in video using only one diff 
frame.

One negative thing I see so far is that after decompression there may be always 
a difference, which equals to 1. However, I hardly doubt anyone will see it 
watching video.

However, there is another problem. If we have a video, which changes frame by 1 
on each frame (kind of effect from black to white, which takes 255 frames) and 
we use only current and previous frame for calculation of difference, then it 
will not work – difference is always 0.

But we could make a property, which could be configured by user to choose 
calculation method.

Anyway, just a though.

Original comment by andrew.k...@gmail.com on 10 Nov 2010 at 10:44

[GoogleCodeExporter]

Commenting myself :)

> However, there is another problem. If we have a video, which changes frame by
> 1 on each frame (kind of effect from black to white, which takes 255 frames)
> and we use only current and previous frame for calculation of difference,
> then it will not work – difference is always 0.
But your implementation has the same problem as you mentioned. If black frame 
changes to white and stays white, then same problem - diff between current and 
previous will not work. As you said we need to do "decompression" on server 
side also, so we know the frame client has and do next diff with it.

So, if you are up to make the suggest extension and introduce property, which 
allows user to choose what they want, then it could be nice. Plus some comments 
in docs to clarify the idea.

Original comment by andrew.k...@gmail.com on 10 Nov 2010 at 11:01

[GoogleCodeExporter]

Original comment by andrew.k...@gmail.com on 17 Nov 2010 at 3:03

• Added labels: Project-Imaging

[GoogleCodeExporter]

Does it mean that there is no further interest on this feature?

Original comment by andrew.k...@gmail.com on 8 May 2011 at 7:27

[GoogleCodeExporter]

Correct. No further interest, at least from me. The project I thought it would 
be useful for got scuttled.

Original comment by dales...@gmail.com on 11 May 2011 at 1:47