ChrisRega
commented
1 year ago Hi! sorry for my delayed answer and happy new year. So regarding your question I have several remarks or suggestions.
What is the goal :)
First of all, you should think about what you want to measure, do you want some strictly mathematical comparison or do rather want to model the perceived difference by a human. The second question we have to clarify is if your input gifs are always of the same temporal resolution (the same frames per second).
How similar is your input wrt temporal resolution
Since it only makes sense to compare frames at the same point in time, it might require you to interpolate the higher resolution one to the exact times of the lower resolution one. E.g. Gif1 has 33fps, Gif2 has 50. It makes sense to use every frame of the 33 fps video and interpolate between the 50fps version frames since interpolating between closer frames reduces the temporal interpolation error.
The comparison
One dimensional approach
Assuming we now have two image stacks with identical temporal resolution and we just want "some diff", we are probably fine using the similarity of the hybrid compare for each frame. The score can then be either just averaged (no need to absolute them here) over all frames or root-mean-squared (sqrt(sum(score*score)). Both are valid choices, see any RMS vs Average Absolute Deviation discussion. I would personally opt for RMS because it respects the gravity of extreme outliers not just their existence.
Multi dimensional approach
While the above will yield some similarity, it will not be very sensitive to anything close to perceived difference. For perception, the problem becomes multidimensional.
1.) How different are the single frames of the two images \del_x(i) = ||A[i] + B[i]|| / 2 (lets call the image-dimension x)
2.) How temporally stable are the frames inside each image \del_t(A, i (should be t)) = ||A[i] + A[i+1] +A[i-1]|| / 3 (and the temporal dimension t)
3.) How temporally stable are the differences between the frames \del_xt(i (should be t)) = (\del_x(i) + \del_x(i+1) + \del_x(i-1)) / 3
The first point was discussed already, but for the second dimension imagine the following: Both imagestacks are stable e.g. each frame is identical. Any difference will be more easy to spot for a human then say an animated explosion with a single wrong frame. I would suggest taking the hybrid-similarity between temporally adjacent frames into the model, maybe as a floating average over something distinguishable by humans e.g. 1/20s or so. In the equation above i just used 3 to clarify the idea. If the difference is higher just down-weight any difference between the two frames of the two images.
The third point is stability of the difference. If a single frame of a temporally correlated image stack is completely different it might not be noticed if short enough. The longer difference persists, the stronger the perception of the difference will be. As an example: Think of a 100 fps image with 4 frames per second being totally different but nearly equidistant in time: frame 2, 33, 66, 99. It will be harder to spot these than 4 adjacent different images because the brain just gets more time to process the differences.
That's it!
So this would be my first shot for modeling something like a perception-like difference metric between temporally equidistant image stacks. I could definitely think of implementing something like the second approach if there's interest for it :)
@cole21771 hope this helps a bit !
Kind regards!
cole21771
From what I can tell while using this crate, the comparison functions seem to just use the first frame of an animated gif for comparison, not all frames.
To get around this, I was considering trying to compare frame by frame manually or by stacking them into a "sprite sheet" of each frame.
I feel neither of these are particularly good solutions, but I'm not really certain what the best way to go about this problem is. Do you have any suggestions?