Box and centroid video has wrong format for inverted colours

[alexarje]

When exporting with inverse colours the box video and centroid videos have the wrong format. Looks like they only take up a small part on the left of the screen:

[alexarje]

This also goes for the motion video exported when using inverted.

[aleksati]

Might have something to do with the [jit. @op absdiff] stuff we added at the very end to the [VA_motion_image] and/or [mgt.toggle.rgb2luma2absdiff] , @balintlaczko . It makes sense that these would affect the motion image, box and centroid processing.

[alexarje]

Looks more like stitching together several images with jit.glue or something like that? But it is only for the inverted video. And it looks nice in the preview.

[balintlaczko]

O-o, that doesn't look good. I have never seen it before though. Yes, it could be the last minute change, which was not tested so thoroughly, but this seems to be a problem with dims and not colors, and that change was (or at least meant to be) a simple color op. Seems like something unintended happens when we switch to inverted colors.

[balintlaczko]

So far I could not manage to reproduce this problem on Windows. Things I have tried include:

• opening the app/patch, drag&drop dance.avi, check inverted colors, and all three video export checkboxes in the menu, then hit Export. (to rule out init problems with cropping which used to be a thing, but quite a while ago)
• cropping and not cropping
• skipping and not skipping
• trimming and not trimming
  Outputs were as expected each time. Real-time preview also looked OK. Did discover however the "burn-in" of the centroid snake, so opened #28.

@alexarje, can you give some info about what you did when the issue happened?

[balintlaczko]

@AleksanderTidemann can you try reproducing this? Maybe it is an OSX thing? I couldn't manage to get it...

[aleksati]

First off, @balintlaczko , I´m experiencing that the same Bbox threshold generates very different videos when switching from inverted to normal colors in realtime. Like this:

(notice the Bbox display)

I will do some more testing with export.

[aleksati]

Different but related bug can be seen here with the bbox:

So it seems that the scaling (or the "bbox-cropping") depends on the color of the image to a considerable amount. I guess this is kind of related to issue #30 , both dealing with problems with inverted videos.

[aleksati]

My experience is that all of the exported videos correspond to the behavior in the Bbox Display, except for "movie.mp4" in the test_videos folder. This could be related to what I experience in issue #20 , that there are some videos that produce strange behavior in the app.

I tested the inverted dance.mp4 with skipping and trimming in:

• Max with h624 .mp4
• Max with JPEG .mov
• Build with h624 .mp4
• Build with JPEF .mov

In all of these, I was able to reproduce this exact same behavior: (The gif shows how the movie is in realtime, versus the exported version of the same video)

Note: the flickering in the exported video shown is much more intense in real-life but is tuned down by the framerate of the gif.

So the answer is No, I have not yet been able to reproduce what Alexander experienced but did uncover strange behavior with specific videos (f.instance dance.mp4) and Bbox processing when switching from normal to inverted colors.

[balintlaczko]

What Alexander experienced is clearly an inappropriate dim problem, the image is squeezed horizontally. Otherwise, regarding the box image, both the change in behavior due to the color flip and the flickering is normal, if we strictly look at the process behind it. None of that is a bug. If we want to get rid of the flickering I have two suggestions:

• allow the user to use the radius parameter of [cv.jit.threshold] too, not just threshold. For example in Alekander's above example, the flickering is caused by some reflection/something not being filtered out by [cv.jit.threshold] because our default radius is 40 pixels, which is quite big, actually. If the radius is around 20, it gets filtered out consistently.
• allow the user to optionally smooth the values returned by [jit.findbounds]. In the above example even if the radius is still the default 40, if I use some [slide] objects to lowpass filter the point coordinate values from [jit.findbounds], it won't be able to jump so big distances back and forth between consecutive frames.

I guess this should be an enhancement for the "more functionality" milestone, I will make an issue for that in a moment. The original problem is still a question mark.

[aleksati]

Good point! Sounds like an enhancement for the next version, yes.

[balintlaczko]

I think I finally reproduced the bug, though a bit accidentally while working on another issue. It was something about the greyscale option and changing the matrices' planecount. It was pretty consistent, I could manage to reproduce it several times in a row with the exact same results. (I think I was going from greyscale to non-greyscale and inverted -> render to do it, but not sure...) Also noticed a warning in the help file for [jit.record] that it only handles 4-plane matrices. So I added a conversion to 4-plane before the record (if needed) in [mgt.write-to-file] and the problem disappeared, I think that was it. But let's test some more on both platforms, and if we really don't see it again in a long time we can close this...

[alexarje]

Cool that you found it. I have my macbook in the office, and can test on Monday.

[aleksati]

Woo, nice catch! That makes sense actually, that there could be issues with going from 1 to more planes in an export scenario. We had similar issues in other places.